# pkg-inetlab
Python helper libraries for Web/GAE/Flask, HTML manipulation and command line tools
This package combines many Python utilities I accumulated over the years and used in some of my projects.
Some are outdated or serve some narrow purpose, other not quite complete. Use at your own risk!
Below if a brief description of the nodules included
* [pkg-inetlab](#pkg-inetlab)
* [inetlab.auth](#inetlabauth)
* [inetlab.sql](#inetlabsql)
* [inetlab.gae](#inetlabgae)
* [inetlab.gae](#inetlabgae-1)
* [inetlab.cli](#inetlabcli)
* [colorterm](#colorterm)
* [genformatter](#genformatter)
* [inputnums](#inputnums)
* [inetlab.mail.xmail](#inetlabmailxmail)
* [inetlab.html](#inetlabhtml)
* [html2xml](#html2xml)
* [htmlbuilder](#htmlbuilder)
* [htmladdons](#htmladdons)
* [inputlib](#inputlib)
* [jsescape](#jsescape)
## inetlab.auth
Utilities to be used in a `flask` project to implement syndicated login (at this time, Microsoft and Google logins are supported).
There is a working usage example [available here](https://github.com/kign/url-shortener). Briefly, follow these steps:
1. Setup environment:
```python
from inetlab.auth import synauth, synlogin
synauth.setup_endpoints('home', 'user')
synlogin.setup_partners(google_client_id=os.getenv('GOOGLE_CLIENT_ID'),
microsoft_client_id=os.getenv('MS_CLIENT_ID'),
microsoft_client_secret=os.getenv('MS_CLIENT_SECRET'))
```
2. Create at least two main and three service URL endpoints
```
/home: <your home page>
/user: <landing page after authentication>
...............................
/auth: synauth.authoriz
/token: synauth.token
/logout: synauth.logout
```
3. On "log in" button click, redirect to a template ([like this](https://github.com/kign/url-shortener/blob/main/external/login.html)) with these parameters:
```python
from flask import session, url_for, render_template
state = str(uuid.uuid4())
sesson['state'] = state
render_template('home.html',
ms_auth_url=synlogin.MSLogin.build_auth_url(
authorized_url=url_for("authorized", _external=True),
state=state),
redirect_succ=url_for('home'),
google_client_id=synlogin.GLogin.CLIENT_ID)
```
## inetlab.sql
This provides a wrapper to run MySQL queries via [SQLAlchemy](https://www.sqlalchemy.org/) interface.
```python
from inetlab.sql.sqldbconn import SQLDBConnector
db = SQLDBConnector(pool, engine_url, engine_url_dbg, echo=False)
db.execute("select col_a, col_b from mytable")
for col_a, col_b in conn:
print(col_a, col_b)
```
Why wouldn't one use a python MySQL wrapper directly? Actually, SQLAlchemy is a recommended way to use Google's
[Cloud SQL](https://cloud.google.com/sql/docs), and it does seem to work best in my testing.
## inetlab.gae
This works with `inetlab.sql` to allocate connection in GAE + CloudSQL project.
```python
from inetlab.gae.gaedbconn import gae_engine_url
from inetlab.sql.sqldbconn import SQLDBConnector
# github.com/GoogleCloudPlatform/python-docs-samples/blob/master/cloud-sql/mysql/sqlalchemy/main.py
pool_config = {
# [START cloud_sql_mysql_sqlalchemy_limit]
# Pool size is the maximum number of permanent connections to keep.
"pool_size": 5,
# Temporarily exceeds the set pool_size if no connections are available.
"max_overflow": 2,
# The total number of concurrent connections for your application will be
# a total of pool_size and max_overflow.
# [END cloud_sql_mysql_sqlalchemy_limit]
# [START cloud_sql_mysql_sqlalchemy_backoff]
# SQLAlchemy automatically uses delays between failed connection attempts,
# but provides no arguments for configuration.
# [END cloud_sql_mysql_sqlalchemy_backoff]
# [START cloud_sql_mysql_sqlalchemy_timeout]
# 'pool_timeout' is the maximum number of seconds to wait when retrieving a
# new connection from the pool. After the specified amount of time, an
# exception will be thrown.
"pool_timeout": 30, # 30 seconds
# [END cloud_sql_mysql_sqlalchemy_timeout]
# [START cloud_sql_mysql_sqlalchemy_lifetime]
# 'pool_recycle' is the maximum number of seconds a connection can persist.
# Connections that live longer than the specified amount of time will be
# reestablished
"pool_recycle": 1800, # 30 minutes
# [END cloud_sql_mysql_sqlalchemy_lifetime]
}
engine_url, dbg_engine_url = gae_engine_url('my_database')
pool, sqla_session = SQLDBConnector.make_pool(engine_url, dbg_engine_url, True, **pool_config)
db = SQLDBConnector (pool)
db.set_dbg_connection_url(dbg_engine_url)
# if need to use SQLAlchemy...
myClass = sqla_session.query(MyClass).filter_by(id=id).one()
# if need to use MySQL directly...
db.execute("select col_a, col_b from mytable")
```
## inetlab.cli
Some utilities commonly used in command line Python scripts
### colorterm
This module provides (1) a convenient wrapper for `Terminal` class from [blessings](https://pypi.org/project/blessings/) module,
and (2) a `blessings`-independent utility `add_coloring_to_emit_ansi` to use in conjunction with `logging`, like this:
```python
import logging
from inetlab.cli.colorterm import add_coloring_to_emit_ansi
logging.basicConfig(format="%(asctime)s.%(msecs)03d %(filename)s:%(lineno)d %(message)s",
level=logging.DEBUG, datefmt='%H:%M:%S')
logging.StreamHandler.emit = add_coloring_to_emit_ansi(logging.StreamHandler.emit)
```
### genformatter
Output tabular-formatted text, e.g.
```python
from inetlab.cli.genformatter import GenericFormatter
out = GenericFormatter("aligned,width=30")
out.writeheader(["x", "x^2", "x^3"])
for x in range(1,10) :
out.writerow([x, x**2, x**3])
out.close ()
```
### inputnums
`input_numbers` allows one to make multiple selection from number of given choices, allowing for intervals (possible overlapping)
and "except <...>" syntax.
```python
def input_numbers(prompt, n, flat: bool, extend=None) :
"""User can input any number or ranges between 1 and n, e.g.: 1,5,8-11
It is also possible to use "except ..." syntax, e.g. "except 10, 15"
Parameters:
- flat (bool, default=False) return flat list of numbers, not list of intervals
- extend(array of strings, default=[]) provide additional list of valid entries, in addition to
o numbers and intervalis
o 'quit', 'all' or 'none' (case insensitive and could be shortened to 1-st letter)
"""
```
## inetlab.mail.xmail
General functionality for sending emails. Supports embedded images and sending via SMTP or GMail API.
```python
def send(subject, html, channel,
send_from=None,
send_to=None,
send_cc=None,
images=None,
invoke_premailer=False,
dry_run=None):
"""
:param subject: Email subject
:param html: Email content
:param channel: Email delivery channel, or save to file option (file=...)
:param send_from: Sender's email
:param send_to: Recipients' email(s). Could be array (see below) or string. If string, module pyparsing required
:param send_cc: CC email(s), comment above for send_to applies
:param images: List of embedded/attached images or other attached files
:param invoke_premailer: apply Python module premailer tp HTML
:param dry_run: Dry run (nothing will be sent if True)
:return: *Nothing*
`images` could be either of :
* List of files to attach (either images or not)
* Dictionary ID => <binary content>; this could be embedded image if (A) <binary content> is in fact an image,
AND (B) `html` content has string "cid:{ID}" embedded somewhere
"""
```
Usage example:
```python
from inetlab.mail.xmail import send
import random
send(f"Testing email using inetlab.mail, random ID: {random.randrange(10 ** 6, 10 ** 7)}",
f"""\
Hi!<br />
This is a test of <code>users.messages.send</code>.<br />
Below we embed image <b>{sample_file}</b>, check it out:<br />
<img src="cid:sample_file" /> <br />
Hope it worked!
""",
'<username>:<password>>@smtp.some.server.com:465',
send_from='my_email@example.com',
send_to='John Doe <john.doe@example.org>',
images={'sample_file': open(sample_file, 'rb').read()})
```
NOTES:
* If you want to send emails from your GMail account, you have two options: (1) using SMTP which requires you to explicitly go to [this page](https://myaccount.google.com/lesssecureapps) allow access to "less secure" apps (and then it'll periodically revert to default, so once you see your SMTP authentication failing you'll need to do it again); or (2) use official GMail API, which requires you to register your "app" with Google's [GCloud](https://console.cloud.google.com/) and to authenticate your account in browser more or less every time you'll need to use it (thus preventing any background use).
* You can specify addressees as an array `[(name1, address1),(name2, address2),...,(nameN, addressN)]` (any `name` could be `None`), or as comma-separated string `name1 <address1>, name2 <address2>,...` (name could have commans if quoted). If using later option, we'll use [pyparsing](https://pypi.org/project/pyparsing/) to parse.
## inetlab.html
Mostly outdated utilities for parsing and generating HTML.
### html2xml
My preferred tool for parsing badly formatted HTML pages by translating them to proper XML fixing
parsing issues as they occur. By no means universal or bullet-proof, but helps to quickly make a
customized parser for a specific site.
### htmlbuilder
In the old days, used this handy library to easily generate HTML tags in Python code.
No longer useful.
### htmladdons
Similarly to `htmlbuilder`, generating more advanced HTML code. No longer useful.
### inputlib
Similarly to `htmlbuilder`, generating HTML forms. No longer useful.
### jsescape
Old utility for escaping strings in JavaScript code generated in Python. No longer useful.
Raw data
{
"_id": null,
"home_page": "https://github.com/kign/pkg-inetlab",
"name": "inetlab",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Konstantin Ignatiev",
"author_email": "kostya@inet-lab.net",
"download_url": "https://files.pythonhosted.org/packages/43/62/79ef567a9472bf545442facad515ee3d5b0c0514c72e8d12b9a932ea635e/inetlab-0.1.9.tar.gz",
"platform": null,
"description": "# pkg-inetlab\nPython helper libraries for Web/GAE/Flask, HTML manipulation and command line tools\n\nThis package combines many Python utilities I accumulated over the years and used in some of my projects. \nSome are outdated or serve some narrow purpose, other not quite complete. Use at your own risk!\n\nBelow if a brief description of the nodules included\n\n* [pkg-inetlab](#pkg-inetlab)\n * [inetlab.auth](#inetlabauth)\n * [inetlab.sql](#inetlabsql)\n * [inetlab.gae](#inetlabgae)\n * [inetlab.gae](#inetlabgae-1)\n * [inetlab.cli](#inetlabcli)\n * [colorterm](#colorterm)\n * [genformatter](#genformatter)\n * [inputnums](#inputnums)\n * [inetlab.mail.xmail](#inetlabmailxmail)\n * [inetlab.html](#inetlabhtml)\n * [html2xml](#html2xml)\n * [htmlbuilder](#htmlbuilder)\n * [htmladdons](#htmladdons)\n * [inputlib](#inputlib)\n * [jsescape](#jsescape)\n\n## inetlab.auth\n\nUtilities to be used in a `flask` project to implement syndicated login (at this time, Microsoft and Google logins are supported).\n\nThere is a working usage example [available here](https://github.com/kign/url-shortener). Briefly, follow these steps:\n\n1. Setup environment:\n\n```python\nfrom inetlab.auth import synauth, synlogin\n\nsynauth.setup_endpoints('home', 'user')\nsynlogin.setup_partners(google_client_id=os.getenv('GOOGLE_CLIENT_ID'),\n microsoft_client_id=os.getenv('MS_CLIENT_ID'),\n microsoft_client_secret=os.getenv('MS_CLIENT_SECRET'))\n\n```\n\n2. Create at least two main and three service URL endpoints\n\n```\n/home: <your home page>\n/user: <landing page after authentication>\n...............................\n/auth: synauth.authoriz\n/token: synauth.token\n/logout: synauth.logout\n```\n\n3. On \"log in\" button click, redirect to a template ([like this](https://github.com/kign/url-shortener/blob/main/external/login.html)) with these parameters:\n\n```python\nfrom flask import session, url_for, render_template\n\nstate = str(uuid.uuid4())\nsesson['state'] = state\n\nrender_template('home.html',\n ms_auth_url=synlogin.MSLogin.build_auth_url(\n authorized_url=url_for(\"authorized\", _external=True),\n state=state),\n redirect_succ=url_for('home'),\n google_client_id=synlogin.GLogin.CLIENT_ID)\n```\n\n## inetlab.sql\n\nThis provides a wrapper to run MySQL queries via [SQLAlchemy](https://www.sqlalchemy.org/) interface.\n\n```python\nfrom inetlab.sql.sqldbconn import SQLDBConnector\n\ndb = SQLDBConnector(pool, engine_url, engine_url_dbg, echo=False)\n\ndb.execute(\"select col_a, col_b from mytable\")\n\nfor col_a, col_b in conn:\n print(col_a, col_b)\n```\n\nWhy wouldn't one use a python MySQL wrapper directly? Actually, SQLAlchemy is a recommended way to use Google's\n[Cloud SQL](https://cloud.google.com/sql/docs), and it does seem to work best in my testing.\n\n## inetlab.gae\n\nThis works with `inetlab.sql` to allocate connection in GAE + CloudSQL project.\n\n```python\nfrom inetlab.gae.gaedbconn import gae_engine_url\nfrom inetlab.sql.sqldbconn import SQLDBConnector\n\n# github.com/GoogleCloudPlatform/python-docs-samples/blob/master/cloud-sql/mysql/sqlalchemy/main.py\npool_config = {\n # [START cloud_sql_mysql_sqlalchemy_limit]\n # Pool size is the maximum number of permanent connections to keep.\n \"pool_size\": 5,\n # Temporarily exceeds the set pool_size if no connections are available.\n \"max_overflow\": 2,\n # The total number of concurrent connections for your application will be\n # a total of pool_size and max_overflow.\n # [END cloud_sql_mysql_sqlalchemy_limit]\n # [START cloud_sql_mysql_sqlalchemy_backoff]\n # SQLAlchemy automatically uses delays between failed connection attempts,\n # but provides no arguments for configuration.\n # [END cloud_sql_mysql_sqlalchemy_backoff]\n # [START cloud_sql_mysql_sqlalchemy_timeout]\n # 'pool_timeout' is the maximum number of seconds to wait when retrieving a\n # new connection from the pool. After the specified amount of time, an\n # exception will be thrown.\n \"pool_timeout\": 30, # 30 seconds\n # [END cloud_sql_mysql_sqlalchemy_timeout]\n # [START cloud_sql_mysql_sqlalchemy_lifetime]\n # 'pool_recycle' is the maximum number of seconds a connection can persist.\n # Connections that live longer than the specified amount of time will be\n # reestablished\n \"pool_recycle\": 1800, # 30 minutes\n # [END cloud_sql_mysql_sqlalchemy_lifetime]\n}\n\nengine_url, dbg_engine_url = gae_engine_url('my_database')\npool, sqla_session = SQLDBConnector.make_pool(engine_url, dbg_engine_url, True, **pool_config)\n\ndb = SQLDBConnector (pool)\ndb.set_dbg_connection_url(dbg_engine_url)\n\n# if need to use SQLAlchemy...\nmyClass = sqla_session.query(MyClass).filter_by(id=id).one()\n\n# if need to use MySQL directly...\ndb.execute(\"select col_a, col_b from mytable\")\n```\n\n## inetlab.cli\n\nSome utilities commonly used in command line Python scripts\n\n### colorterm\n\nThis module provides (1) a convenient wrapper for `Terminal` class from [blessings](https://pypi.org/project/blessings/) module, \nand (2) a `blessings`-independent utility `add_coloring_to_emit_ansi` to use in conjunction with `logging`, like this:\n\n```python\nimport logging\nfrom inetlab.cli.colorterm import add_coloring_to_emit_ansi\n\nlogging.basicConfig(format=\"%(asctime)s.%(msecs)03d %(filename)s:%(lineno)d %(message)s\",\n level=logging.DEBUG, datefmt='%H:%M:%S')\nlogging.StreamHandler.emit = add_coloring_to_emit_ansi(logging.StreamHandler.emit)\n```\n\n### genformatter\n\nOutput tabular-formatted text, e.g.\n\n```python\nfrom inetlab.cli.genformatter import GenericFormatter\nout = GenericFormatter(\"aligned,width=30\")\nout.writeheader([\"x\", \"x^2\", \"x^3\"])\nfor x in range(1,10) :\n out.writerow([x, x**2, x**3])\nout.close ()\n```\n\n### inputnums\n\n`input_numbers` allows one to make multiple selection from number of given choices, allowing for intervals (possible overlapping) \nand \"except <...>\" syntax.\n\n```python\ndef input_numbers(prompt, n, flat: bool, extend=None) :\n \"\"\"User can input any number or ranges between 1 and n, e.g.: 1,5,8-11\n\n It is also possible to use \"except ...\" syntax, e.g. \"except 10, 15\"\n\n Parameters:\n\n - flat (bool, default=False) return flat list of numbers, not list of intervals\n\n - extend(array of strings, default=[]) provide additional list of valid entries, in addition to\n o numbers and intervalis\n o 'quit', 'all' or 'none' (case insensitive and could be shortened to 1-st letter)\n \"\"\"\n```\n\n## inetlab.mail.xmail\n\nGeneral functionality for sending emails. Supports embedded images and sending via SMTP or GMail API.\n\n```python\ndef send(subject, html, channel,\n send_from=None,\n send_to=None,\n send_cc=None,\n images=None,\n invoke_premailer=False,\n dry_run=None):\n \"\"\"\n :param subject: Email subject\n :param html: Email content\n :param channel: Email delivery channel, or save to file option (file=...)\n :param send_from: Sender's email\n :param send_to: Recipients' email(s). Could be array (see below) or string. If string, module pyparsing required\n :param send_cc: CC email(s), comment above for send_to applies\n :param images: List of embedded/attached images or other attached files\n :param invoke_premailer: apply Python module premailer tp HTML\n :param dry_run: Dry run (nothing will be sent if True)\n :return: *Nothing*\n \n `images` could be either of :\n * List of files to attach (either images or not)\n * Dictionary ID => <binary content>; this could be embedded image if (A) <binary content> is in fact an image, \n AND (B) `html` content has string \"cid:{ID}\" embedded somewhere\n \"\"\"\n```\nUsage example:\n\n```python\nfrom inetlab.mail.xmail import send\nimport random\n\nsend(f\"Testing email using inetlab.mail, random ID: {random.randrange(10 ** 6, 10 ** 7)}\",\n f\"\"\"\\\nHi!<br />\nThis is a test of <code>users.messages.send</code>.<br />\nBelow we embed image <b>{sample_file}</b>, check it out:<br />\n<img src=\"cid:sample_file\" /> <br />\nHope it worked!\n\"\"\",\n '<username>:<password>>@smtp.some.server.com:465',\n send_from='my_email@example.com',\n send_to='John Doe <john.doe@example.org>',\n images={'sample_file': open(sample_file, 'rb').read()})\n```\n\nNOTES:\n\n * If you want to send emails from your GMail account, you have two options: (1) using SMTP which requires you to explicitly go to [this page](https://myaccount.google.com/lesssecureapps) allow access to \"less secure\" apps (and then it'll periodically revert to default, so once you see your SMTP authentication failing you'll need to do it again); or (2) use official GMail API, which requires you to register your \"app\" with Google's [GCloud](https://console.cloud.google.com/) and to authenticate your account in browser more or less every time you'll need to use it (thus preventing any background use).\n * You can specify addressees as an array `[(name1, address1),(name2, address2),...,(nameN, addressN)]` (any `name` could be `None`), or as comma-separated string `name1 <address1>, name2 <address2>,...` (name could have commans if quoted). If using later option, we'll use [pyparsing](https://pypi.org/project/pyparsing/) to parse.\n\n## inetlab.html\n\nMostly outdated utilities for parsing and generating HTML. \n\n### html2xml\n\nMy preferred tool for parsing badly formatted HTML pages by translating them to proper XML fixing \nparsing issues as they occur. By no means universal or bullet-proof, but helps to quickly make a \ncustomized parser for a specific site.\n\n### htmlbuilder\n\nIn the old days, used this handy library to easily generate HTML tags in Python code.\nNo longer useful.\n\n### htmladdons\n\nSimilarly to `htmlbuilder`, generating more advanced HTML code. No longer useful.\n\n### inputlib\n\nSimilarly to `htmlbuilder`, generating HTML forms. No longer useful.\n\n### jsescape\n\nOld utility for escaping strings in JavaScript code generated in Python. No longer useful.\n\n\n\n\n\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Python helper libraries for Web/GAE/Flask, HTML manipulation and command line tools",
"version": "0.1.9",
"project_urls": {
"Bug Tracker": "https://github.com/kign/pkg-inetlab/issues",
"Homepage": "https://github.com/kign/pkg-inetlab"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d157a4284ce58e1a73f53c775d10317318dce57d7aeec3be667ae1c2c76b0ea2",
"md5": "04fb1e27802faf871df5551d82f0531b",
"sha256": "9f763af0099370ce5c87a2b1c777c397106f9e3a675a9d8d565aee92f76973f9"
},
"downloads": -1,
"filename": "inetlab-0.1.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "04fb1e27802faf871df5551d82f0531b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 49942,
"upload_time": "2024-11-23T04:55:59",
"upload_time_iso_8601": "2024-11-23T04:55:59.124016Z",
"url": "https://files.pythonhosted.org/packages/d1/57/a4284ce58e1a73f53c775d10317318dce57d7aeec3be667ae1c2c76b0ea2/inetlab-0.1.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "436279ef567a9472bf545442facad515ee3d5b0c0514c72e8d12b9a932ea635e",
"md5": "0b929422422853567cfa0d0cab76ae72",
"sha256": "660dedf8ad0b37ce0a96e922a9ea22d221be46f2e89b9a94ecf97446c1f0d29d"
},
"downloads": -1,
"filename": "inetlab-0.1.9.tar.gz",
"has_sig": false,
"md5_digest": "0b929422422853567cfa0d0cab76ae72",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 47548,
"upload_time": "2024-11-23T04:56:01",
"upload_time_iso_8601": "2024-11-23T04:56:01.011042Z",
"url": "https://files.pythonhosted.org/packages/43/62/79ef567a9472bf545442facad515ee3d5b0c0514c72e8d12b9a932ea635e/inetlab-0.1.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-23 04:56:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kign",
"github_project": "pkg-inetlab",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "inetlab"
}
JSON
