=========
libadvian
=========
Small helpers that do not warrant their own library.
Notable helpers
---------------
- init_logging (libadvian.logging): Initializes default logger to our standard log format.
- b64_to_uuid and uuid_to_b64 (libadvian.binpackers): URL-safe base64 UUID decode/encode.
- ensure_[utf8|str] (libadvian.binpackers): For making sure you are dealing with bytes or strings.
- TaskMaster.singleton().create_task(my_task()) (libadvian.tasks): for true fire-and-forget tasks.
- chunked(iterable, size) (libadvian.iter): divide iterable into chunks.
There is more, everything is type hinted and documented, just look around.
HTTP Logging notes
------------------
Note: requires you install the extra feature "http" to use
If LOG_HTTP_TARGET_URI is set in ENV, init_logging will add BufferedHTTPMultiRecordHandler handler
for shipping logs to Vector/Logtash (or similar that can take POSTs with text body), for username and password use
LOG_HTTP_USERNAME and LOG_HTTP_PASSWORD variables.
Any extra properties set by logger.level("message", extra={"key": "value"}) will be resolved and added to
labels_json -header in the POST. If LOG_GLOBAL_LABELS_JSON is set in ENV that will be decoded and filter
added to init_logging -pipeline that will add those labels as extras to all logrecords.
You can of course initialize those handlers manually and use them as you wish. For quick and very dirty
way of adding headers to the HTTP requests muck with libadvian.logging.httpmulti.HTTP_LOGGING_CONFIG
before initializing logging::
libadvian.logging.httpmulti.HTTP_LOGGING_CONFIG["handlers"]["http"].update({"session_options": { "headers": {
"Myheader": "myval"
}}})
Making a deep copy and using logging.dictConfig() on the copy would be much cleaner though.
Docker
------
For more controlled deployments and to get rid of "works on my computer" -syndrome, we always
make sure our software works under docker.
It's also a quick way to get started with a standard development environment.
SSH agent forwarding
^^^^^^^^^^^^^^^^^^^^
We need buildkit_::
export DOCKER_BUILDKIT=1
.. _buildkit: https://docs.docker.com/develop/develop-images/build_enhancements/
And also the exact way for forwarding agent to running instance is different on OSX::
export DOCKER_SSHAGENT="-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock"
and Linux::
export DOCKER_SSHAGENT="-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK"
Creating a development container
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Build image, create container and start it::
docker build --ssh default --target devel_shell -t libadvian:devel_shell .
docker create --name libadvian_devel -v `pwd`":/app" -it `echo $DOCKER_SSHAGENT` libadvian:devel_shell
docker start -i libadvian_devel
pre-commit considerations
^^^^^^^^^^^^^^^^^^^^^^^^^
If working in Docker instead of native env you need to run the pre-commit checks in docker too::
docker exec -i libadvian_devel /bin/bash -c "pre-commit install"
docker exec -i libadvian_devel /bin/bash -c "pre-commit run --all-files"
You need to have the container running, see above. Or alternatively use the docker run syntax but using
the running container is faster::
docker run --rm -it -v `pwd`":/app" libadvian:devel_shell -c "pre-commit run --all-files"
Test suite
^^^^^^^^^^
You can use the devel shell to run py.test when doing development, for CI use
the "tox" target in the Dockerfile::
docker build --ssh default --target tox -t libadvian:tox .
docker run --rm -it -v `pwd`":/app" libadvian:tox
Development
-----------
TLDR:
- Create and activate a Python 3.9 virtualenv (assuming virtualenvwrapper)::
mkvirtualenv -p `which python3.9` my_virtualenv
- change to a branch::
git checkout -b my_branch
- install Poetry: https://python-poetry.org/docs/#installation
- Install project deps and pre-commit hooks::
poetry install
pre-commit install
pre-commit run --all-files
- Ready to go.
Remember to activate your virtualenv whenever working on the repo, this is needed
because pylint and mypy pre-commit hooks use the "system" python for now (because reasons).
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/advian-oss/python-libadvian/",
"name": "libadvian",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8.1,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Eero af Heurlin",
"author_email": "eero.afheurlin@advian.fi",
"download_url": "https://files.pythonhosted.org/packages/9a/2d/43bafaf02c8583c0c78525dfbb855c0c7d410931b355e1a873f0ba3b18ec/libadvian-1.7.0.tar.gz",
"platform": null,
"description": "=========\nlibadvian\n=========\n\nSmall helpers that do not warrant their own library.\n\nNotable helpers\n---------------\n\n - init_logging (libadvian.logging): Initializes default logger to our standard log format.\n - b64_to_uuid and uuid_to_b64 (libadvian.binpackers): URL-safe base64 UUID decode/encode.\n - ensure_[utf8|str] (libadvian.binpackers): For making sure you are dealing with bytes or strings.\n - TaskMaster.singleton().create_task(my_task()) (libadvian.tasks): for true fire-and-forget tasks.\n - chunked(iterable, size) (libadvian.iter): divide iterable into chunks.\n\nThere is more, everything is type hinted and documented, just look around.\n\nHTTP Logging notes\n------------------\n\nNote: requires you install the extra feature \"http\" to use\n\nIf LOG_HTTP_TARGET_URI is set in ENV, init_logging will add BufferedHTTPMultiRecordHandler handler\nfor shipping logs to Vector/Logtash (or similar that can take POSTs with text body), for username and password use\nLOG_HTTP_USERNAME and LOG_HTTP_PASSWORD variables.\n\nAny extra properties set by logger.level(\"message\", extra={\"key\": \"value\"}) will be resolved and added to\nlabels_json -header in the POST. If LOG_GLOBAL_LABELS_JSON is set in ENV that will be decoded and filter\nadded to init_logging -pipeline that will add those labels as extras to all logrecords.\n\nYou can of course initialize those handlers manually and use them as you wish. For quick and very dirty\nway of adding headers to the HTTP requests muck with libadvian.logging.httpmulti.HTTP_LOGGING_CONFIG\nbefore initializing logging::\n\n libadvian.logging.httpmulti.HTTP_LOGGING_CONFIG[\"handlers\"][\"http\"].update({\"session_options\": { \"headers\": {\n \"Myheader\": \"myval\"\n }}})\n\nMaking a deep copy and using logging.dictConfig() on the copy would be much cleaner though.\n\nDocker\n------\n\nFor more controlled deployments and to get rid of \"works on my computer\" -syndrome, we always\nmake sure our software works under docker.\n\nIt's also a quick way to get started with a standard development environment.\n\nSSH agent forwarding\n^^^^^^^^^^^^^^^^^^^^\n\nWe need buildkit_::\n\n export DOCKER_BUILDKIT=1\n\n.. _buildkit: https://docs.docker.com/develop/develop-images/build_enhancements/\n\nAnd also the exact way for forwarding agent to running instance is different on OSX::\n\n export DOCKER_SSHAGENT=\"-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock\"\n\nand Linux::\n\n export DOCKER_SSHAGENT=\"-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK\"\n\nCreating a development container\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBuild image, create container and start it::\n\n docker build --ssh default --target devel_shell -t libadvian:devel_shell .\n docker create --name libadvian_devel -v `pwd`\":/app\" -it `echo $DOCKER_SSHAGENT` libadvian:devel_shell\n docker start -i libadvian_devel\n\npre-commit considerations\n^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf working in Docker instead of native env you need to run the pre-commit checks in docker too::\n\n docker exec -i libadvian_devel /bin/bash -c \"pre-commit install\"\n docker exec -i libadvian_devel /bin/bash -c \"pre-commit run --all-files\"\n\nYou need to have the container running, see above. Or alternatively use the docker run syntax but using\nthe running container is faster::\n\n docker run --rm -it -v `pwd`\":/app\" libadvian:devel_shell -c \"pre-commit run --all-files\"\n\nTest suite\n^^^^^^^^^^\n\nYou can use the devel shell to run py.test when doing development, for CI use\nthe \"tox\" target in the Dockerfile::\n\n docker build --ssh default --target tox -t libadvian:tox .\n docker run --rm -it -v `pwd`\":/app\" libadvian:tox\n\nDevelopment\n-----------\n\nTLDR:\n\n- Create and activate a Python 3.9 virtualenv (assuming virtualenvwrapper)::\n\n mkvirtualenv -p `which python3.9` my_virtualenv\n\n- change to a branch::\n\n git checkout -b my_branch\n\n- install Poetry: https://python-poetry.org/docs/#installation\n- Install project deps and pre-commit hooks::\n\n poetry install\n pre-commit install\n pre-commit run --all-files\n\n- Ready to go.\n\nRemember to activate your virtualenv whenever working on the repo, this is needed\nbecause pylint and mypy pre-commit hooks use the \"system\" python for now (because reasons).\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Small helpers that do not warrant their own library",
"version": "1.7.0",
"project_urls": {
"Homepage": "https://gitlab.com/advian-oss/python-libadvian/",
"Repository": "https://gitlab.com/advian-oss/python-libadvian/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "28258fcc9d45c55ed994e99c8e842087baed045e3f4bb7c5d887bcfca9536839",
"md5": "e16077c84707e75a86bec2003a39229e",
"sha256": "380db468c46c10591d963a5334699128812c16a36e159f94ff98ee1afa908829"
},
"downloads": -1,
"filename": "libadvian-1.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e16077c84707e75a86bec2003a39229e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.1,<4.0",
"size": 15532,
"upload_time": "2024-03-05T08:04:34",
"upload_time_iso_8601": "2024-03-05T08:04:34.709308Z",
"url": "https://files.pythonhosted.org/packages/28/25/8fcc9d45c55ed994e99c8e842087baed045e3f4bb7c5d887bcfca9536839/libadvian-1.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9a2d43bafaf02c8583c0c78525dfbb855c0c7d410931b355e1a873f0ba3b18ec",
"md5": "40ee495cc354209f661067c1859c1eaa",
"sha256": "624b67337d0941175bac95214e9b86eef5c393edf61ffad9338d86e09f96eff2"
},
"downloads": -1,
"filename": "libadvian-1.7.0.tar.gz",
"has_sig": false,
"md5_digest": "40ee495cc354209f661067c1859c1eaa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.1,<4.0",
"size": 14183,
"upload_time": "2024-03-05T08:04:36",
"upload_time_iso_8601": "2024-03-05T08:04:36.526193Z",
"url": "https://files.pythonhosted.org/packages/9a/2d/43bafaf02c8583c0c78525dfbb855c0c7d410931b355e1a873f0ba3b18ec/libadvian-1.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-05 08:04:36",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "advian-oss",
"gitlab_project": "python-libadvian",
"lcname": "libadvian"
}
JSON