====================
datastreamservicelib
====================
AsyncIO eventloop helpers and Abstract Base Classes for making services that use ZMQ nice, easy and DRY
Usage
-----
Use the CookieCutter template at https://gitlab.com/advian-oss/python-datastreamserviceapp_template
You can also take a look at src/datastreamservicelib/console.py for some very simple test examples.
Docker
------
For more controlled deployments and to get rid of "works on my computer" -syndrome, we always
make sure our software works under docker.
It's also a quick way to get started with a standard development environment.
SSH agent forwarding
^^^^^^^^^^^^^^^^^^^^
We need buildkit_::
export DOCKER_BUILDKIT=1
.. _buildkit: https://docs.docker.com/develop/develop-images/build_enhancements/
And also the exact way for forwarding agent to running instance is different on OSX::
export DOCKER_SSHAGENT="-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock"
and Linux::
export DOCKER_SSHAGENT="-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK"
Creating a development container
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Build image, create container and start it::
docker build --ssh default --target devel_shell -t datastreamservicelib:devel_shell .
docker create --name datastreamservicelib_devel -v `pwd`":/app" -it -v /tmp:/tmp `echo $DOCKER_SSHAGENT` datastreamservicelib:devel_shell
docker start -i datastreamservicelib_devel
pre-commit considerations
^^^^^^^^^^^^^^^^^^^^^^^^^
If working in Docker instead of native env you need to run the pre-commit checks in docker too::
docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit install"
docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit run --all-files"
You need to have the container running, see above. Or alternatively use the docker run syntax but using
the running container is faster::
docker run -it --rm -v `pwd`":/app" datastreamservicelib:devel_shell -c "pre-commit run --all-files"
Test suite
^^^^^^^^^^
You can use the devel shell to run py.test when doing development, for CI use
the "tox" target in the Dockerfile::
docker build --ssh default --target tox -t datastreamservicelib:tox .
docker run -it --rm -v `pwd`":/app" `echo $DOCKER_SSHAGENT` datastreamservicelib:tox
Production docker
^^^^^^^^^^^^^^^^^
There's a "production" target as well for quick running of testsubscriber and testpublisher
docker build --ssh default --target production -t datastreamservicelib:latest .
docker run --rm -it -v /tmp:/tmp datastreamservicelib:latest testpublisher -s ipc:///tmp/test_pub.sock -t foo
docker run --rm -it -v /tmp:/tmp datastreamservicelib:latest testsubscriber -s ipc:///tmp/test_pub.sock -t foo
Note that on non-linux platforms the IPC sockets may not as expected between host and container over volume mounts.
Local Development
-----------------
TLDR:
- Create and activate a Python 3.6 virtualenv (assuming virtualenvwrapper)::
mkvirtualenv -p `which python3.6` my_virtualenv
- change to a branch::
git checkout -b my_branch
- install Poetry: https://python-poetry.org/docs/#installation
- Install project deps and pre-commit hooks::
poetry install
pre-commit install
pre-commit run --all-files
- Ready to go, try the following::
testpublisher --help
testsubscriber --help
Use Python 3.6 for development since it's the lowest supported version so you don't accidentally
use features only available in higher versions and then have to re-do everything when CI tests fail
on 3.6.
Remember to activate your virtualenv whenever working on the repo, this is needed
because pylint and mypy pre-commit hooks use the "system" python for now (because reasons).
Running "pre-commit run --all-files" and "py.test -v" regularly during development and
especially before committing will save you some headache.
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/advian-oss/python-datastreamservicelib/",
"name": "datastreamservicelib",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6.2,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Eero af Heurlin",
"author_email": "eero.afheurlin@iki.fi",
"download_url": "https://files.pythonhosted.org/packages/de/9f/584ee5ed6ab82180711072f0ae8d271afef78f1b0985d399662602d058ef/datastreamservicelib-1.13.0.tar.gz",
"platform": null,
"description": "====================\ndatastreamservicelib\n====================\n\nAsyncIO eventloop helpers and Abstract Base Classes for making services that use ZMQ nice, easy and DRY\n\nUsage\n-----\n\nUse the CookieCutter template at https://gitlab.com/advian-oss/python-datastreamserviceapp_template\n\nYou can also take a look at src/datastreamservicelib/console.py for some very simple test examples.\n\nDocker\n------\n\nFor more controlled deployments and to get rid of \"works on my computer\" -syndrome, we always\nmake sure our software works under docker.\n\nIt's also a quick way to get started with a standard development environment.\n\nSSH agent forwarding\n^^^^^^^^^^^^^^^^^^^^\n\nWe need buildkit_::\n\n export DOCKER_BUILDKIT=1\n\n.. _buildkit: https://docs.docker.com/develop/develop-images/build_enhancements/\n\nAnd also the exact way for forwarding agent to running instance is different on OSX::\n\n export DOCKER_SSHAGENT=\"-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock\"\n\nand Linux::\n\n export DOCKER_SSHAGENT=\"-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK\"\n\nCreating a development container\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBuild image, create container and start it::\n\n docker build --ssh default --target devel_shell -t datastreamservicelib:devel_shell .\n docker create --name datastreamservicelib_devel -v `pwd`\":/app\" -it -v /tmp:/tmp `echo $DOCKER_SSHAGENT` datastreamservicelib:devel_shell\n docker start -i datastreamservicelib_devel\n\npre-commit considerations\n^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf working in Docker instead of native env you need to run the pre-commit checks in docker too::\n\n docker exec -i datastreamservicelib_devel /bin/bash -c \"pre-commit install\"\n docker exec -i datastreamservicelib_devel /bin/bash -c \"pre-commit run --all-files\"\n\nYou need to have the container running, see above. Or alternatively use the docker run syntax but using\nthe running container is faster::\n\n docker run -it --rm -v `pwd`\":/app\" datastreamservicelib:devel_shell -c \"pre-commit run --all-files\"\n\nTest suite\n^^^^^^^^^^\n\nYou can use the devel shell to run py.test when doing development, for CI use\nthe \"tox\" target in the Dockerfile::\n\n docker build --ssh default --target tox -t datastreamservicelib:tox .\n docker run -it --rm -v `pwd`\":/app\" `echo $DOCKER_SSHAGENT` datastreamservicelib:tox\n\nProduction docker\n^^^^^^^^^^^^^^^^^\n\nThere's a \"production\" target as well for quick running of testsubscriber and testpublisher\n\n docker build --ssh default --target production -t datastreamservicelib:latest .\n docker run --rm -it -v /tmp:/tmp datastreamservicelib:latest testpublisher -s ipc:///tmp/test_pub.sock -t foo\n docker run --rm -it -v /tmp:/tmp datastreamservicelib:latest testsubscriber -s ipc:///tmp/test_pub.sock -t foo\n\nNote that on non-linux platforms the IPC sockets may not as expected between host and container over volume mounts.\n\n\nLocal Development\n-----------------\n\nTLDR:\n\n- Create and activate a Python 3.6 virtualenv (assuming virtualenvwrapper)::\n\n mkvirtualenv -p `which python3.6` my_virtualenv\n\n- change to a branch::\n\n git checkout -b my_branch\n\n- install Poetry: https://python-poetry.org/docs/#installation\n- Install project deps and pre-commit hooks::\n\n poetry install\n pre-commit install\n pre-commit run --all-files\n\n- Ready to go, try the following::\n\n testpublisher --help\n testsubscriber --help\n\nUse Python 3.6 for development since it's the lowest supported version so you don't accidentally\nuse features only available in higher versions and then have to re-do everything when CI tests fail\non 3.6.\n\nRemember to activate your virtualenv whenever working on the repo, this is needed\nbecause pylint and mypy pre-commit hooks use the \"system\" python for now (because reasons).\n\nRunning \"pre-commit run --all-files\" and \"py.test -v\" regularly during development and\nespecially before committing will save you some headache.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "AsyncIO eventloop helpers and Abstract Base Classes for making services that use ZMQ nice, easy and DRY",
"version": "1.13.0",
"project_urls": {
"Homepage": "https://gitlab.com/advian-oss/python-datastreamservicelib/",
"Repository": "https://gitlab.com/advian-oss/python-datastreamservicelib/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "267f379a2097d4190bcce3004494f4f812898d3c738f7be31db3e83ac1580e35",
"md5": "c2fec9bee453b92c3059eec692403524",
"sha256": "44e7d0a07f0c8012accb0bc70845e2047b18498667dd04f40ee5779810f829fe"
},
"downloads": -1,
"filename": "datastreamservicelib-1.13.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c2fec9bee453b92c3059eec692403524",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.2,<4.0",
"size": 14510,
"upload_time": "2023-05-09T10:30:29",
"upload_time_iso_8601": "2023-05-09T10:30:29.423468Z",
"url": "https://files.pythonhosted.org/packages/26/7f/379a2097d4190bcce3004494f4f812898d3c738f7be31db3e83ac1580e35/datastreamservicelib-1.13.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "de9f584ee5ed6ab82180711072f0ae8d271afef78f1b0985d399662602d058ef",
"md5": "75373e3dc2c501b72455f6678cdff96c",
"sha256": "c9ad90506b3860e93285f163264ea927ed22608f330f3170f0676c634f0bedf6"
},
"downloads": -1,
"filename": "datastreamservicelib-1.13.0.tar.gz",
"has_sig": false,
"md5_digest": "75373e3dc2c501b72455f6678cdff96c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.2,<4.0",
"size": 13785,
"upload_time": "2023-05-09T10:30:30",
"upload_time_iso_8601": "2023-05-09T10:30:30.669145Z",
"url": "https://files.pythonhosted.org/packages/de/9f/584ee5ed6ab82180711072f0ae8d271afef78f1b0985d399662602d058ef/datastreamservicelib-1.13.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-09 10:30:30",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "advian-oss",
"gitlab_project": "python-datastreamservicelib",
"lcname": "datastreamservicelib"
}
JSON