# Asyncio zookeeper client (aiozk)
[](https://pypi.python.org/pypi/aiozk)
[](https://github.com/micro-fan/aiozk/actions)
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Asyncio zookeeper client (aiozk)](#asyncio-zookeeper-client-aiozk)
- [Status](#status)
- [Installation](#installation)
- [Quick Example](#quick-example)
- [Recipes](#recipes)
- [Caution](#caution)
- [Testing](#testing)
- [Run tests](#run-tests)
- [Testing approach](#testing-approach)
- [Recipes testing](#recipes-testing)
- [Run some tests directly](#run-some-tests-directly)
- [References](#references)
<!-- markdown-toc end -->
## Status
Have no major bugs in client/session/connection, but recipes need more test
code to become more robust.
Any help and interest are welcome 😀
## Installation
```bash
$ pip install aiozk
```
## Quick Example
```python
import asyncio
from aiozk import ZKClient
async def main():
zk = ZKClient('localhost')
await zk.start()
await zk.create('/foo', data=b'bazz', ephemeral=True)
assert b'bazz' == await zk.get_data('/foo')
await zk.close()
asyncio.run(main())
```
## Recipes
You may use recipes, similar to zoonado, kazoo, and other libs:
```python
# assuming zk is aiozk.ZKClient
# Lock
async with await zk.recipes.Lock('/path/to/lock').acquire():
# ... Do some stuff ...
pass
# Barrier
barrier = zk.recipes.Barrier('/path/to/barrier)
await barrier.create()
await barrier.lift()
await barrier.wait()
# DoubleBarrier
double_barrier = zk.recipes.DoubleBarrier('/path/to/double/barrier', min_participants=4)
await double_barrier.enter(timeout=0.5)
# ... Do some stuff ...
await double_barrier.leave(timeout=0.5)
```
You can find full list of recipes provided by aiozk here:
[aiozk recipes](https://github.com/micro-fan/aiozk/tree/master/aiozk/recipes)
To understand ideas behind recipes [please read
this](https://zookeeper.apache.org/doc/current/recipes.html) and [even more
recipes here](http://curator.apache.org/curator-recipes/index.html). Make sure
you're familiar with all recipes before doing something new by yourself,
especially when it involves more than few zookeeper calls.
### Caution
Don't mix different type of recipes at the same znode path. For example,
creating a Lock and a DoubleBarrier object at the same path. It may cause
undefined behavior 😓
## Testing
**NB**: please ensure that you're using recent `docker-compose` version. You can get it by running
```
pip install --user -U docker-compose
```
### Run tests
```
# you should have access to docker
docker-compose build
./test-runner.sh
```
Or you can run tests with tox
```
pip install --user tox tox-docker
tox
```
### Testing approach
Most of tests are integration tests and running on real zookeeper instances.
We've chosen `zookeeper 3.5` version since it has an ability to dynamic reconfiguration and we're going to do all connecting/reconnecting/watches tests on zk docker cluster as this gives us the ability to restart any server and see what happens.
```sh
# first terminal: launch zookeeper cluster
docker-compose rm -fv && docker-compose build zk && docker-compose up --scale zk=7 zk_seed zk
# it will launch cluster in this terminal and remain. last lines should be like this:
zk_6 | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181'
zk_6 | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181
zk_6 | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181
zk_6 | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181
zk_6 | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181
zk_6 | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181
zk_6 | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181
zk_6 | server.7=172.23.0.6:2888:3888:observer;0.0.0.0:2181
zk_6 |
zk_6 |
zk_6 | Reconfiguring...
zk_6 | ethernal loop
zk_7 | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181\nserver.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181'
zk_7 | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181
zk_7 | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181
zk_7 | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181
zk_7 | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181
zk_7 | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181
zk_7 | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181
zk_7 | server.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181
zk_7 | server.8=172.23.0.8:2888:3888:observer;0.0.0.0:2181
zk_7 |
zk_7 |
zk_7 | Reconfiguring...
zk_7 | ethernal loop
```
Run tests in docker:
```sh
docker-compose run --no-deps aiozk
# last lines will be about testing results
............lot of lines ommited........
.
----------------------------------------------------------------------
Ran 3 tests in 1.059s
OK
```
Run tests locally:
```sh
# ZK_IP can be something from logs above, like: ZK_HOST=172.21.0.6:2181
ZK_HOST=<ZK_IP> ./venv/bin/pytest
```
### Recipes testing
It seems that usually recipes require several things to be tested:
* That recipe flow is working as expected
* Timeouts: reproduce every timeout with meaningful values (timeout 0.5s and block for 0.6s)
### Run some tests directly
Another way to run tests only which you are interested in quickly. Or this is
useful when you run tests under the other version of python.
```sh
# Run zookeeper container
docker run -p 2181:2181 zookeeper
# Run pytest directly at the development source tree
export ZK_HOST=localhost
pytest -s --log-cli-level=DEBUG aiozk/test/test_barrier.py
```
## References
* It is based on [wglass/zoonado](https://github.com/wglass/zoonado/tree/master/zoonado) implementation
Raw data
{
"_id": null,
"home_page": "http://github.com/micro-fan/aiozk",
"name": "aiozk",
"maintainer": "Kirill Pinchuk",
"docs_url": null,
"requires_python": null,
"maintainer_email": "cybergrind@gmail.com",
"keywords": "zookeeper, asyncio, async",
"author": "Kirill Pinchuk",
"author_email": "cybergrind@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/2a/96/efff751130fbb58e1c257f7f855eb647ccd9fb9ef9d63e9fbb4fd76d6e93/aiozk-0.31.0.tar.gz",
"platform": null,
"description": "# Asyncio zookeeper client (aiozk)\n\n[](https://pypi.python.org/pypi/aiozk)\n\n[](https://github.com/micro-fan/aiozk/actions)\n\n\n<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->\n**Table of Contents**\n\n- [Asyncio zookeeper client (aiozk)](#asyncio-zookeeper-client-aiozk)\n - [Status](#status)\n - [Installation](#installation)\n - [Quick Example](#quick-example)\n - [Recipes](#recipes)\n - [Caution](#caution)\n - [Testing](#testing)\n - [Run tests](#run-tests)\n - [Testing approach](#testing-approach)\n - [Recipes testing](#recipes-testing)\n - [Run some tests directly](#run-some-tests-directly)\n - [References](#references)\n\n<!-- markdown-toc end -->\n\n\n## Status\n\nHave no major bugs in client/session/connection, but recipes need more test\ncode to become more robust.\n\nAny help and interest are welcome \ud83d\ude00\n\n## Installation\n\n```bash\n$ pip install aiozk\n```\n\n\n## Quick Example\n\n```python\nimport asyncio\nfrom aiozk import ZKClient\n\n\nasync def main():\n zk = ZKClient('localhost')\n await zk.start()\n await zk.create('/foo', data=b'bazz', ephemeral=True)\n assert b'bazz' == await zk.get_data('/foo')\n await zk.close()\n\nasyncio.run(main())\n```\n\n## Recipes\n\nYou may use recipes, similar to zoonado, kazoo, and other libs:\n\n```python\n# assuming zk is aiozk.ZKClient\n\n# Lock\nasync with await zk.recipes.Lock('/path/to/lock').acquire():\n # ... Do some stuff ...\n pass\n\n# Barrier\nbarrier = zk.recipes.Barrier('/path/to/barrier)\nawait barrier.create()\nawait barrier.lift()\nawait barrier.wait()\n\n# DoubleBarrier\ndouble_barrier = zk.recipes.DoubleBarrier('/path/to/double/barrier', min_participants=4)\nawait double_barrier.enter(timeout=0.5)\n# ... Do some stuff ...\nawait double_barrier.leave(timeout=0.5)\n```\n\nYou can find full list of recipes provided by aiozk here:\n[aiozk recipes](https://github.com/micro-fan/aiozk/tree/master/aiozk/recipes)\n\nTo understand ideas behind recipes [please read\nthis](https://zookeeper.apache.org/doc/current/recipes.html) and [even more\nrecipes here](http://curator.apache.org/curator-recipes/index.html). Make sure\nyou're familiar with all recipes before doing something new by yourself,\nespecially when it involves more than few zookeeper calls.\n\n### Caution\nDon't mix different type of recipes at the same znode path. For example,\ncreating a Lock and a DoubleBarrier object at the same path. It may cause\nundefined behavior \ud83d\ude13\n\n## Testing\n\n**NB**: please ensure that you're using recent `docker-compose` version. You can get it by running\n\n```\npip install --user -U docker-compose\n```\n\n\n### Run tests\n\n```\n# you should have access to docker\n\ndocker-compose build\n./test-runner.sh\n```\n\nOr you can run tests with tox\n\n```\npip install --user tox tox-docker\ntox\n```\n\n### Testing approach\n\nMost of tests are integration tests and running on real zookeeper instances.\nWe've chosen `zookeeper 3.5` version since it has an ability to dynamic reconfiguration and we're going to do all connecting/reconnecting/watches tests on zk docker cluster as this gives us the ability to restart any server and see what happens.\n\n```sh\n# first terminal: launch zookeeper cluster\ndocker-compose rm -fv && docker-compose build zk && docker-compose up --scale zk=7 zk_seed zk\n\n# it will launch cluster in this terminal and remain. last lines should be like this:\n\nzk_6 | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181'\nzk_6 | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181\nzk_6 | server.7=172.23.0.6:2888:3888:observer;0.0.0.0:2181\nzk_6 |\nzk_6 |\nzk_6 | Reconfiguring...\nzk_6 | ethernal loop\nzk_7 | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181\\nserver.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181'\nzk_7 | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181\nzk_7 | server.8=172.23.0.8:2888:3888:observer;0.0.0.0:2181\nzk_7 |\nzk_7 |\nzk_7 | Reconfiguring...\nzk_7 | ethernal loop\n```\n\nRun tests in docker:\n\n```sh\ndocker-compose run --no-deps aiozk\n# last lines will be about testing results\n\n............lot of lines ommited........\n.\n----------------------------------------------------------------------\nRan 3 tests in 1.059s\n\nOK\n\n```\n\nRun tests locally:\n```sh\n# ZK_IP can be something from logs above, like: ZK_HOST=172.21.0.6:2181\nZK_HOST=<ZK_IP> ./venv/bin/pytest\n```\n\n### Recipes testing\n\nIt seems that usually recipes require several things to be tested:\n\n* That recipe flow is working as expected\n* Timeouts: reproduce every timeout with meaningful values (timeout 0.5s and block for 0.6s)\n\n\n### Run some tests directly\n\nAnother way to run tests only which you are interested in quickly. Or this is\nuseful when you run tests under the other version of python.\n\n```sh\n# Run zookeeper container\ndocker run -p 2181:2181 zookeeper\n\n# Run pytest directly at the development source tree\nexport ZK_HOST=localhost\npytest -s --log-cli-level=DEBUG aiozk/test/test_barrier.py\n```\n\n## References\n* It is based on [wglass/zoonado](https://github.com/wglass/zoonado/tree/master/zoonado) implementation\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Asyncio client for Zookeeper.",
"version": "0.31.0",
"project_urls": {
"Homepage": "http://github.com/micro-fan/aiozk"
},
"split_keywords": [
"zookeeper",
" asyncio",
" async"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6d20093bbb69400826d280288e846b1ef36488c5440a7c9fb4801c5c736677fd",
"md5": "bfb377282c607123ca5ef49eceb13214",
"sha256": "8d422cc4e586071bc28cf089af41c8feedf6b8c1ea3ff65d4fae2cbf8a3162fc"
},
"downloads": -1,
"filename": "aiozk-0.31.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bfb377282c607123ca5ef49eceb13214",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 61213,
"upload_time": "2024-04-18T10:35:41",
"upload_time_iso_8601": "2024-04-18T10:35:41.808784Z",
"url": "https://files.pythonhosted.org/packages/6d/20/093bbb69400826d280288e846b1ef36488c5440a7c9fb4801c5c736677fd/aiozk-0.31.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2a96efff751130fbb58e1c257f7f855eb647ccd9fb9ef9d63e9fbb4fd76d6e93",
"md5": "2d238ba599be79c3b9fd87c15da396a1",
"sha256": "592b3b7817ffdc07c0274f8727a0631b3c1538b4eb19f756d29c71af0618d941"
},
"downloads": -1,
"filename": "aiozk-0.31.0.tar.gz",
"has_sig": false,
"md5_digest": "2d238ba599be79c3b9fd87c15da396a1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 44701,
"upload_time": "2024-04-18T10:35:44",
"upload_time_iso_8601": "2024-04-18T10:35:44.807879Z",
"url": "https://files.pythonhosted.org/packages/2a/96/efff751130fbb58e1c257f7f855eb647ccd9fb9ef9d63e9fbb4fd76d6e93/aiozk-0.31.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-18 10:35:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "micro-fan",
"github_project": "aiozk",
"travis_ci": true,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "aiozk"
}
JSON
