# Celery AsyncIO Pool
<p align="center" width="100%">
<img width="55%" src="https://raw.githubusercontent.com/the-wondersmith/celery-aio-pool/main/icon.svg">
</p>
> Free software: GNU Affero General Public License v3+
## Getting Started
### Installation
#### Using Poetry _(preferred)_
```
poetry add celery-aio-pool
```
#### Using `pip` & [PyPI.org](https://pypi.org/project/celery-aio-pool/)
```
pip install celery-aio-pool
```
#### Using `pip` & [GitHub](https://github.com/the-wondersmith/celery-aio-pool.git)
```
pip install git+https://github.com/the-wondersmith/celery-aio-pool.git
```
### Using `pip` & A Local Copy Of The Repo
```
git clone https://github.com/the-wondersmith/celery-aio-pool.git
cd celery-aio-pool
pip install -e "$(pwd)"
```
### Configure Celery
#### Using `celery-aio-pool`'s Provided Patcher _(non-preferred)_
- Import `celery_aio_pool` in the same module where your Celery "app" is defined
- Ensure that the `patch_celery_tracer` utility is called **_before_** any other
Celery code is called
```python
"""My super awesome Celery app."""
# ...
from celery import Celery
# add the following import
import celery_aio_pool as aio_pool
# ensure the patcher is called *before*
# your Celery app is defined
assert aio_pool.patch_celery_tracer() is True
app = Celery(
"my-super-awesome-celery-app",
broker="amqp://guest@localhost//",
# add the following keyword argument
worker_pool=aio_pool.pool.AsyncIOPool,
)
```
#### Using (Upcoming) _Out-Of-Tree_ Worker Pool _(preferred)_
At the time of writing, [Celery](https://github.com/celery/celery) does not have
built-in support for out-of-tree pools like `celery-aio-pool`, but support should
be included starting with the first non-beta release of Celery 5.3. (note: [PR #7880](https://github.com/celery/celery/pull/7880) was merged on `2022-11-15`).
The official release of Celery 5.3 enables the configuration of custom worker pool classes thusly:
- Set the environment variable `CELERY_CUSTOM_WORKER_POOL` to the name of
your desired worker pool implementation implementation.
- **NOTE:** _The value of the environment variable must be formatted in
the standard Python/Celery format of_ `package:class`
- ```bash
% export CELERY_CUSTOM_WORKER_POOL='celery_aio_pool.pool:AsyncIOPool'
```
- Tell Celery to use your desired pool by specifying `--pool=custom` when running your worker instance(s)
- ```bash
% celery worker --pool=custom --loglevel=INFO --logfile="$(pwd)/worker.log"
```
To verify the pool implementation, examine the output of the `celery inspect stats`
command:
```bash
% celery --app=your_celery_project inspect stats
-> celery@freenas: OK
{
...
"pool": {
...
"implementation": "celery_aio_pool.pool:AsyncIOPool",
...
```
## Developing / Testing / Contributing
> **NOTE:** _Our preferred packaging and dependency manager is [Poetry](https://python-poetry.org/)._
> _Installation instructions can be found [here](https://python-poetry.org/docs/#installing-with-the-official-installer)._
### Developing
Clone the repo and install the dependencies
```bash
$ git clone https://github.com/the-wondersmith/celery-aio-pool.git \
&& cd celery-aio-pool \
&& poetry install --sync
```
Optionally, if you do not have or prefer _not_ to use Poetry, `celery-aio-pool` is
fully PEP-517 compliant and can be installed directly by any PEP-517-compliant package
manager.
```bash
$ cd celery-aio-pool \
&& pip install -e "$(pwd)"
```
> **TODO:** _Coming Soon™_
### Testing
To run the test suite:
```bash
$ poetry run pytest tests/
```
### Contributing
> **TODO:** _Coming Soon™_
Raw data
{
"_id": null,
"home_page": "https://github.com/the-wondersmith/celery-aio-pool",
"name": "celery-aio-pool",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.13",
"maintainer_email": "",
"keywords": "pool,celery,asyncio,concurrency,background-jobs",
"author": "Mark S.",
"author_email": "the@wondersmithd.dev",
"download_url": "https://files.pythonhosted.org/packages/43/42/08102ac737614592b024b36295a8069f234dc25cfd4c8563fa4e6f52e46e/celery_aio_pool-0.1.0rc6.tar.gz",
"platform": null,
"description": "# Celery AsyncIO Pool\n\n\n\n\n\n\n<p align=\"center\" width=\"100%\">\n <img width=\"55%\" src=\"https://raw.githubusercontent.com/the-wondersmith/celery-aio-pool/main/icon.svg\">\n</p>\n\n> Free software: GNU Affero General Public License v3+\n\n## Getting Started\n\n### Installation\n\n#### Using Poetry _(preferred)_\n\n```\npoetry add celery-aio-pool\n```\n\n#### Using `pip` & [PyPI.org](https://pypi.org/project/celery-aio-pool/)\n\n```\npip install celery-aio-pool\n```\n\n#### Using `pip` & [GitHub](https://github.com/the-wondersmith/celery-aio-pool.git)\n\n```\npip install git+https://github.com/the-wondersmith/celery-aio-pool.git\n```\n\n### Using `pip` & A Local Copy Of The Repo\n\n```\ngit clone https://github.com/the-wondersmith/celery-aio-pool.git\ncd celery-aio-pool\npip install -e \"$(pwd)\"\n```\n\n\n### Configure Celery\n\n#### Using `celery-aio-pool`'s Provided Patcher _(non-preferred)_\n\n- Import `celery_aio_pool` in the same module where your Celery \"app\" is defined\n- Ensure that the `patch_celery_tracer` utility is called **_before_** any other\n Celery code is called\n\n```python\n\"\"\"My super awesome Celery app.\"\"\"\n\n# ...\nfrom celery import Celery\n\n# add the following import\nimport celery_aio_pool as aio_pool\n\n# ensure the patcher is called *before*\n# your Celery app is defined\n\nassert aio_pool.patch_celery_tracer() is True\n\napp = Celery(\n \"my-super-awesome-celery-app\",\n broker=\"amqp://guest@localhost//\",\n # add the following keyword argument\n worker_pool=aio_pool.pool.AsyncIOPool,\n)\n```\n\n#### Using (Upcoming) _Out-Of-Tree_ Worker Pool _(preferred)_\n\nAt the time of writing, [Celery](https://github.com/celery/celery) does not have\nbuilt-in support for out-of-tree pools like `celery-aio-pool`, but support should\nbe included starting with the first non-beta release of Celery 5.3. (note: [PR #7880](https://github.com/celery/celery/pull/7880) was merged on `2022-11-15`).\n\nThe official release of Celery 5.3 enables the configuration of custom worker pool classes thusly:\n\n- Set the environment variable `CELERY_CUSTOM_WORKER_POOL` to the name of\n your desired worker pool implementation implementation.\n - **NOTE:** _The value of the environment variable must be formatted in\n the standard Python/Celery format of_ `package:class`\n - ```bash\n % export CELERY_CUSTOM_WORKER_POOL='celery_aio_pool.pool:AsyncIOPool'\n ```\n\n- Tell Celery to use your desired pool by specifying `--pool=custom` when running your worker instance(s)\n - ```bash\n % celery worker --pool=custom --loglevel=INFO --logfile=\"$(pwd)/worker.log\"\n ```\n\nTo verify the pool implementation, examine the output of the `celery inspect stats`\ncommand:\n\n```bash\n% celery --app=your_celery_project inspect stats\n-> celery@freenas: OK\n {\n ...\n \"pool\": {\n ...\n \"implementation\": \"celery_aio_pool.pool:AsyncIOPool\",\n ...\n```\n\n\n## Developing / Testing / Contributing\n\n> **NOTE:** _Our preferred packaging and dependency manager is [Poetry](https://python-poetry.org/)._\n> _Installation instructions can be found [here](https://python-poetry.org/docs/#installing-with-the-official-installer)._\n\n### Developing\n\nClone the repo and install the dependencies\n```bash\n$ git clone https://github.com/the-wondersmith/celery-aio-pool.git \\\n && cd celery-aio-pool \\\n && poetry install --sync\n```\n\nOptionally, if you do not have or prefer _not_ to use Poetry, `celery-aio-pool` is\nfully PEP-517 compliant and can be installed directly by any PEP-517-compliant package\nmanager.\n\n```bash\n$ cd celery-aio-pool \\\n && pip install -e \"$(pwd)\"\n```\n\n> **TODO:** _Coming Soon\u2122_\n\n### Testing\n\nTo run the test suite:\n\n```bash\n$ poetry run pytest tests/\n```\n\n### Contributing\n\n> **TODO:** _Coming Soon\u2122_\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "Celery worker pool with support for asyncio coroutines as tasks",
"version": "0.1.0rc6",
"project_urls": {
"Documentation": "https://github.com/the-wondersmith/celery-aio-pool",
"Homepage": "https://github.com/the-wondersmith/celery-aio-pool",
"Repository": "https://github.com/the-wondersmith/celery-aio-pool"
},
"split_keywords": [
"pool",
"celery",
"asyncio",
"concurrency",
"background-jobs"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d2ca208d09406909c6d2f75e9f4080e758cedc8f85da9cc707e6a57bda7a98ac",
"md5": "2875b40d756ca3e89f3a059327c8f34a",
"sha256": "ad3820a2906cfa3215d1a3b9012c84e437181ab509626459e15c724c3c72945e"
},
"downloads": -1,
"filename": "celery_aio_pool-0.1.0rc6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2875b40d756ca3e89f3a059327c8f34a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.13",
"size": 23213,
"upload_time": "2024-02-26T20:36:36",
"upload_time_iso_8601": "2024-02-26T20:36:36.588571Z",
"url": "https://files.pythonhosted.org/packages/d2/ca/208d09406909c6d2f75e9f4080e758cedc8f85da9cc707e6a57bda7a98ac/celery_aio_pool-0.1.0rc6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "434208102ac737614592b024b36295a8069f234dc25cfd4c8563fa4e6f52e46e",
"md5": "5275d1f520f38bab72f209c116e1759b",
"sha256": "7bca528e2c4a377defdd87d6817e3bc946c4b8b5ee21f9fe16d1a0f2b2573c1a"
},
"downloads": -1,
"filename": "celery_aio_pool-0.1.0rc6.tar.gz",
"has_sig": false,
"md5_digest": "5275d1f520f38bab72f209c116e1759b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.13",
"size": 24434,
"upload_time": "2024-02-26T20:36:37",
"upload_time_iso_8601": "2024-02-26T20:36:37.615937Z",
"url": "https://files.pythonhosted.org/packages/43/42/08102ac737614592b024b36295a8069f234dc25cfd4c8563fa4e6f52e46e/celery_aio_pool-0.1.0rc6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-26 20:36:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "the-wondersmith",
"github_project": "celery-aio-pool",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "celery-aio-pool"
}
JSON
