# Celery AsyncIO Pool
<p align="center" width="100%">
<img width="55%" src="https://raw.githubusercontent.com/the-wondersmith/celery-aio-pool/main/icon.svg">
</p>
> Free software: GNU Affero General Public License v3+
## Getting Started
### Installation
#### Using Poetry _(preferred)_
```
poetry add celery-aio-pool
```
#### Using `pip` & [PyPI.org](https://pypi.org/project/celery-aio-pool/)
```
pip install celery-aio-pool
```
#### Using `pip` & [GitHub](https://github.com/the-wondersmith/celery-aio-pool.git)
```
pip install git+https://github.com/the-wondersmith/celery-aio-pool.git
```
### Using `pip` & A Local Copy Of The Repo
```
git clone https://github.com/the-wondersmith/celery-aio-pool.git
cd celery-aio-pool
pip install -e "$(pwd)"
```
### Configure Celery
#### Using `celery-aio-pool`'s Provided Patcher _(non-preferred)_
- Import `celery_aio_pool` in the same module where your Celery "app" is defined
- Ensure that the `patch_celery_tracer` utility is called **_before_** any other
Celery code is called
```python
"""My super awesome Celery app."""
# ...
from celery import Celery
# add the following import
import celery_aio_pool as aio_pool
# ensure the patcher is called *before*
# your Celery app is defined
assert aio_pool.patch_celery_tracer() is True
app = Celery(
"my-super-awesome-celery-app",
broker="amqp://guest@localhost//",
# add the following keyword argument
worker_pool=aio_pool.pool.AsyncIOPool,
)
```
#### Using (Upcoming) _Out-Of-Tree_ Worker Pool _(preferred)_
At the time of writing, [Celery](https://github.com/celery/celery) does not have
built-in support for out-of-tree pools like `celery-aio-pool`, but support should
be included starting with the first non-beta release of Celery 5.3. (note: [PR #7880](https://github.com/celery/celery/pull/7880) was merged on `2022-11-15`).
The official release of Celery 5.3 enables the configuration of custom worker pool classes thusly:
- Set the environment variable `CELERY_CUSTOM_WORKER_POOL` to the name of
your desired worker pool implementation implementation.
- **NOTE:** _The value of the environment variable must be formatted in
the standard Python/Celery format of_ `package:class`
- ```bash
% export CELERY_CUSTOM_WORKER_POOL='celery_aio_pool.pool:AsyncIOPool'
```
- Tell Celery to use your desired pool by specifying `--pool=custom` when running your worker instance(s)
- ```bash
% celery worker --pool=custom --loglevel=INFO --logfile="$(pwd)/worker.log"
```
To verify the pool implementation, examine the output of the `celery inspect stats`
command:
```bash
% celery --app=your_celery_project inspect stats
-> celery@freenas: OK
{
...
"pool": {
...
"implementation": "celery_aio_pool.pool:AsyncIOPool",
...
```
## Developing / Testing / Contributing
> **NOTE:** _Our preferred packaging and dependency manager is [Poetry](https://python-poetry.org/)._
> _Installation instructions can be found [here](https://python-poetry.org/docs/#installing-with-the-official-installer)._
### Developing
Clone the repo and install the dependencies
```bash
$ git clone https://github.com/the-wondersmith/celery-aio-pool.git \
&& cd celery-aio-pool \
&& poetry install --sync
```
Optionally, if you do not have or prefer _not_ to use Poetry, `celery-aio-pool` is
fully PEP-517 compliant and can be installed directly by any PEP-517-compliant package
manager.
```bash
$ cd celery-aio-pool \
&& pip install -e "$(pwd)"
```
> **TODO:** _Coming Soon™_
### Testing
To run the test suite:
```bash
$ poetry run pytest tests/
```
### Contributing
> **TODO:** _Coming Soon™_
Raw data
{
"_id": null,
"home_page": "https://github.com/the-wondersmith/celery-aio-pool",
"name": "celery-aio-pool",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": "pool, celery, asyncio, concurrency, background-jobs",
"author": "Mark S.",
"author_email": "the@wondersmithd.dev",
"download_url": "https://files.pythonhosted.org/packages/80/92/2eff07cff3aca2e381151889c0d3e8555d98d2915a81c9063398dd0a7953/celery_aio_pool-0.1.0rc7.tar.gz",
"platform": null,
"description": "# Celery AsyncIO Pool\n\n\n\n\n\n\n<p align=\"center\" width=\"100%\">\n <img width=\"55%\" src=\"https://raw.githubusercontent.com/the-wondersmith/celery-aio-pool/main/icon.svg\">\n</p>\n\n> Free software: GNU Affero General Public License v3+\n\n## Getting Started\n\n### Installation\n\n#### Using Poetry _(preferred)_\n\n```\npoetry add celery-aio-pool\n```\n\n#### Using `pip` & [PyPI.org](https://pypi.org/project/celery-aio-pool/)\n\n```\npip install celery-aio-pool\n```\n\n#### Using `pip` & [GitHub](https://github.com/the-wondersmith/celery-aio-pool.git)\n\n```\npip install git+https://github.com/the-wondersmith/celery-aio-pool.git\n```\n\n### Using `pip` & A Local Copy Of The Repo\n\n```\ngit clone https://github.com/the-wondersmith/celery-aio-pool.git\ncd celery-aio-pool\npip install -e \"$(pwd)\"\n```\n\n\n### Configure Celery\n\n#### Using `celery-aio-pool`'s Provided Patcher _(non-preferred)_\n\n- Import `celery_aio_pool` in the same module where your Celery \"app\" is defined\n- Ensure that the `patch_celery_tracer` utility is called **_before_** any other\n Celery code is called\n\n```python\n\"\"\"My super awesome Celery app.\"\"\"\n\n# ...\nfrom celery import Celery\n\n# add the following import\nimport celery_aio_pool as aio_pool\n\n# ensure the patcher is called *before*\n# your Celery app is defined\n\nassert aio_pool.patch_celery_tracer() is True\n\napp = Celery(\n \"my-super-awesome-celery-app\",\n broker=\"amqp://guest@localhost//\",\n # add the following keyword argument\n worker_pool=aio_pool.pool.AsyncIOPool,\n)\n```\n\n#### Using (Upcoming) _Out-Of-Tree_ Worker Pool _(preferred)_\n\nAt the time of writing, [Celery](https://github.com/celery/celery) does not have\nbuilt-in support for out-of-tree pools like `celery-aio-pool`, but support should\nbe included starting with the first non-beta release of Celery 5.3. (note: [PR #7880](https://github.com/celery/celery/pull/7880) was merged on `2022-11-15`).\n\nThe official release of Celery 5.3 enables the configuration of custom worker pool classes thusly:\n\n- Set the environment variable `CELERY_CUSTOM_WORKER_POOL` to the name of\n your desired worker pool implementation implementation.\n - **NOTE:** _The value of the environment variable must be formatted in\n the standard Python/Celery format of_ `package:class`\n - ```bash\n % export CELERY_CUSTOM_WORKER_POOL='celery_aio_pool.pool:AsyncIOPool'\n ```\n\n- Tell Celery to use your desired pool by specifying `--pool=custom` when running your worker instance(s)\n - ```bash\n % celery worker --pool=custom --loglevel=INFO --logfile=\"$(pwd)/worker.log\"\n ```\n\nTo verify the pool implementation, examine the output of the `celery inspect stats`\ncommand:\n\n```bash\n% celery --app=your_celery_project inspect stats\n-> celery@freenas: OK\n {\n ...\n \"pool\": {\n ...\n \"implementation\": \"celery_aio_pool.pool:AsyncIOPool\",\n ...\n```\n\n\n## Developing / Testing / Contributing\n\n> **NOTE:** _Our preferred packaging and dependency manager is [Poetry](https://python-poetry.org/)._\n> _Installation instructions can be found [here](https://python-poetry.org/docs/#installing-with-the-official-installer)._\n\n### Developing\n\nClone the repo and install the dependencies\n```bash\n$ git clone https://github.com/the-wondersmith/celery-aio-pool.git \\\n && cd celery-aio-pool \\\n && poetry install --sync\n```\n\nOptionally, if you do not have or prefer _not_ to use Poetry, `celery-aio-pool` is\nfully PEP-517 compliant and can be installed directly by any PEP-517-compliant package\nmanager.\n\n```bash\n$ cd celery-aio-pool \\\n && pip install -e \"$(pwd)\"\n```\n\n> **TODO:** _Coming Soon\u2122_\n\n### Testing\n\nTo run the test suite:\n\n```bash\n$ poetry run pytest tests/\n```\n\n### Contributing\n\n> **TODO:** _Coming Soon\u2122_\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "Celery worker pool with support for asyncio coroutines as tasks",
"version": "0.1.0rc7",
"project_urls": {
"Documentation": "https://github.com/the-wondersmith/celery-aio-pool",
"Homepage": "https://github.com/the-wondersmith/celery-aio-pool",
"Repository": "https://github.com/the-wondersmith/celery-aio-pool"
},
"split_keywords": [
"pool",
" celery",
" asyncio",
" concurrency",
" background-jobs"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c1ba8a7fc15089e14cca48f3a34aeec566d6ec618d35ef41fdff7770749a9f57",
"md5": "515712d77a7d9c4f9fefadb2430dc005",
"sha256": "157c786c1e38f717345aae903866569c4b44cdecb24c912e2eadd029b9cd883c"
},
"downloads": -1,
"filename": "celery_aio_pool-0.1.0rc7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "515712d77a7d9c4f9fefadb2430dc005",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 23472,
"upload_time": "2024-09-13T16:25:22",
"upload_time_iso_8601": "2024-09-13T16:25:22.523868Z",
"url": "https://files.pythonhosted.org/packages/c1/ba/8a7fc15089e14cca48f3a34aeec566d6ec618d35ef41fdff7770749a9f57/celery_aio_pool-0.1.0rc7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "80922eff07cff3aca2e381151889c0d3e8555d98d2915a81c9063398dd0a7953",
"md5": "4bfa2a6ba610061531e78e9bb0718c6a",
"sha256": "d6a93229a5e0eb337c97787f1faafeb48f3df136de944abca300547f497ded6e"
},
"downloads": -1,
"filename": "celery_aio_pool-0.1.0rc7.tar.gz",
"has_sig": false,
"md5_digest": "4bfa2a6ba610061531e78e9bb0718c6a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8",
"size": 24695,
"upload_time": "2024-09-13T16:25:23",
"upload_time_iso_8601": "2024-09-13T16:25:23.811023Z",
"url": "https://files.pythonhosted.org/packages/80/92/2eff07cff3aca2e381151889c0d3e8555d98d2915a81c9063398dd0a7953/celery_aio_pool-0.1.0rc7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-13 16:25:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "the-wondersmith",
"github_project": "celery-aio-pool",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "celery-aio-pool"
}
JSON
