# Concurrency-Limiter
Concurrency-Limiter is a python decorator to limit the number of concurrent executor in asyncio.gather
### Demonstration:
```python
@pytest.mark.asyncio
async def test_concurrency_limiter():
@concurrency_limiter(max_concurrent=5)
async def fetch_data(id):
logger.info(f"Fetch data: {id}")
await asyncio.sleep(1)
await asyncio.gather(*(fetch_data(i) for i in range(10)))
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:4]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:3]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:2]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:1]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>
INFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>
INFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:1]>
INFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:2]>
INFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:3]>
INFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:4]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:4]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:3]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:2]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:1]>
DEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>
```
## Table of Contents
- [concurrency_limiter](#concurrency_limiter)
- [Table of Contents](#table-of-contents)
- [Description](#description)
- [Installation](#installation)
- [Usage](#usage)
- [License](#license)
- [Contact](#contact)
## Description
Concurrency-Limiter is a Python library that provides a decorator to limit the number of concurrent executions in asyncio.gather. This ensures that your asynchronous tasks do not overwhelm system resources, allowing for better control over concurrency and improved performance in applications that rely on asynchronous operations.
## Installation
```bash
# Install the dependency
pip install concurrency-limiter
uv add concurrency-limiter
poetry add concurrency-limiter
```
## Usage
```python
from concurrency_limiter import concurrency_limiter
import asyncio
import random
# List of flowers for our examples
flowers = ["Rose", "Tulip", "Sunflower", "Daisy", "Lily"]
# Asynchronous function example to plant a flower
@concurrency_limiter(max_concurrent=3)
async def plant_flower():
flower = random.choice(flowers)
print(f"Attempting to plant a {flower}")
await asyncio.sleep(0.1) # Simulating planting time
if random.random() < 0.8: # 80% chance of failure
raise Exception(f"The {flower} didn't take root")
return f"{flower} planted successfully"
# Start of Selection
if __name__ == "__main__":
asyncio.run(asyncio.gather(*(plant_flower() for i in range(5))))
# End of Selection
```
## License
Concurrency-Limiter is released under the MIT License. See the [LICENSE](LICENSE) file for more details.
## Contact
For questions, suggestions, or issues related to Concurrency-Limiter, please open an issue on the GitHub repository.
Raw data
{
"_id": null,
"home_page": null,
"name": "concurrency-limiter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "async, async gen, decorator, exception handling, exponential backoff, generator, python, retry, sync",
"author": null,
"author_email": "Sylvain Mouquet <sylvain.mouquet@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/db/69/d8f436d35f921d1a8609862eb76d89dc9fc8dba5e1e327b539605c49d159/concurrency_limiter-1.1.0.tar.gz",
"platform": null,
"description": "# Concurrency-Limiter\n\nConcurrency-Limiter is a python decorator to limit the number of concurrent executor in asyncio.gather\n\n### Demonstration:\n\n```python\n@pytest.mark.asyncio\nasync def test_concurrency_limiter():\n @concurrency_limiter(max_concurrent=5)\n async def fetch_data(id):\n logger.info(f\"Fetch data: {id}\")\n await asyncio.sleep(1)\n\n await asyncio.gather(*(fetch_data(i) for i in range(10)))\n\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:4]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:3]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:2]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [unlocked, value:1]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>\nINFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>\nINFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:1]>\nINFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:2]>\nINFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:3]>\nINFO concurrency-limiter:__init__.py:59 [WAITING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:4]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:4]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:3]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:2]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked, waiters:1]>\nDEBUG concurrency-limiter:__init__.py:61 [RUNNING] func: fetch_data - <asyncio.locks.BoundedSemaphore object at 0x102778cd0 [locked]>\n\n```\n\n## Table of Contents\n\n- [concurrency_limiter](#concurrency_limiter)\n - [Table of Contents](#table-of-contents)\n - [Description](#description)\n - [Installation](#installation)\n - [Usage](#usage)\n - [License](#license)\n - [Contact](#contact)\n\n## Description\n\nConcurrency-Limiter is a Python library that provides a decorator to limit the number of concurrent executions in asyncio.gather. This ensures that your asynchronous tasks do not overwhelm system resources, allowing for better control over concurrency and improved performance in applications that rely on asynchronous operations.\n\n## Installation\n\n```bash\n# Install the dependency\npip install concurrency-limiter\nuv add concurrency-limiter\npoetry add concurrency-limiter\n```\n\n## Usage\n\n```python\nfrom concurrency_limiter import concurrency_limiter\nimport asyncio\nimport random\n\n# List of flowers for our examples\nflowers = [\"Rose\", \"Tulip\", \"Sunflower\", \"Daisy\", \"Lily\"]\n\n# Asynchronous function example to plant a flower\n@concurrency_limiter(max_concurrent=3)\nasync def plant_flower():\n flower = random.choice(flowers)\n print(f\"Attempting to plant a {flower}\")\n await asyncio.sleep(0.1) # Simulating planting time\n if random.random() < 0.8: # 80% chance of failure\n raise Exception(f\"The {flower} didn't take root\")\n return f\"{flower} planted successfully\"\n\n# Start of Selection\nif __name__ == \"__main__\":\n asyncio.run(asyncio.gather(*(plant_flower() for i in range(5))))\n# End of Selection\n```\n\n\n## License\n\nConcurrency-Limiter is released under the MIT License. See the [LICENSE](LICENSE) file for more details.\n\n## Contact\n\nFor questions, suggestions, or issues related to Concurrency-Limiter, please open an issue on the GitHub repository.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Limit the number of concurrent executions in asyncio.gather",
"version": "1.1.0",
"project_urls": {
"changelog": "https://github.com/sylvainmouquet/concurrency-limiter/releases",
"documentation": "https://github.com/sylvainmouquet/concurrency-limiter",
"homepage": "https://github.com/sylvainmouquet/concurrency-limiter",
"repository": "https://github.com/sylvainmouquet/concurrency-limiter"
},
"split_keywords": [
"async",
" async gen",
" decorator",
" exception handling",
" exponential backoff",
" generator",
" python",
" retry",
" sync"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "531cd4a6510dd97973d9668f6a4f2fb2ab9b8fb5391b056a6f39291c3c5a5f7a",
"md5": "1517d8ce9ca41f45ce83b760e5ef2cfb",
"sha256": "113c9e28e5ed22d92e19353c653cb8a807c8d68a50e5dbfda9d9e169116886c7"
},
"downloads": -1,
"filename": "concurrency_limiter-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1517d8ce9ca41f45ce83b760e5ef2cfb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 4303,
"upload_time": "2024-10-09T19:39:09",
"upload_time_iso_8601": "2024-10-09T19:39:09.110232Z",
"url": "https://files.pythonhosted.org/packages/53/1c/d4a6510dd97973d9668f6a4f2fb2ab9b8fb5391b056a6f39291c3c5a5f7a/concurrency_limiter-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "db69d8f436d35f921d1a8609862eb76d89dc9fc8dba5e1e327b539605c49d159",
"md5": "d79b80e01674317f447ffec1fbd49724",
"sha256": "e60666211a66881aec6f777714dd6308979773b5866f731be1c7cba0eb08efac"
},
"downloads": -1,
"filename": "concurrency_limiter-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "d79b80e01674317f447ffec1fbd49724",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 36657,
"upload_time": "2024-10-09T19:39:10",
"upload_time_iso_8601": "2024-10-09T19:39:10.267225Z",
"url": "https://files.pythonhosted.org/packages/db/69/d8f436d35f921d1a8609862eb76d89dc9fc8dba5e1e327b539605c49d159/concurrency_limiter-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-09 19:39:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sylvainmouquet",
"github_project": "concurrency-limiter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "concurrency-limiter"
}
JSON
