# AsyncMQ
<p align="center">
<a href="https://asyncmq.dymmond.com"><img src="https://res.cloudinary.com/dymmond/image/upload/v1746002620/asyncmq/oq2qhgqdlra7rudxaqhl.png" alt="AsyncMQ Logo"></a>
</p>
<p align="center">
<span>⚡ Supercharge your async applications with tasks so fast, you’ll think you’re bending time itself. ⚡</span>
</p>
<p align="center">
<a href="https://github.com/dymmond/asyncmq/actions/workflows/test-suite.yml/badge.svg?event=push&branch=main" target="_blank">
<img src="https://github.com/dymmond/asyncmq/actions/workflows/test-suite.yml/badge.svg?event=push&branch=main" alt="Test Suite">
</a>
<a href="https://pypi.org/project/asyncmq" target="_blank">
<img src="https://img.shields.io/pypi/v/asyncmq?color=%2334D058&label=pypi%20package" alt="Package version">
</a>
<a href="https://img.shields.io/pypi/pyversions/asyncmq.svg?color=%2334D058" target="_blank">
<img src="https://img.shields.io/pypi/pyversions/asyncmq.svg?color=%2334D058" alt="Supported Python versions">
</a>
</p>
---
Welcome to **AsyncMQ**, the modern task queue that brings **powerful**, **flexible**, and **lightning-fast** background processing to your Python stack. Whether you're building microservices, handling high-throughput data pipelines, or just want reliable delayed jobs, AsyncMQ has your back.
## 🚀 Why Choose AsyncMQ?
1. **Asyncio & AnyIO-Native**
No more wrestling with threads or callbacks—AsyncMQ is built on `asyncio` and `anyio`, so your tasks are non-blocking and integrate seamlessly into modern async frameworks like FastAPI and Esmerald.
2. **Multi-Backend Flexibility**
From the blistering speed of Redis to the ACID guarantees of Postgres, or the schema-less power of MongoDB, AsyncMQ supports multiple backends out of the box—and you can write your own by implementing `BaseBackend`.
3. **Advanced Scheduling**
* **Delayed Jobs**: Fire off tasks in the future with precise delays.
* **Repeatable & Cron**: Create heartbeat jobs or cron-like schedules with a single line.
* **Scan & Scheduler**: Smart polling intervals and cron logic keep your timings accurate without burning CPU cycles.
4. **Robust Error Handling**
* **Retries & Backoff**: Exponential or custom backoff strategies to gracefully handle failures.
* **Dead Letter Queue (DLQ)**: Automatically route permanently failed tasks for later inspection and reprocessing.
5. **Observability & Event Hooks**
Tap into lifecycle events (`job:started`, `job:completed`, `job:failed`, `job:progress`) to power real-time dashboards, metrics, or custom alerts.
6. **Rate Limiting & Concurrency Control**
Fine-tune throughput with token-bucket rate limiting and capacity limiters. Scale your worker concurrency without overloading downstream systems.
7. **Sandboxed Execution**
Run untrusted or CPU-bound tasks in isolated subprocesses with timeouts to protect your main workers from rogue code or infinite loops.
8. **Flow/DAG Orchestration**
Create complex task graphs with dependencies using `FlowProducer`. Enqueue entire pipelines atomically or fall back to safe sequential logic.
9. **CLI & Dev Experience**
A feature-rich CLI for managing queues, jobs, and workers—intuitive commands, JSON output support, and built-in help for every scenario.
10. **Seamless ASGI Integration**
Out-of-the-box compatibility with FastAPI, Esmerald, or any ASGI application. Manage workers within your app’s lifecycle events.
---
## Installation
AsyncMQ requires Python 3.10+ to run and that its because the EOL will be later on only.
You can install in many different ways, the default brings `redis` to use with `RedisBackend`.
```shell
$ pip install asyncmq
```
**Postgres**
If you prefer the Postgres backend.
```shell
$ pip install asyncmq[postgres]
```
**Mongo DB**
You might want also Mongo DB backend.
```shell
$ pip install asyncmq[mongo]
```
**All in one go**
You might want to install everything and see what is the best for you.
```shell
$ pip install asyncmq[all]
```
### Dashboard
AsyncMQ comes with a dashboard and that requires some additional setup but nothing special but to install it.
```shell
$ pip install asyncmq[dashboard]
```
---
## Comparison with Other Python Task Queues
| Feature | **AsyncMQ** | Celery | RQ | Dramatiq | Huey |
| -------------------------- | -------------------------------- |-----------------| ---------- | ------------ | ---------- |
| **AsyncIO Native** | ✅ | ❌ | ❌ | ✅ | ❌ |
| **Pluggable Backends** | ✅ Redis, Postgres, Mongo, In-Mem | RabbitMQ/others | Redis only | Redis only | Redis only |
| **Rate Limiting** | ✅ built-in | ❌ | ❌ | ✅ via addon | ❌ |
| **Cron & Repeatable Jobs** | ✅ interval & cron expressions | ✅ | ✅ | ✅ | ✅ |
| **Progress Reporting** | ✅ event hooks | ✅ callbacks | ❌ | ✅ hooks | ❌ |
| **Dead Letter Queue** | ✅ | ✅ | ❌ | ✅ | ✅ |
| **Flow / DAG Support** | ✅ `FlowProducer` | ✅ chords | ❌ | ✅ extensions | ❌ |
| **ASGI-Friendly** | ✅ FastAPI/Esmerald integration | ❌ | ❌ | ❌ | ❌ |
| **CLI Management** | ✅ rich, JSON-friendly | ✅ | ✅ | ✅ | ✅ |
---
### Where **AsyncMQ** Shines
* **True AsyncIO & AnyIO integration**: zero thread hacks, full non-blocking background tasks.
* **Backend flexibility**: swap Redis, Postgres, MongoDB, or In-Memory with a single setting.
* **Built-in rate limiting & concurrency control**: protect downstream services out-of-the-box.
* **Event-driven hooks**: subscribe to `job:started`, `completed`, `failed`, `progress`, etc., for metrics and alerts.
* **ASGI integration**: manage workers within FastAPI or Esmerald lifecycles—no extra wrappers needed.
---
## ⚡ Core Features at a Glance
| Category | Highlights |
| ---------------------- | ----------------------------------------------------------------------------------------- |
| **Task Definition** | `@task` decorator, `.enqueue()`, progress callbacks, TTL, retries, dependencies, repeats |
| **Queue API** | `add()`, `add_bulk()`, `add_repeatable()`, `pause()`, `resume()`, `clean()`, stats |
| **Worker Engine** | `process_job()`, capacity & rate limiters, sandbox, event emitter, DLQ handling |
| **Scheduling** | `delayed_job_scanner`, `repeatable_scheduler`, cron vs interval support |
| **Flow Orchestration** | `FlowProducer.add_flow()`, atomic backend support, fallback mode for dependency wiring |
| **Configuration** | Central `Settings` dataclass, env var override, dynamic tuning of concurrency & intervals |
| **Observability** | LoggingConfig protocol, structured logs, event subscriptions, metrics integration |
| **CLI Management** | `asyncmq queue`, `asyncmq job`, `asyncmq worker`, `asyncmq info` commands |
---
## 🎬 Quickstart
1. **Install**
```bash
pip install asyncmq
```
2. **Configure**
Override the default settings `from asyncmq.conf import monkay` and create your own, then make it global.
```bash
export ASYNCMQ_SETTINGS_MODULE=myapp.settings.Settings
```
3. **Define a Task**
```python
from asyncmq.tasks import task
@task(queue="emails", retries=2, ttl=120)
async def send_welcome(email: str):
# Imagine real email-sending logic here
print(f"📧 Sent welcome to {email}")
```
4. **Enqueue & Run**
```python
import anyio
from asyncmq.queues import Queue
async def main():
q = Queue("emails")
await send_welcome.enqueue(q.backend, "alice@example.com", delay=10)
anyio.run(main)
```
5. **Start Workers**
```bash
asyncmq worker start emails --concurrency 5
```
---
AsyncMQ is more than just a task queue, it’s the swiss army knife of async background processing.
Dive into the **Learn** section to master every feature, or jump into the **Features** docs for quick reference.
Ready to bend time?
**Get started today** and experience async tasking at warp speed! 🎉
Raw data
{
"_id": null,
"home_page": null,
"name": "asyncmq",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "anyio, asyncio, background-jobs, celery-alternative, concurrency, cron, dead-letter-queue, esmerald, fastapi, job-scheduler, microservices, mongodb, parallelism, postgresql, rate-limiter, redis, retry-policy, task-queue",
"author": null,
"author_email": "Tiago Silva <tiago@tarsild.io>",
"download_url": "https://files.pythonhosted.org/packages/8a/7f/8f39108e8e5cf8bf2192c2b1be3f8d2c7bdfbaf0ad8a1de321e7beb72370/asyncmq-0.4.0.tar.gz",
"platform": null,
"description": "# AsyncMQ\n\n<p align=\"center\">\n <a href=\"https://asyncmq.dymmond.com\"><img src=\"https://res.cloudinary.com/dymmond/image/upload/v1746002620/asyncmq/oq2qhgqdlra7rudxaqhl.png\" alt=\"AsyncMQ Logo\"></a>\n</p>\n\n<p align=\"center\">\n <span>\u26a1 Supercharge your async applications with tasks so fast, you\u2019ll think you\u2019re bending time itself. \u26a1</span>\n</p>\n\n<p align=\"center\">\n <a href=\"https://github.com/dymmond/asyncmq/actions/workflows/test-suite.yml/badge.svg?event=push&branch=main\" target=\"_blank\">\n <img src=\"https://github.com/dymmond/asyncmq/actions/workflows/test-suite.yml/badge.svg?event=push&branch=main\" alt=\"Test Suite\">\n </a>\n <a href=\"https://pypi.org/project/asyncmq\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/v/asyncmq?color=%2334D058&label=pypi%20package\" alt=\"Package version\">\n </a>\n <a href=\"https://img.shields.io/pypi/pyversions/asyncmq.svg?color=%2334D058\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/pyversions/asyncmq.svg?color=%2334D058\" alt=\"Supported Python versions\">\n </a>\n</p>\n\n---\n\nWelcome to **AsyncMQ**, the modern task queue that brings **powerful**, **flexible**, and **lightning-fast** background processing to your Python stack. Whether you're building microservices, handling high-throughput data pipelines, or just want reliable delayed jobs, AsyncMQ has your back.\n\n## \ud83d\ude80 Why Choose AsyncMQ?\n\n1. **Asyncio & AnyIO-Native**\n No more wrestling with threads or callbacks\u2014AsyncMQ is built on `asyncio` and `anyio`, so your tasks are non-blocking and integrate seamlessly into modern async frameworks like FastAPI and Esmerald.\n\n2. **Multi-Backend Flexibility**\n From the blistering speed of Redis to the ACID guarantees of Postgres, or the schema-less power of MongoDB, AsyncMQ supports multiple backends out of the box\u2014and you can write your own by implementing `BaseBackend`.\n\n3. **Advanced Scheduling**\n\n * **Delayed Jobs**: Fire off tasks in the future with precise delays.\n * **Repeatable & Cron**: Create heartbeat jobs or cron-like schedules with a single line.\n * **Scan & Scheduler**: Smart polling intervals and cron logic keep your timings accurate without burning CPU cycles.\n\n4. **Robust Error Handling**\n\n * **Retries & Backoff**: Exponential or custom backoff strategies to gracefully handle failures.\n * **Dead Letter Queue (DLQ)**: Automatically route permanently failed tasks for later inspection and reprocessing.\n\n5. **Observability & Event Hooks**\n Tap into lifecycle events (`job:started`, `job:completed`, `job:failed`, `job:progress`) to power real-time dashboards, metrics, or custom alerts.\n\n6. **Rate Limiting & Concurrency Control**\n Fine-tune throughput with token-bucket rate limiting and capacity limiters. Scale your worker concurrency without overloading downstream systems.\n\n7. **Sandboxed Execution**\n Run untrusted or CPU-bound tasks in isolated subprocesses with timeouts to protect your main workers from rogue code or infinite loops.\n\n8. **Flow/DAG Orchestration**\n Create complex task graphs with dependencies using `FlowProducer`. Enqueue entire pipelines atomically or fall back to safe sequential logic.\n\n9. **CLI & Dev Experience**\n A feature-rich CLI for managing queues, jobs, and workers\u2014intuitive commands, JSON output support, and built-in help for every scenario.\n\n10. **Seamless ASGI Integration**\n Out-of-the-box compatibility with FastAPI, Esmerald, or any ASGI application. Manage workers within your app\u2019s lifecycle events.\n\n---\n\n## Installation\n\nAsyncMQ requires Python 3.10+ to run and that its because the EOL will be later on only.\n\nYou can install in many different ways, the default brings `redis` to use with `RedisBackend`.\n\n```shell\n$ pip install asyncmq\n```\n\n**Postgres**\n\nIf you prefer the Postgres backend.\n\n```shell\n$ pip install asyncmq[postgres]\n```\n\n**Mongo DB**\n\nYou might want also Mongo DB backend.\n\n```shell\n$ pip install asyncmq[mongo]\n```\n\n**All in one go**\n\nYou might want to install everything and see what is the best for you.\n\n```shell\n$ pip install asyncmq[all]\n```\n\n### Dashboard\n\nAsyncMQ comes with a dashboard and that requires some additional setup but nothing special but to install it.\n\n```shell\n$ pip install asyncmq[dashboard]\n```\n\n---\n\n## Comparison with Other Python Task Queues\n\n| Feature | **AsyncMQ** | Celery | RQ | Dramatiq | Huey |\n| -------------------------- | -------------------------------- |-----------------| ---------- | ------------ | ---------- |\n| **AsyncIO Native** | \u2705 | \u274c | \u274c | \u2705 | \u274c |\n| **Pluggable Backends** | \u2705 Redis, Postgres, Mongo, In-Mem | RabbitMQ/others | Redis only | Redis only | Redis only |\n| **Rate Limiting** | \u2705 built-in | \u274c | \u274c | \u2705 via addon | \u274c |\n| **Cron & Repeatable Jobs** | \u2705 interval & cron expressions | \u2705 | \u2705 | \u2705 | \u2705 |\n| **Progress Reporting** | \u2705 event hooks | \u2705 callbacks | \u274c | \u2705 hooks | \u274c |\n| **Dead Letter Queue** | \u2705 | \u2705 | \u274c | \u2705 | \u2705 |\n| **Flow / DAG Support** | \u2705 `FlowProducer` | \u2705 chords | \u274c | \u2705 extensions | \u274c |\n| **ASGI-Friendly** | \u2705 FastAPI/Esmerald integration | \u274c | \u274c | \u274c | \u274c |\n| **CLI Management** | \u2705 rich, JSON-friendly | \u2705 | \u2705 | \u2705 | \u2705 |\n\n---\n\n### Where **AsyncMQ** Shines\n\n* **True AsyncIO & AnyIO integration**: zero thread hacks, full non-blocking background tasks.\n* **Backend flexibility**: swap Redis, Postgres, MongoDB, or In-Memory with a single setting.\n* **Built-in rate limiting & concurrency control**: protect downstream services out-of-the-box.\n* **Event-driven hooks**: subscribe to `job:started`, `completed`, `failed`, `progress`, etc., for metrics and alerts.\n* **ASGI integration**: manage workers within FastAPI or Esmerald lifecycles\u2014no extra wrappers needed.\n\n---\n\n## \u26a1 Core Features at a Glance\n\n| Category | Highlights |\n| ---------------------- | ----------------------------------------------------------------------------------------- |\n| **Task Definition** | `@task` decorator, `.enqueue()`, progress callbacks, TTL, retries, dependencies, repeats |\n| **Queue API** | `add()`, `add_bulk()`, `add_repeatable()`, `pause()`, `resume()`, `clean()`, stats |\n| **Worker Engine** | `process_job()`, capacity & rate limiters, sandbox, event emitter, DLQ handling |\n| **Scheduling** | `delayed_job_scanner`, `repeatable_scheduler`, cron vs interval support |\n| **Flow Orchestration** | `FlowProducer.add_flow()`, atomic backend support, fallback mode for dependency wiring |\n| **Configuration** | Central `Settings` dataclass, env var override, dynamic tuning of concurrency & intervals |\n| **Observability** | LoggingConfig protocol, structured logs, event subscriptions, metrics integration |\n| **CLI Management** | `asyncmq queue`, `asyncmq job`, `asyncmq worker`, `asyncmq info` commands |\n\n---\n\n## \ud83c\udfac Quickstart\n\n1. **Install**\n\n ```bash\n pip install asyncmq\n ```\n2. **Configure**\n\n Override the default settings `from asyncmq.conf import monkay` and create your own, then make it global.\n\n ```bash\n export ASYNCMQ_SETTINGS_MODULE=myapp.settings.Settings\n ```\n3. **Define a Task**\n\n ```python\n from asyncmq.tasks import task\n\n @task(queue=\"emails\", retries=2, ttl=120)\n async def send_welcome(email: str):\n # Imagine real email-sending logic here\n print(f\"\ud83d\udce7 Sent welcome to {email}\")\n ```\n4. **Enqueue & Run**\n\n ```python\n import anyio\n from asyncmq.queues import Queue\n\n async def main():\n q = Queue(\"emails\")\n await send_welcome.enqueue(q.backend, \"alice@example.com\", delay=10)\n anyio.run(main)\n ```\n5. **Start Workers**\n\n ```bash\n asyncmq worker start emails --concurrency 5\n ```\n\n---\n\nAsyncMQ is more than just a task queue, it\u2019s the swiss army knife of async background processing.\nDive into the **Learn** section to master every feature, or jump into the **Features** docs for quick reference.\n\nReady to bend time?\n\n**Get started today** and experience async tasking at warp speed! \ud83c\udf89\n",
"bugtrack_url": null,
"license": null,
"summary": "Async task queue with BullMQ-like features",
"version": "0.4.0",
"project_urls": {
"Changelog": "https://asyncmq.dymmond.com/release-notes/",
"Documentation": "https://asyncmq.dymmond.com",
"Funding": "https://github.com/sponsors/tarsil",
"Homepage": "https://github.com/dymmond/asyncmq",
"Source": "https://github.com/dymmond/asyncmq"
},
"split_keywords": [
"anyio",
" asyncio",
" background-jobs",
" celery-alternative",
" concurrency",
" cron",
" dead-letter-queue",
" esmerald",
" fastapi",
" job-scheduler",
" microservices",
" mongodb",
" parallelism",
" postgresql",
" rate-limiter",
" redis",
" retry-policy",
" task-queue"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "d899aa3f81fc267f118f6c41a419b76a83bb78b187f38d486f312a3a4a0fb619",
"md5": "bc57ee031aaca84a5bd90602110e07d4",
"sha256": "8593efc2c6cac4f6be9bb9086c21c522b14dbacecf2d82fbf9850766b422d42a"
},
"downloads": -1,
"filename": "asyncmq-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bc57ee031aaca84a5bd90602110e07d4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 176208,
"upload_time": "2025-07-17T16:11:35",
"upload_time_iso_8601": "2025-07-17T16:11:35.170892Z",
"url": "https://files.pythonhosted.org/packages/d8/99/aa3f81fc267f118f6c41a419b76a83bb78b187f38d486f312a3a4a0fb619/asyncmq-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8a7f8f39108e8e5cf8bf2192c2b1be3f8d2c7bdfbaf0ad8a1de321e7beb72370",
"md5": "8db1ac786a6919ad51d5996dcea3c5ca",
"sha256": "4b7b0c478f0b7e4ffcd93e412fd2f30d2a47e9d758b7357aa8b5a72f48a58fe4"
},
"downloads": -1,
"filename": "asyncmq-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "8db1ac786a6919ad51d5996dcea3c5ca",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 133602,
"upload_time": "2025-07-17T16:11:36",
"upload_time_iso_8601": "2025-07-17T16:11:36.243260Z",
"url": "https://files.pythonhosted.org/packages/8a/7f/8f39108e8e5cf8bf2192c2b1be3f8d2c7bdfbaf0ad8a1de321e7beb72370/asyncmq-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-17 16:11:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sponsors",
"github_project": "tarsil",
"github_not_found": true,
"lcname": "asyncmq"
}
JSON
