| Name | rq JSON |
| Version |
2.4.1
JSON |
| download |
| home_page | None |
| Summary | RQ is a simple, lightweight, library for creating background jobs, and processing them. |
| upload_time | 2025-07-20 11:54:01 |
| maintainer | Selwin Ong |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
|
RQ (_Redis Queue_) is a simple Python library for queueing jobs and processing
them in the background with workers. It is backed by Redis or Valkey and is designed
to have a low barrier to entry while scaling incredibly well for large applications.
It can be integrated into your web stack easily, making it suitable for projects
of any size—from simple applications to high-volume enterprise systems.
RQ requires Redis >= 5 or Valkey >= 7.2.
[](https://github.com/rq/rq/actions?query=workflow%3A%22Test%22)
[](https://pypi.python.org/pypi/rq)
[](https://codecov.io/gh/rq/rq)
[](https://github.com/astral-sh/ruff)
Full documentation can be found [here][d].
## Support RQ
If you find RQ useful, please consider supporting this project via [Tidelift](https://tidelift.com/subscription/pkg/pypi-rq?utm_source=pypi-rq&utm_medium=referral&utm_campaign=readme).
## Getting started
First, run a Redis server, of course:
```console
$ redis-server
```
To put jobs on queues, you don't have to do anything special, just define
your typically lengthy or blocking function:
```python
import requests
def count_words_at_url(url):
"""Just an example function that's called async."""
resp = requests.get(url)
return len(resp.text.split())
```
Then, create an RQ queue:
```python
from redis import Redis
from rq import Queue
queue = Queue(connection=Redis())
```
And enqueue the function call:
```python
from my_module import count_words_at_url
job = queue.enqueue(count_words_at_url, 'https://stamps.id')
```
## Job Prioritization
By default, jobs are added to the end of a single queue. RQ offers two ways to give certain jobs higher priority:
#### 1. Enqueue at the front
You can enqueue a job at the front of its queue so it’s picked up before other jobs:
```python
job = queue.enqueue(count_words_at_url, 'https://stamps.id', at_front=True)
```
#### 2. Use multiple queues
You can create multiple queues and enqueue jobs into different queues based on their priority:
```python
from rq import Queue
high_priority_queue = Queue('high', connection=Redis())
low_priority_queue = Queue('low', connection=Redis())
# This job will be picked up before jobs in the low priority queue
# even if it was enqueued later
high_priority_queue.enqueue(urgent_task)
low_priority_queue.enqueue(non_urgent_task)
```
Then start workers with a prioritized queue list:
```console
$ rq worker high low
```
This command starts a worker that listens to both `high` and `low` queues. The worker will process
jobs from the `high` queue first, followed by the `low` queue. You can also run different workers
for different queues, allowing you to scale your workers based on the number of jobs in each queue.
## Scheduling Jobs
Scheduling jobs is also easy:
```python
# Schedule job to run at 9:15, October 10th
job = queue.enqueue_at(datetime(2019, 10, 10, 9, 15), say_hello)
# Schedule job to run in 10 seconds
job = queue.enqueue_in(timedelta(seconds=10), say_hello)
```
## Repeating Jobs
To execute a `Job` multiple times, use the `Repeat` class:
```python
from rq import Queue, Repeat
# Repeat job 3 times after successful execution, with 30 second intervals
queue.enqueue(my_function, repeat=Repeat(times=3, interval=30))
# Repeat job 3 times with different intervals between runs
queue.enqueue(my_function, repeat=Repeat(times=3, interval=[5, 10, 15]))
```
## Retrying Failed Jobs
Retrying failed jobs is also supported:
```python
from rq import Retry
# Retry up to 3 times, failed job will be requeued immediately
queue.enqueue(say_hello, retry=Retry(max=3))
# Retry up to 3 times, with configurable intervals between retries
queue.enqueue(say_hello, retry=Retry(max=3, interval=[10, 30, 60]))
```
For a more complete example, refer to the [docs][d]. But this is the essence.
## Cron Style Job Scheduling
To schedule jobs to be enqueued at specific intervals, RQ >= 2.4 now provides a cron-like feature (support for cron syntax coming soon).
First, create a configuration file (e.g., `cron_config.py`) that defines the jobs you want to run periodically.
```python
from rq import cron
from myapp import cleanup_database, send_daily_report
# Run database cleanup every 5 minutes
cron.register(
cleanup_database,
queue_name='default',
interval=300 # 5 minutes in seconds
)
# Send daily reports every 24 hours
cron.register(
send_daily_report,
queue_name='repeating_tasks',
args=('daily',),
kwargs={'format': 'pdf'},
interval=86400 # 24 hours in seconds
)
```
And then start the `rq cron` command to enqueue these jobs at specified intervals:
```sh
$ rq cron cron_config.py
```
More details on functionality can be found in the [docs](https://python-rq.org/docs/cron/).
### The Worker
To start executing enqueued function calls in the background, start a worker
from your project's directory:
```console
$ rq worker --with-scheduler
*** Listening for work on default
Got count_words_at_url('http://nvie.com') from default
Job result = 818
*** Listening for work on default
```
To run multiple workers in production, use process managers like `systemd`. RQ also ships with a beta version of `worker-pool` that lets you run multiple worker processes with a single command.
```console
$ rq worker-pool -n 4
```
More options are documented on [python-rq.org](https://python-rq.org/docs/workers/).
## Installation
Simply use the following command to install the latest released version:
```console
$ pip install rq
```
## Notes on Performance
**TL;DR — run `Worker` or `SpawnWorker` in production.**
In a simple hello world [microbenchmark](docs/benchmark.md), `SimpleWorker` processed 1,000 jobs in just 1.02 seconds vs. 6.64 seconds with the default `Worker`), more than 6x faster.
`SimpleWorker` is faster because it skips `fork()` or `spawn()` and runs jobs in process. `Worker` and `SpawnWorker` run each job in a separate process, acting as a sandbox that isolates crashes, memory leaks and enforce hard time-outs.
Although `SimpleWorker` is faster in benchmarks, this overhead is negligible in most real world applications like sending emails, generating reports, processing images, etc. In production systems, the time spent performing jobs usually dwarfs any queueing/worker overhead.
Use `SimpleWorker` in production only if:
* Your jobs are extremely short-lived (single digit milliseconds).
* The `fork()` or `spawn()` latency is a proven bottleneck at your traffic levels.
* Your job code is 100% trusted and known to be free of resource leaks and the possibility of crashing/segfaults.
## Docs
To build and run the docs, install [jekyll](https://jekyllrb.com/docs/) and run:
```shell
cd docs
jekyll serve
```
## Related Projects
If you use RQ, Check out these below repos which might be useful in your rq based project.
- [django-rq](https://github.com/rq/django-rq)
- [rq-dashboard](https://github.com/Parallels/rq-dashboard)
- [rqmonitor](https://github.com/pranavgupta1234/rqmonitor)
- [Flask-RQ2](https://github.com/rq/Flask-RQ2)
- [rq-scheduler](https://github.com/rq/rq-scheduler)
- [rq-dashboard-fastAPI](https://github.com/Hannes221/rq-dashboard-fast)
## Project history
This project has been inspired by the good parts of [Celery][1], [Resque][2]
and [this snippet][3], and has been created as a lightweight alternative to the
heaviness of Celery or other AMQP-based queueing implementations.
RQ is maintained by [Stamps](https://stamps.id), an Indonesian based company that provides enterprise grade CRM and order management systems.
[d]: http://python-rq.org/
[m]: http://pypi.python.org/pypi/mailer
[p]: http://docs.python.org/library/pickle.html
[1]: http://docs.celeryq.dev/
[2]: https://github.com/resque/resque
[3]: https://github.com/fengsp/flask-snippets/blob/1f65833a4291c5b833b195a09c365aa815baea4e/utilities/rq.py
Raw data
{
"_id": null,
"home_page": null,
"name": "rq",
"maintainer": "Selwin Ong",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Selwin Ong <selwin.ong@gmail.com>, Vincent Driessen <vincent@3rdcloud.com>",
"download_url": "https://files.pythonhosted.org/packages/9f/1a/76bd814898c4c574bc0e6100c4626247fc08c0194372d4d3b7bfcf752eae/rq-2.4.1.tar.gz",
"platform": null,
"description": "RQ (_Redis Queue_) is a simple Python library for queueing jobs and processing\nthem in the background with workers. It is backed by Redis or Valkey and is designed\nto have a low barrier to entry while scaling incredibly well for large applications.\nIt can be integrated into your web stack easily, making it suitable for projects\nof any size\u2014from simple applications to high-volume enterprise systems.\n\nRQ requires Redis >= 5 or Valkey >= 7.2.\n\n[](https://github.com/rq/rq/actions?query=workflow%3A%22Test%22)\n[](https://pypi.python.org/pypi/rq)\n[](https://codecov.io/gh/rq/rq)\n[](https://github.com/astral-sh/ruff)\n\n\nFull documentation can be found [here][d].\n\n\n## Support RQ\n\nIf you find RQ useful, please consider supporting this project via [Tidelift](https://tidelift.com/subscription/pkg/pypi-rq?utm_source=pypi-rq&utm_medium=referral&utm_campaign=readme).\n\n\n## Getting started\n\nFirst, run a Redis server, of course:\n\n```console\n$ redis-server\n```\n\nTo put jobs on queues, you don't have to do anything special, just define\nyour typically lengthy or blocking function:\n\n```python\nimport requests\n\ndef count_words_at_url(url):\n \"\"\"Just an example function that's called async.\"\"\"\n resp = requests.get(url)\n return len(resp.text.split())\n```\n\nThen, create an RQ queue:\n\n```python\nfrom redis import Redis\nfrom rq import Queue\n\nqueue = Queue(connection=Redis())\n```\n\nAnd enqueue the function call:\n\n```python\nfrom my_module import count_words_at_url\njob = queue.enqueue(count_words_at_url, 'https://stamps.id')\n```\n\n## Job Prioritization\n\nBy default, jobs are added to the end of a single queue. RQ offers two ways to give certain jobs higher priority:\n\n#### 1. Enqueue at the front\n\nYou can enqueue a job at the front of its queue so it\u2019s picked up before other jobs:\n\n```python\njob = queue.enqueue(count_words_at_url, 'https://stamps.id', at_front=True)\n```\n\n#### 2. Use multiple queues\nYou can create multiple queues and enqueue jobs into different queues based on their priority:\n\n```python\nfrom rq import Queue\nhigh_priority_queue = Queue('high', connection=Redis())\nlow_priority_queue = Queue('low', connection=Redis())\n\n# This job will be picked up before jobs in the low priority queue\n# even if it was enqueued later\nhigh_priority_queue.enqueue(urgent_task)\nlow_priority_queue.enqueue(non_urgent_task)\n```\n\nThen start workers with a prioritized queue list:\n```console\n$ rq worker high low\n```\nThis command starts a worker that listens to both `high` and `low` queues. The worker will process\njobs from the `high` queue first, followed by the `low` queue. You can also run different workers\nfor different queues, allowing you to scale your workers based on the number of jobs in each queue.\n\n## Scheduling Jobs\n\nScheduling jobs is also easy:\n\n```python\n# Schedule job to run at 9:15, October 10th\njob = queue.enqueue_at(datetime(2019, 10, 10, 9, 15), say_hello)\n\n# Schedule job to run in 10 seconds\njob = queue.enqueue_in(timedelta(seconds=10), say_hello)\n```\n\n## Repeating Jobs\n\nTo execute a `Job` multiple times, use the `Repeat` class:\n\n```python\nfrom rq import Queue, Repeat\n\n# Repeat job 3 times after successful execution, with 30 second intervals\nqueue.enqueue(my_function, repeat=Repeat(times=3, interval=30))\n\n# Repeat job 3 times with different intervals between runs\nqueue.enqueue(my_function, repeat=Repeat(times=3, interval=[5, 10, 15]))\n```\n\n## Retrying Failed Jobs\n\nRetrying failed jobs is also supported:\n\n```python\nfrom rq import Retry\n\n# Retry up to 3 times, failed job will be requeued immediately\nqueue.enqueue(say_hello, retry=Retry(max=3))\n\n# Retry up to 3 times, with configurable intervals between retries\nqueue.enqueue(say_hello, retry=Retry(max=3, interval=[10, 30, 60]))\n```\n\nFor a more complete example, refer to the [docs][d]. But this is the essence.\n\n## Cron Style Job Scheduling\n\nTo schedule jobs to be enqueued at specific intervals, RQ >= 2.4 now provides a cron-like feature (support for cron syntax coming soon).\n\nFirst, create a configuration file (e.g., `cron_config.py`) that defines the jobs you want to run periodically.\n\n```python\nfrom rq import cron\nfrom myapp import cleanup_database, send_daily_report\n\n# Run database cleanup every 5 minutes\ncron.register(\n cleanup_database,\n queue_name='default',\n interval=300 # 5 minutes in seconds\n)\n\n# Send daily reports every 24 hours\ncron.register(\n send_daily_report,\n queue_name='repeating_tasks',\n args=('daily',),\n kwargs={'format': 'pdf'},\n interval=86400 # 24 hours in seconds\n)\n```\n\nAnd then start the `rq cron` command to enqueue these jobs at specified intervals:\n\n```sh\n$ rq cron cron_config.py\n```\n\nMore details on functionality can be found in the [docs](https://python-rq.org/docs/cron/).\n\n### The Worker\n\nTo start executing enqueued function calls in the background, start a worker\nfrom your project's directory:\n\n```console\n$ rq worker --with-scheduler\n*** Listening for work on default\nGot count_words_at_url('http://nvie.com') from default\nJob result = 818\n*** Listening for work on default\n```\n\nTo run multiple workers in production, use process managers like `systemd`. RQ also ships with a beta version of `worker-pool` that lets you run multiple worker processes with a single command.\n\n```console\n$ rq worker-pool -n 4\n```\n\nMore options are documented on [python-rq.org](https://python-rq.org/docs/workers/).\n\n\n## Installation\n\nSimply use the following command to install the latest released version:\n```console\n$ pip install rq\n```\n\n## Notes on Performance\n\n**TL;DR \u2014 run `Worker` or `SpawnWorker` in production.**\n\nIn a simple hello world [microbenchmark](docs/benchmark.md), `SimpleWorker` processed 1,000 jobs in just 1.02 seconds vs. 6.64 seconds with the default `Worker`), more than 6x faster.\n\n`SimpleWorker` is faster because it skips `fork()` or `spawn()` and runs jobs in process. `Worker` and `SpawnWorker` run each job in a separate process, acting as a sandbox that isolates crashes, memory leaks and enforce hard time-outs.\n\nAlthough `SimpleWorker` is faster in benchmarks, this overhead is negligible in most real world applications like sending emails, generating reports, processing images, etc. In production systems, the time spent performing jobs usually dwarfs any queueing/worker overhead.\n\nUse `SimpleWorker` in production only if:\n* Your jobs are extremely short-lived (single digit milliseconds).\n* The `fork()` or `spawn()` latency is a proven bottleneck at your traffic levels.\n* Your job code is 100% trusted and known to be free of resource leaks and the possibility of crashing/segfaults.\n\n\n## Docs\n\nTo build and run the docs, install [jekyll](https://jekyllrb.com/docs/) and run:\n\n```shell\ncd docs\njekyll serve\n```\n\n## Related Projects\n\nIf you use RQ, Check out these below repos which might be useful in your rq based project.\n\n- [django-rq](https://github.com/rq/django-rq)\n- [rq-dashboard](https://github.com/Parallels/rq-dashboard)\n- [rqmonitor](https://github.com/pranavgupta1234/rqmonitor)\n- [Flask-RQ2](https://github.com/rq/Flask-RQ2)\n- [rq-scheduler](https://github.com/rq/rq-scheduler)\n- [rq-dashboard-fastAPI](https://github.com/Hannes221/rq-dashboard-fast)\n\n\n\n## Project history\n\nThis project has been inspired by the good parts of [Celery][1], [Resque][2]\nand [this snippet][3], and has been created as a lightweight alternative to the\nheaviness of Celery or other AMQP-based queueing implementations.\n\nRQ is maintained by [Stamps](https://stamps.id), an Indonesian based company that provides enterprise grade CRM and order management systems.\n\n\n[d]: http://python-rq.org/\n[m]: http://pypi.python.org/pypi/mailer\n[p]: http://docs.python.org/library/pickle.html\n[1]: http://docs.celeryq.dev/\n[2]: https://github.com/resque/resque\n[3]: https://github.com/fengsp/flask-snippets/blob/1f65833a4291c5b833b195a09c365aa815baea4e/utilities/rq.py\n",
"bugtrack_url": null,
"license": null,
"summary": "RQ is a simple, lightweight, library for creating background jobs, and processing them.",
"version": "2.4.1",
"project_urls": {
"changelog": "https://github.com/rq/rq/blob/master/CHANGES.md",
"documentation": "https://python-rq.org/docs/",
"homepage": "https://python-rq.org/",
"repository": "https://github.com/rq/rq/"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8ac4ffd7a6d9a706a50ab91c8bd42ff54cd9b228613d6bb80f7728a5144518b1",
"md5": "fd8c9d4114a963242d5ab9189833a678",
"sha256": "a3a0839ba3213a9be013b398670caf71d9360a0c8525f343687cf2c2199e5ec8"
},
"downloads": -1,
"filename": "rq-2.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fd8c9d4114a963242d5ab9189833a678",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 108014,
"upload_time": "2025-07-20T11:53:59",
"upload_time_iso_8601": "2025-07-20T11:53:59.355056Z",
"url": "https://files.pythonhosted.org/packages/8a/c4/ffd7a6d9a706a50ab91c8bd42ff54cd9b228613d6bb80f7728a5144518b1/rq-2.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9f1a76bd814898c4c574bc0e6100c4626247fc08c0194372d4d3b7bfcf752eae",
"md5": "c5ece96639e27838327400fef5f40785",
"sha256": "40ba01af3edacc008ab376009a3a547278d2bfe02a77cd4434adc0b01788239f"
},
"downloads": -1,
"filename": "rq-2.4.1.tar.gz",
"has_sig": false,
"md5_digest": "c5ece96639e27838327400fef5f40785",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 664540,
"upload_time": "2025-07-20T11:54:01",
"upload_time_iso_8601": "2025-07-20T11:54:01.519035Z",
"url": "https://files.pythonhosted.org/packages/9f/1a/76bd814898c4c574bc0e6100c4626247fc08c0194372d4d3b7bfcf752eae/rq-2.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-20 11:54:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rq",
"github_project": "rq",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "rq"
}
