[Dramatiq](https://dramatiq.io/) is a simple task queue implementation for
Python3. dramatiq-pg provides a Postgres-based implementation of a dramatiq
broker.
## Features
- Super simple deployment: Single table, no ORM.
- Stores message payload and results as native JSONb.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Implements delayed task.
- Reliable thanks to Postgres MVCC.
- Self-healing: automatic purge of old messages. Automatic recovery after
crash.
- Utility CLI for maintainance: flush, purge, stats, etc.
Note that dramatiq assumes tasks are idempotent. This broker makes the same
assumptions for recovering after a crash.
## Installation
- Install dramatiq-pg package from PyPI:
``` console
$ pip install dramatiq-pg psycopg2-binary
```
Ensure you have either psycopg2 or psycopg2-binary installed.
- Init database schema with `init` command.
``` console
$ dramatiq-pg init
```
Or adapt `dramatiq-pg/schema.sql` to your needs.
- Before importing actors, define global broker with a connection
pool:
``` python
import dramatiq
import psycopg2.pool
from dramatiq_pg import PostgresBroker
dramatiq.set_broker(PostgresBroker(i))
@dramatiq.actor
def myactor():
...
```
Now declare/import actors and manage worker just like any [dramatiq
setup](https://dramatiq.io/guide.html). An [example
script](https://gitlab.com/dalibo/dramatiq-pg/blob/master/example.py) is
available, tested on CI.
The CLI tool `dramatiq-pg` allows you to requeue messages, purge old messages
and show stats on the queue. See `--help` for details.
[Dramatiq-pg
documentation](https://gitlab.com/dalibo/dramatiq-pg/blob/master/docs/index.rst)
is hosted on GitLab and give you more details on deployment and operation of
Postgres as a Dramatiq broker.
## Integration
**Django** : Use
[django-dramatiq-pg](https://github.com/uptick/django-dramatiq-pg/) by [Curtis
Maloney](https://gitlab.com/FunkyBob). It includes configuration, ORM model and
database migration.
## Support
If you encounter a bug or miss a feature, please [open an issue on
GitLab](https://gitlab.com/dalibo/dramatiq-pg/issues/new) with as much
information as possible.
dramatiq_pg is available under the PostgreSQL licence.
## Credit
Thanks to all contributors :
- Andy Freeland
- Curtis Maloney, Django support.
- Federico Caselli, bugfixes.
- Giuseppe Papallo, bugfixes.
- Rafal Kwasny, improvements.
The logo is a creation of [Damien CAZEILS](http://www.damiencazeils.com/)
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/dalibo/dramatiq-pg",
"name": "dramatiq-pg",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.6",
"maintainer_email": null,
"keywords": "postgres, task queue, dramatiq",
"author": "\u00c9tienne BERSAC",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/94/98/85c41e5dce7aece26641b28d82fd306325c49f7a92f1ccce0866308462c5/dramatiq_pg-0.12.0.tar.gz",
"platform": null,
"description": "\n\n[Dramatiq](https://dramatiq.io/) is a simple task queue implementation for\nPython3. dramatiq-pg provides a Postgres-based implementation of a dramatiq\nbroker.\n\n\n## Features\n\n- Super simple deployment: Single table, no ORM.\n- Stores message payload and results as native JSONb.\n- Uses LISTEN/NOTIFY to keep worker sync. No polling.\n- Implements delayed task.\n- Reliable thanks to Postgres MVCC.\n- Self-healing: automatic purge of old messages. Automatic recovery after\n crash.\n- Utility CLI for maintainance: flush, purge, stats, etc.\n\nNote that dramatiq assumes tasks are idempotent. This broker makes the same\nassumptions for recovering after a crash.\n\n\n## Installation\n\n- Install dramatiq-pg package from PyPI:\n ``` console\n $ pip install dramatiq-pg psycopg2-binary\n ```\n Ensure you have either psycopg2 or psycopg2-binary installed.\n- Init database schema with `init` command.\n ``` console\n $ dramatiq-pg init\n ```\n Or adapt `dramatiq-pg/schema.sql` to your needs.\n- Before importing actors, define global broker with a connection\n pool:\n ``` python\n import dramatiq\n import psycopg2.pool\n from dramatiq_pg import PostgresBroker\n\n dramatiq.set_broker(PostgresBroker(i))\n\n @dramatiq.actor\n def myactor():\n ...\n ```\n\nNow declare/import actors and manage worker just like any [dramatiq\nsetup](https://dramatiq.io/guide.html). An [example\nscript](https://gitlab.com/dalibo/dramatiq-pg/blob/master/example.py) is\navailable, tested on CI.\n\nThe CLI tool `dramatiq-pg` allows you to requeue messages, purge old messages\nand show stats on the queue. See `--help` for details.\n\n[Dramatiq-pg\ndocumentation](https://gitlab.com/dalibo/dramatiq-pg/blob/master/docs/index.rst)\nis hosted on GitLab and give you more details on deployment and operation of\nPostgres as a Dramatiq broker.\n\n\n## Integration\n\n**Django** : Use\n[django-dramatiq-pg](https://github.com/uptick/django-dramatiq-pg/) by [Curtis\nMaloney](https://gitlab.com/FunkyBob). It includes configuration, ORM model and\ndatabase migration.\n\n\n## Support\n\nIf you encounter a bug or miss a feature, please [open an issue on\nGitLab](https://gitlab.com/dalibo/dramatiq-pg/issues/new) with as much\ninformation as possible.\n\ndramatiq_pg is available under the PostgreSQL licence.\n\n\n## Credit\n\nThanks to all contributors :\n\n- Andy Freeland\n- Curtis Maloney, Django support.\n- Federico Caselli, bugfixes.\n- Giuseppe Papallo, bugfixes.\n- Rafal Kwasny, improvements.\n\n\nThe logo is a creation of [Damien CAZEILS](http://www.damiencazeils.com/)\n",
"bugtrack_url": null,
"license": "PostgreSQL",
"summary": "Postgres Broker for Dramatiq Task Queue",
"version": "0.12.0",
"project_urls": {
"Documentation": "https://gitlab.com/dalibo/dramatiq-pg/blob/master/docs/index.rst",
"Homepage": "https://gitlab.com/dalibo/dramatiq-pg"
},
"split_keywords": [
"postgres",
" task queue",
" dramatiq"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "02cc59114fca251d37c980b048bdb837fbdbf02ec8d2d64dd653c78628e6afd1",
"md5": "44328b62daf940e0b1760167bc8e6f0e",
"sha256": "808bbe854d1266922570bd5afeacac0ddb93f6a7dbf68624dde2f76d72d95b58"
},
"downloads": -1,
"filename": "dramatiq_pg-0.12.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "44328b62daf940e0b1760167bc8e6f0e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.6",
"size": 14992,
"upload_time": "2024-07-24T11:50:58",
"upload_time_iso_8601": "2024-07-24T11:50:58.822022Z",
"url": "https://files.pythonhosted.org/packages/02/cc/59114fca251d37c980b048bdb837fbdbf02ec8d2d64dd653c78628e6afd1/dramatiq_pg-0.12.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "949885c41e5dce7aece26641b28d82fd306325c49f7a92f1ccce0866308462c5",
"md5": "90b356372568e965657df88d89c11c49",
"sha256": "828245fcf3700f6098ec1bff9eb13e50acff69d9c490ceeec5b69ee75d0f9fb1"
},
"downloads": -1,
"filename": "dramatiq_pg-0.12.0.tar.gz",
"has_sig": false,
"md5_digest": "90b356372568e965657df88d89c11c49",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.6",
"size": 13716,
"upload_time": "2024-07-24T11:51:00",
"upload_time_iso_8601": "2024-07-24T11:51:00.767809Z",
"url": "https://files.pythonhosted.org/packages/94/98/85c41e5dce7aece26641b28d82fd306325c49f7a92f1ccce0866308462c5/dramatiq_pg-0.12.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-24 11:51:00",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "dalibo",
"gitlab_project": "dramatiq-pg",
"lcname": "dramatiq-pg"
}
JSON