taskiq


Nametaskiq JSON
Version 0.11.0 PyPI version JSON
download
home_pagehttps://taskiq-python.github.io/
SummaryDistributed task queue with full async support
upload_time2024-02-05 23:38:45
maintainerPavel Kirilin
docs_urlNone
authorPavel Kirilin
requires_python>=3.8.1,<4.0.0
licenseLICENSE
keywords taskiq tasks distributed async
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/taskiq?style=for-the-badge)](https://pypi.org/project/taskiq/)
[![PyPI](https://img.shields.io/pypi/v/taskiq?style=for-the-badge)](https://pypi.org/project/taskiq/)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/taskiq?style=for-the-badge)](https://pypistats.org/packages/taskiq)

<div align="center">
<a href="https://taskiq-python.github.io/"><img src="https://raw.githubusercontent.com/taskiq-python/taskiq/master/imgs/logo.svg" width=600></a>
<hr/>
</div>

Documentation: https://taskiq-python.github.io/

## What is taskiq?

Taskiq is an asynchronous distributed task queue for python.
This project takes inspiration from big projects such as [Celery](https://docs.celeryq.dev) and [Dramatiq](https://dramatiq.io/).
But taskiq can send and run both the sync and async functions, has
integration with popular async frameworks, such as [FastAPI](https://fastapi.tiangolo.com/) and [AioHTTP](https://docs.aiohttp.org/en/stable/).

Also, we use [PEP-612](https://peps.python.org/pep-0612/) to provide the best autosuggestions possible. All code is type-hinted.

# Installation

This project can be installed using pip:
```bash
pip install taskiq
```

Or it can be installed directly from git:

```bash
pip install git+https://github.com/taskiq-python/taskiq
```

# Usage

At first you need to create a broker. Broker is an object that can communicate to workers using distributed queues.

We have differet brokers for different queue backends. For example, we have a broker for [NATS](https://github.com/taskiq-python/taskiq-nats), [Redis](https://github.com/taskiq-python/taskiq-redis), [RabbitMQ](https://github.com/taskiq-python/taskiq-aio-pika), [Kafka](https://github.com/taskiq-python/taskiq-aio-kafka) and even more. Choose the one that fits you and create an instance.

```python
from taskiq_nats import JetStreamBroker

broker = JetStreamBroker("nats://localhost:4222", queue="my_queue")
```

Declaring tasks is as easy as declaring a function. Just add a decorator to your function and you are ready to go.

```python
import asyncio

from taskiq_nats import JetStreamBroker

broker = JetStreamBroker("nats://localhost:4222", queue="my_queue2")


@broker.task
async def my_task(a: int, b: int) -> None:
    print("AB", a + b)


async def main():
    await broker.startup()

    await my_task.kiq(1, 2)

    await broker.shutdown()


if __name__ == "__main__":
    asyncio.run(main())


```

The message is going to be sent to the broker and then to the worker. The worker will execute the function. To start worker processes, just run the following command:

```bash
taskiq worker path.to.the.module:broker
```

Where `path.to.the.module` is the path to the module where the broker is defined and `broker` is the name of the broker variable.

If you have tasks in different modules, you can ask taskiq to automatically import them by passing the `--fs-discover` flag:

```bash
taskiq worker path.to.the.module:broker --fs-discover
```

It will import all modules called `tasks.py` in the current directory and all subdirectories.

Also, we support hot reload for workers. To enable it, just pass the `--reload` flag. It will reload the worker when the code changes (To use it, install taskiq with reload extra. E.g `pip install taskiq[reload]`).


Also, we have cool integrations with popular async frameworks. For example, we have an integration with [FastAPI](https://taskiq-python.github.io/framework_integrations/taskiq-with-fastapi.html) or [AioHTTP](https://taskiq-python.github.io/framework_integrations/taskiq-with-aiohttp.html). You can use it to reuse dependencies from your web app in your tasks.

Read about all features in our documentation: https://taskiq-python.github.io/

# Local development


## Linting

We use pre-commit to do linting locally.

After cloning this project, please install [pre-commit](https://pre-commit.com/#install). It helps fix files before committing changes.

```bash
pre-commit install
```


## Testing

Pytest can run without any additional actions or options.

```bash
pytest
```

## Docs

To run docs locally, you need to install [yarn](https://yarnpkg.com/getting-started/install).

First, you need to install dependencies.
```
yarn install
```

After that you can set up a docs server by running:

```
yarn docs:dev
```

            

Raw data

            {
    "_id": null,
    "home_page": "https://taskiq-python.github.io/",
    "name": "taskiq",
    "maintainer": "Pavel Kirilin",
    "docs_url": null,
    "requires_python": ">=3.8.1,<4.0.0",
    "maintainer_email": "win10@list.ru",
    "keywords": "taskiq,tasks,distributed,async",
    "author": "Pavel Kirilin",
    "author_email": "win10@list.ru",
    "download_url": "https://files.pythonhosted.org/packages/d0/c4/ce1c79093604ef88172f9ab2c97e6917d8a8dde26a7080d31eca6bd3d150/taskiq-0.11.0.tar.gz",
    "platform": null,
    "description": "[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/taskiq?style=for-the-badge)](https://pypi.org/project/taskiq/)\n[![PyPI](https://img.shields.io/pypi/v/taskiq?style=for-the-badge)](https://pypi.org/project/taskiq/)\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/taskiq?style=for-the-badge)](https://pypistats.org/packages/taskiq)\n\n<div align=\"center\">\n<a href=\"https://taskiq-python.github.io/\"><img src=\"https://raw.githubusercontent.com/taskiq-python/taskiq/master/imgs/logo.svg\" width=600></a>\n<hr/>\n</div>\n\nDocumentation: https://taskiq-python.github.io/\n\n## What is taskiq?\n\nTaskiq is an asynchronous distributed task queue for python.\nThis project takes inspiration from big projects such as [Celery](https://docs.celeryq.dev) and [Dramatiq](https://dramatiq.io/).\nBut taskiq can send and run both the sync and async functions, has\nintegration with popular async frameworks, such as [FastAPI](https://fastapi.tiangolo.com/) and [AioHTTP](https://docs.aiohttp.org/en/stable/).\n\nAlso, we use [PEP-612](https://peps.python.org/pep-0612/) to provide the best autosuggestions possible. All code is type-hinted.\n\n# Installation\n\nThis project can be installed using pip:\n```bash\npip install taskiq\n```\n\nOr it can be installed directly from git:\n\n```bash\npip install git+https://github.com/taskiq-python/taskiq\n```\n\n# Usage\n\nAt first you need to create a broker. Broker is an object that can communicate to workers using distributed queues.\n\nWe have differet brokers for different queue backends. For example, we have a broker for [NATS](https://github.com/taskiq-python/taskiq-nats), [Redis](https://github.com/taskiq-python/taskiq-redis), [RabbitMQ](https://github.com/taskiq-python/taskiq-aio-pika), [Kafka](https://github.com/taskiq-python/taskiq-aio-kafka) and even more. Choose the one that fits you and create an instance.\n\n```python\nfrom taskiq_nats import JetStreamBroker\n\nbroker = JetStreamBroker(\"nats://localhost:4222\", queue=\"my_queue\")\n```\n\nDeclaring tasks is as easy as declaring a function. Just add a decorator to your function and you are ready to go.\n\n```python\nimport asyncio\n\nfrom taskiq_nats import JetStreamBroker\n\nbroker = JetStreamBroker(\"nats://localhost:4222\", queue=\"my_queue2\")\n\n\n@broker.task\nasync def my_task(a: int, b: int) -> None:\n    print(\"AB\", a + b)\n\n\nasync def main():\n    await broker.startup()\n\n    await my_task.kiq(1, 2)\n\n    await broker.shutdown()\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n\n\n```\n\nThe message is going to be sent to the broker and then to the worker. The worker will execute the function. To start worker processes, just run the following command:\n\n```bash\ntaskiq worker path.to.the.module:broker\n```\n\nWhere `path.to.the.module` is the path to the module where the broker is defined and `broker` is the name of the broker variable.\n\nIf you have tasks in different modules, you can ask taskiq to automatically import them by passing the `--fs-discover` flag:\n\n```bash\ntaskiq worker path.to.the.module:broker --fs-discover\n```\n\nIt will import all modules called `tasks.py` in the current directory and all subdirectories.\n\nAlso, we support hot reload for workers. To enable it, just pass the `--reload` flag. It will reload the worker when the code changes (To use it, install taskiq with reload extra. E.g `pip install taskiq[reload]`).\n\n\nAlso, we have cool integrations with popular async frameworks. For example, we have an integration with [FastAPI](https://taskiq-python.github.io/framework_integrations/taskiq-with-fastapi.html) or [AioHTTP](https://taskiq-python.github.io/framework_integrations/taskiq-with-aiohttp.html). You can use it to reuse dependencies from your web app in your tasks.\n\nRead about all features in our documentation: https://taskiq-python.github.io/\n\n# Local development\n\n\n## Linting\n\nWe use pre-commit to do linting locally.\n\nAfter cloning this project, please install [pre-commit](https://pre-commit.com/#install). It helps fix files before committing changes.\n\n```bash\npre-commit install\n```\n\n\n## Testing\n\nPytest can run without any additional actions or options.\n\n```bash\npytest\n```\n\n## Docs\n\nTo run docs locally, you need to install [yarn](https://yarnpkg.com/getting-started/install).\n\nFirst, you need to install dependencies.\n```\nyarn install\n```\n\nAfter that you can set up a docs server by running:\n\n```\nyarn docs:dev\n```\n",
    "bugtrack_url": null,
    "license": "LICENSE",
    "summary": "Distributed task queue with full async support",
    "version": "0.11.0",
    "project_urls": {
        "Documentation": "https://taskiq-python.github.io/",
        "Homepage": "https://taskiq-python.github.io/",
        "Repository": "https://github.com/taskiq-python/taskiq"
    },
    "split_keywords": [
        "taskiq",
        "tasks",
        "distributed",
        "async"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "df4e0b55a174fe12fef10e68f873edb90a1e9d17e2ab17dacd040ed420cb0d2d",
                "md5": "90d4c258a2aa5998bb7381d4af4442d8",
                "sha256": "9544dc661065fbf5551253425f55d0b3aae56ce4bbbedb9ea1c66e81e4532929"
            },
            "downloads": -1,
            "filename": "taskiq-0.11.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "90d4c258a2aa5998bb7381d4af4442d8",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8.1,<4.0.0",
            "size": 72747,
            "upload_time": "2024-02-05T23:38:43",
            "upload_time_iso_8601": "2024-02-05T23:38:43.770760Z",
            "url": "https://files.pythonhosted.org/packages/df/4e/0b55a174fe12fef10e68f873edb90a1e9d17e2ab17dacd040ed420cb0d2d/taskiq-0.11.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d0c4ce1c79093604ef88172f9ab2c97e6917d8a8dde26a7080d31eca6bd3d150",
                "md5": "288b22f6275c1cc4239df647efb28035",
                "sha256": "b986d9a4b83689e6e1b9525c08df54d99ded240771db41d96f96b45d67cf120b"
            },
            "downloads": -1,
            "filename": "taskiq-0.11.0.tar.gz",
            "has_sig": false,
            "md5_digest": "288b22f6275c1cc4239df647efb28035",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8.1,<4.0.0",
            "size": 50758,
            "upload_time": "2024-02-05T23:38:45",
            "upload_time_iso_8601": "2024-02-05T23:38:45.606095Z",
            "url": "https://files.pythonhosted.org/packages/d0/c4/ce1c79093604ef88172f9ab2c97e6917d8a8dde26a7080d31eca6bd3d150/taskiq-0.11.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-02-05 23:38:45",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "taskiq-python",
    "github_project": "taskiq",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "tox": true,
    "lcname": "taskiq"
}
        
Elapsed time: 0.18335s