# taskiq-memphis
This library provides you with memphis broker for taskiq.
**You need python version >=3.8**
### Usage:
```python
from taskiq_memphis import MemphisBroker
broker = MemphisBroker(
memphis_host="localhost",
username="root",
password="memphis",
)
@broker.task
async def the_best_task_ever() -> None:
print("The best task ever")
```
## Configuration
MemphisBroker parameters:
* `memphis_host` - host to memphis.
* `port` - memphis server port.
* `username` - username.
* `connection_token` - connection token.
* `password` - password for username.
* `result_backend` - custom result backend.
* `task_id_generator` - custom task_id genertaor.
* `reconnect` - turn on/off reconnection while connection is lost.
* `max_reconnect` - maximum reconnection attempts.
* `reconnect_interval_ms` - interval in milliseconds between reconnect attempts.
* `timeout_ms` - connection timeout in milliseconds.
* `cert_file` - path to tls cert file.
* `key_file` - path to tls key file.
* `ca_file` - path to tls ca file.
* `consumer_batch_size` - batch size for the consumer.
* `destroy_station_on_shutdown` - close station on shutdown.
* `destroy_producer_on_shutdown` - close producer on shutdown.
* `destroy_consumer_on_shutdown` - close consumer on shutdown.
## Non-obvious things
You can configure memphis topic, consumer, producer and `produce` method with:
```python
# Create broker
broker = MemphisBroker(
memphis_host="localhost",
username="root",
password="memphis",
destroy_station_on_shutdown=True,
)
# Configure station
broker.configure_station(...)
# Configure producer
broker.configure_producer(...)
# Configure produce method
broker.configure_produce_method(...)
# Configure consumer
broker.configure_consumer(...)
```
Memphis `station` parameters you can configure:
* `name` - name of the station. Required.
* `retention_type` - type of message retention.
* `retention_value` - how long it takes to keep message, based on retention_type.
* `storage_type` - type of the storage, DISK or MEMORY.
* `replicas` - number of replicas.
* `idempotency_window_ms` - time frame in which idempotent messages will be tracked.
* `schema_name` - name of the schema. (You can create it only via memphis UI now)
* `send_poison_msg_to_dls` - send poisoned message to dead letter station or not.
* `send_schema_failed_msg_to_dls` - send schema failed message to dead letter station or not.
* `tiered_storage_enabled` - tiered storage enabled or not.
Memphis `producer` parameters you can configure:
* `producer_name` - producer name. Required.
* `generate_random_suffix` - add suffix to producer name. Default - `True`.
**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE PRODUCER.**
Memphis `produce` method parameters you can configure:
* `ack_wait_sec` - wait ack time in seconds.
* `headers` - `Headers` instance from memphis.
* `async_produce` - produce message in async way or not.
Memphis `consumer` parameters you can configure:
* `consumer_name` - name of the consumer. Required.
* `consumer_group` - name of the consumer group.
* `pull_interval_ms` - interval in milliseconds between pulls.
* `batch_size` - pull batch size.
* `batch_max_time_to_wait_ms` - max time in milliseconds to wait between pulls.
* `max_ack_time_ms` - max time for ack a message in milliseconds.
* `max_msg_deliveries` - max number of message deliveries.
* `generate_random_suffix` - concatenate a random suffix to consumer's name.
**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE CONSUMER.**
* `start_consume_from_sequence` - start consuming from a specific sequence.
* `last_messages` - consume the last N messages.
Raw data
{
"_id": null,
"home_page": "",
"name": "taskiq-memphis",
"maintainer": "Taskiq team",
"docs_url": null,
"requires_python": ">=3.8.1,<4.0.0",
"maintainer_email": "taskiq@no-reply.com",
"keywords": "taskiq,tasks,distributed,async,memphis,memphis-py",
"author": "Taskiq team",
"author_email": "taskiq@no-reply.com",
"download_url": "https://files.pythonhosted.org/packages/1a/b1/238e44ad6256fd07a11a6e180f24d4654e0cbef0b197f07fb25dc442ba61/taskiq_memphis-0.2.0.tar.gz",
"platform": null,
"description": "# taskiq-memphis\n\nThis library provides you with memphis broker for taskiq.\n**You need python version >=3.8**\n\n### Usage:\n```python\nfrom taskiq_memphis import MemphisBroker\n\nbroker = MemphisBroker(\n memphis_host=\"localhost\",\n username=\"root\",\n password=\"memphis\",\n)\n\n\n@broker.task\nasync def the_best_task_ever() -> None:\n print(\"The best task ever\")\n```\n\n## Configuration\n\nMemphisBroker parameters:\n* `memphis_host` - host to memphis.\n* `port` - memphis server port.\n* `username` - username.\n* `connection_token` - connection token.\n* `password` - password for username.\n* `result_backend` - custom result backend.\n* `task_id_generator` - custom task_id genertaor.\n* `reconnect` - turn on/off reconnection while connection is lost.\n* `max_reconnect` - maximum reconnection attempts.\n* `reconnect_interval_ms` - interval in milliseconds between reconnect attempts.\n* `timeout_ms` - connection timeout in milliseconds.\n* `cert_file` - path to tls cert file.\n* `key_file` - path to tls key file.\n* `ca_file` - path to tls ca file.\n* `consumer_batch_size` - batch size for the consumer.\n* `destroy_station_on_shutdown` - close station on shutdown.\n* `destroy_producer_on_shutdown` - close producer on shutdown.\n* `destroy_consumer_on_shutdown` - close consumer on shutdown.\n\n\n## Non-obvious things\n\nYou can configure memphis topic, consumer, producer and `produce` method with:\n```python\n# Create broker\nbroker = MemphisBroker(\n memphis_host=\"localhost\",\n username=\"root\",\n password=\"memphis\",\n destroy_station_on_shutdown=True,\n)\n\n# Configure station\nbroker.configure_station(...)\n\n# Configure producer\nbroker.configure_producer(...)\n\n# Configure produce method\nbroker.configure_produce_method(...)\n\n# Configure consumer\nbroker.configure_consumer(...)\n```\n\nMemphis `station` parameters you can configure:\n* `name` - name of the station. Required.\n* `retention_type` - type of message retention.\n* `retention_value` - how long it takes to keep message, based on retention_type.\n* `storage_type` - type of the storage, DISK or MEMORY.\n* `replicas` - number of replicas.\n* `idempotency_window_ms` - time frame in which idempotent messages will be tracked.\n* `schema_name` - name of the schema. (You can create it only via memphis UI now)\n* `send_poison_msg_to_dls` - send poisoned message to dead letter station or not.\n* `send_schema_failed_msg_to_dls` - send schema failed message to dead letter station or not.\n* `tiered_storage_enabled` - tiered storage enabled or not.\n\nMemphis `producer` parameters you can configure:\n* `producer_name` - producer name. Required.\n* `generate_random_suffix` - add suffix to producer name. Default - `True`.\n**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE PRODUCER.**\n\nMemphis `produce` method parameters you can configure:\n* `ack_wait_sec` - wait ack time in seconds.\n* `headers` - `Headers` instance from memphis.\n* `async_produce` - produce message in async way or not.\n\nMemphis `consumer` parameters you can configure:\n\n* `consumer_name` - name of the consumer. Required.\n* `consumer_group` - name of the consumer group.\n* `pull_interval_ms` - interval in milliseconds between pulls.\n* `batch_size` - pull batch size.\n* `batch_max_time_to_wait_ms` - max time in milliseconds to wait between pulls.\n* `max_ack_time_ms` - max time for ack a message in milliseconds.\n* `max_msg_deliveries` - max number of message deliveries.\n* `generate_random_suffix` - concatenate a random suffix to consumer's name.\n**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE CONSUMER.**\n* `start_consume_from_sequence` - start consuming from a specific sequence.\n* `last_messages` - consume the last N messages.\n",
"bugtrack_url": null,
"license": "LICENSE",
"summary": "Memphis broker for taskiq",
"version": "0.2.0",
"project_urls": null,
"split_keywords": [
"taskiq",
"tasks",
"distributed",
"async",
"memphis",
"memphis-py"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b6128691cb405ae496ba1f4df463d814cdcf59bda90719ec6eb4e34d1156a67b",
"md5": "083e995cc4ea4e2432a49a5106c7a435",
"sha256": "29dd06407136f55452470ac737cb82b6624b987c526dde1a533dfe8966c8399c"
},
"downloads": -1,
"filename": "taskiq_memphis-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "083e995cc4ea4e2432a49a5106c7a435",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.1,<4.0.0",
"size": 8258,
"upload_time": "2023-06-11T13:05:22",
"upload_time_iso_8601": "2023-06-11T13:05:22.730156Z",
"url": "https://files.pythonhosted.org/packages/b6/12/8691cb405ae496ba1f4df463d814cdcf59bda90719ec6eb4e34d1156a67b/taskiq_memphis-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1ab1238e44ad6256fd07a11a6e180f24d4654e0cbef0b197f07fb25dc442ba61",
"md5": "d0cbacb97582f7adab5c24e724a0292b",
"sha256": "47bd6e737733f0e97dd965e1fef5b37d9f4015316d7fa5cb3250077ef08232e5"
},
"downloads": -1,
"filename": "taskiq_memphis-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "d0cbacb97582f7adab5c24e724a0292b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.1,<4.0.0",
"size": 7750,
"upload_time": "2023-06-11T13:05:24",
"upload_time_iso_8601": "2023-06-11T13:05:24.044017Z",
"url": "https://files.pythonhosted.org/packages/1a/b1/238e44ad6256fd07a11a6e180f24d4654e0cbef0b197f07fb25dc442ba61/taskiq_memphis-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-11 13:05:24",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "taskiq-memphis"
}
JSON