# django-simpletask3
基于Redis及SimQ的异步任务处理。
## 安装
```shell
pip install django-simpletask3
```
## 使用
### SimQ引擎设置
在*pro/settings.py*中设置，或在环境变量中设置：`DJANGO_SIMPLETASK3_SIMQ_REDIS_URL`。一般来说，各业务系统都需要根据业务系统实际情况设置该配置项值。
### 在*pro/settings.py*中引入`django_simpletask3`
```python
INSTALLED_APPS = [
...
"django_simpletask3",
...
]
```
### 在*app/models.py*中定义异步任务列表
```python
from django.db import models
from django_simpletask3.models import SimpleTask
class Task1(SimpleTask):
# 每个工作节点启动的工作线程数
django_simpletask3_worker_number = 2
task_biz_field = models.CharField(max_length=64, null=True, blank=True)
def do_task(self):
"""异步任务处理主函数。
"""
self.task_biz_field = "task biz data"
return "OK"
```
### 启动工作节点
```shell
python manage.py django-simpletask3-server
```
### 异步任务执行过程
- 当任务保存时，并自动往SimQ队列中推送一个消息。
- 工作节点取得消息后，执行异步任务处理主函数。
- 如果异步任务处理主函数没有抛出异常，则认为任务被成功处理，处理主函数的返回结果将被记录在`result_data`字段中。
- 如果异步任务处理主函数中抛出异常，则认为任务处理失败，错误信息将被记录任务的`error_data`字段中。
## 配置选项
- DJANGO_SIMPLETASK3_SIMQ_REDIS_URL = "redis://localhost:6379/0"
- （配置项别名）
- DJANGO_SIMPLETASK3_SIMQ_REDIS
- SIMQ_REDIS_URL
- SIMQ_REDIS
- REDIS_URL
- REDIS
- DJANGO_SIMPLETASK3_SIMQ_POP_TIMEOUT = 5
- 工作线程从SimQ中获取消息的超时时长。如果设置很长，则结束节点时需要等待的时长也会很长。
- DJANGO_SIMPLETASK3_DEFAULT_SIMQ_WORKER_NUMBER = 5
- 未在异步任务列表模型中指定`django_simpletask3_worker_number`时默认的每节点工作线程数。
- DJANGO_SIMPLETASK3_SIMQ_TIMEOUT_TASK_RECOVERY_INTERVAL = 60 * 2
- 超时任务回收频率。默认每两分钟执行一次任务扫描，并回收已经超时的任务。
- DJANGO_SIMPLETASK3_SIMQ_DEFAULT_RUNNING_TIMEOUT_ACTION = "recover"
- 超时任务处理策略。默认为：recover表示回收。
- 其它选项有：drop表示丢弃。
- DJANGO_SIMPLETASK3_SIMQ_PREFIX = "simq"
- DJANGO_SIMPLETASK3_SIMQ_ACK_EVENT_EXPIRE = 60 * 60 * 24
- DJANGO_SIMPLETASK3_SIMQ_DONE_ITEM_EXPIRE = 60 * 60 * 24 * 7
- DJANGO_SIMPLETASK3_SIMQ_WORKER_STATUS_EXPIRE = 60 * 5
- DJANGO_SIMPLETASK3_SIMQ_RUNNING_TIMEOUT = 60 * 5
- DJANGO_SIMPLETASK3_SIMQ_RUNNING_TIMEOUT_HANDLER_POLICIES = None
- DJANGO_SIMPLETASK3_EVENT_QUEUE_CHANNEL_TEMPLATE = "django-simpletask3:{app_label}.{model_name}"
- DJANGO_SIMPLETASK3_TASK_ID_TEMPLATE = "django-simpletask3:{app_label}.{model_name}:{id}"
## 版本记录
### v0.1.0
- 版本首发。
- 基于Redis及SimQ的异步任务处理框架基础功能。
### v0.1.1
- 添加SimpleTaskAdmin基础类。
- 添加django_simpletask3_execute_for_selected和django_simpletask3_reset_for_selected批处理动作。
- 修改SimpleTask.execute执行机制。
### v0.1.2
- `SimpleTaskAdmin`中添加相关`actions`。
Raw data
{
"_id": null,
"home_page": null,
"name": "django-simpletask3",
"maintainer": "rRR0VrFP",
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "django-simpletask3, simqcore",
"author": "rRR0VrFP",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/d1/78/237dc78fa296bcf7b0ed1dc55d4c609221b12da3ad53c9e406721533257a/django-simpletask3-0.1.2.tar.gz",
"platform": null,
"description": "# django-simpletask3\n\n\u57fa\u4e8eRedis\u53caSimQ\u7684\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u3002\n\n## \u5b89\u88c5\n\n```shell\npip install django-simpletask3\n```\n\n## \u4f7f\u7528\n\n### SimQ\u5f15\u64ce\u8bbe\u7f6e\n\n\u5728*pro/settings.py*\u4e2d\u8bbe\u7f6e\uff0c\u6216\u5728\u73af\u5883\u53d8\u91cf\u4e2d\u8bbe\u7f6e\uff1a`DJANGO_SIMPLETASK3_SIMQ_REDIS_URL`\u3002\u4e00\u822c\u6765\u8bf4\uff0c\u5404\u4e1a\u52a1\u7cfb\u7edf\u90fd\u9700\u8981\u6839\u636e\u4e1a\u52a1\u7cfb\u7edf\u5b9e\u9645\u60c5\u51b5\u8bbe\u7f6e\u8be5\u914d\u7f6e\u9879\u503c\u3002\n\n### \u5728*pro/settings.py*\u4e2d\u5f15\u5165`django_simpletask3`\n\n```python\nINSTALLED_APPS = [\n ...\n \"django_simpletask3\",\n ...\n]\n```\n\n### \u5728*app/models.py*\u4e2d\u5b9a\u4e49\u5f02\u6b65\u4efb\u52a1\u5217\u8868\n\n```python\nfrom django.db import models\nfrom django_simpletask3.models import SimpleTask\n\n\nclass Task1(SimpleTask):\n\n # \u6bcf\u4e2a\u5de5\u4f5c\u8282\u70b9\u542f\u52a8\u7684\u5de5\u4f5c\u7ebf\u7a0b\u6570\n django_simpletask3_worker_number = 2\n\n task_biz_field = models.CharField(max_length=64, null=True, blank=True)\n\n def do_task(self):\n \"\"\"\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u4e3b\u51fd\u6570\u3002\n \"\"\"\n self.task_biz_field = \"task biz data\"\n return \"OK\"\n```\n\n### \u542f\u52a8\u5de5\u4f5c\u8282\u70b9\n\n```shell\npython manage.py django-simpletask3-server\n```\n\n### \u5f02\u6b65\u4efb\u52a1\u6267\u884c\u8fc7\u7a0b\n\n- \u5f53\u4efb\u52a1\u4fdd\u5b58\u65f6\uff0c\u5e76\u81ea\u52a8\u5f80SimQ\u961f\u5217\u4e2d\u63a8\u9001\u4e00\u4e2a\u6d88\u606f\u3002\n- \u5de5\u4f5c\u8282\u70b9\u53d6\u5f97\u6d88\u606f\u540e\uff0c\u6267\u884c\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u4e3b\u51fd\u6570\u3002\n- \u5982\u679c\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u4e3b\u51fd\u6570\u6ca1\u6709\u629b\u51fa\u5f02\u5e38\uff0c\u5219\u8ba4\u4e3a\u4efb\u52a1\u88ab\u6210\u529f\u5904\u7406\uff0c\u5904\u7406\u4e3b\u51fd\u6570\u7684\u8fd4\u56de\u7ed3\u679c\u5c06\u88ab\u8bb0\u5f55\u5728`result_data`\u5b57\u6bb5\u4e2d\u3002\n- \u5982\u679c\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u4e3b\u51fd\u6570\u4e2d\u629b\u51fa\u5f02\u5e38\uff0c\u5219\u8ba4\u4e3a\u4efb\u52a1\u5904\u7406\u5931\u8d25\uff0c\u9519\u8bef\u4fe1\u606f\u5c06\u88ab\u8bb0\u5f55\u4efb\u52a1\u7684`error_data`\u5b57\u6bb5\u4e2d\u3002\n\n## \u914d\u7f6e\u9009\u9879\n\n\n- DJANGO_SIMPLETASK3_SIMQ_REDIS_URL = \"redis://localhost:6379/0\"\n - \uff08\u914d\u7f6e\u9879\u522b\u540d\uff09\n - DJANGO_SIMPLETASK3_SIMQ_REDIS\n - SIMQ_REDIS_URL\n - SIMQ_REDIS\n - REDIS_URL\n - REDIS\n- DJANGO_SIMPLETASK3_SIMQ_POP_TIMEOUT = 5\n - \u5de5\u4f5c\u7ebf\u7a0b\u4eceSimQ\u4e2d\u83b7\u53d6\u6d88\u606f\u7684\u8d85\u65f6\u65f6\u957f\u3002\u5982\u679c\u8bbe\u7f6e\u5f88\u957f\uff0c\u5219\u7ed3\u675f\u8282\u70b9\u65f6\u9700\u8981\u7b49\u5f85\u7684\u65f6\u957f\u4e5f\u4f1a\u5f88\u957f\u3002\n- DJANGO_SIMPLETASK3_DEFAULT_SIMQ_WORKER_NUMBER = 5\n - \u672a\u5728\u5f02\u6b65\u4efb\u52a1\u5217\u8868\u6a21\u578b\u4e2d\u6307\u5b9a`django_simpletask3_worker_number`\u65f6\u9ed8\u8ba4\u7684\u6bcf\u8282\u70b9\u5de5\u4f5c\u7ebf\u7a0b\u6570\u3002\n- DJANGO_SIMPLETASK3_SIMQ_TIMEOUT_TASK_RECOVERY_INTERVAL = 60 * 2\n - \u8d85\u65f6\u4efb\u52a1\u56de\u6536\u9891\u7387\u3002\u9ed8\u8ba4\u6bcf\u4e24\u5206\u949f\u6267\u884c\u4e00\u6b21\u4efb\u52a1\u626b\u63cf\uff0c\u5e76\u56de\u6536\u5df2\u7ecf\u8d85\u65f6\u7684\u4efb\u52a1\u3002\n- DJANGO_SIMPLETASK3_SIMQ_DEFAULT_RUNNING_TIMEOUT_ACTION = \"recover\"\n - \u8d85\u65f6\u4efb\u52a1\u5904\u7406\u7b56\u7565\u3002\u9ed8\u8ba4\u4e3a\uff1arecover\u8868\u793a\u56de\u6536\u3002\n - \u5176\u5b83\u9009\u9879\u6709\uff1adrop\u8868\u793a\u4e22\u5f03\u3002\n- DJANGO_SIMPLETASK3_SIMQ_PREFIX = \"simq\"\n- DJANGO_SIMPLETASK3_SIMQ_ACK_EVENT_EXPIRE = 60 * 60 * 24\n- DJANGO_SIMPLETASK3_SIMQ_DONE_ITEM_EXPIRE = 60 * 60 * 24 * 7\n- DJANGO_SIMPLETASK3_SIMQ_WORKER_STATUS_EXPIRE = 60 * 5\n- DJANGO_SIMPLETASK3_SIMQ_RUNNING_TIMEOUT = 60 * 5\n- DJANGO_SIMPLETASK3_SIMQ_RUNNING_TIMEOUT_HANDLER_POLICIES = None\n- DJANGO_SIMPLETASK3_EVENT_QUEUE_CHANNEL_TEMPLATE = \"django-simpletask3:{app_label}.{model_name}\"\n- DJANGO_SIMPLETASK3_TASK_ID_TEMPLATE = \"django-simpletask3:{app_label}.{model_name}:{id}\"\n\n## \u7248\u672c\u8bb0\u5f55\n\n### v0.1.0\n\n- \u7248\u672c\u9996\u53d1\u3002\n- \u57fa\u4e8eRedis\u53caSimQ\u7684\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u6846\u67b6\u57fa\u7840\u529f\u80fd\u3002\n\n### v0.1.1\n\n- \u6dfb\u52a0SimpleTaskAdmin\u57fa\u7840\u7c7b\u3002\n- \u6dfb\u52a0django_simpletask3_execute_for_selected\u548cdjango_simpletask3_reset_for_selected\u6279\u5904\u7406\u52a8\u4f5c\u3002\n- \u4fee\u6539SimpleTask.execute\u6267\u884c\u673a\u5236\u3002\n\n### v0.1.2\n\n- `SimpleTaskAdmin`\u4e2d\u6dfb\u52a0\u76f8\u5173`actions`\u3002\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "\u57fa\u4e8eRedis\u53caSimQ\u7684\u5f02\u6b65\u4efb\u52a1\u5904\u7406\u3002",
"version": "0.1.2",
"project_urls": null,
"split_keywords": [
"django-simpletask3",
" simqcore"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2b5db48296c6c94273001a5f07686c6750cee52997d92ad8808b55b405c442b3",
"md5": "d48b6a904af4e83457ea6ea1924417b0",
"sha256": "4871af097d5cfba0f9d8e4f7517ab88a0972b723403087dfd7601799517eaff7"
},
"downloads": -1,
"filename": "django_simpletask3-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d48b6a904af4e83457ea6ea1924417b0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 11748,
"upload_time": "2024-12-27T11:38:44",
"upload_time_iso_8601": "2024-12-27T11:38:44.872949Z",
"url": "https://files.pythonhosted.org/packages/2b/5d/b48296c6c94273001a5f07686c6750cee52997d92ad8808b55b405c442b3/django_simpletask3-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d178237dc78fa296bcf7b0ed1dc55d4c609221b12da3ad53c9e406721533257a",
"md5": "6e10b485413ed13c6c2d684831834de3",
"sha256": "de1bcd04de1dc46051bc62739aa401176790ef2064c61e1bdf339341ad8d6988"
},
"downloads": -1,
"filename": "django-simpletask3-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "6e10b485413ed13c6c2d684831834de3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10345,
"upload_time": "2024-12-27T11:38:46",
"upload_time_iso_8601": "2024-12-27T11:38:46.919456Z",
"url": "https://files.pythonhosted.org/packages/d1/78/237dc78fa296bcf7b0ed1dc55d4c609221b12da3ad53c9e406721533257a/django-simpletask3-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-27 11:38:46",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "django-simpletask3"
}
JSON