# 任务退避重试框架
一个基于Redis的任务退避重试框架，支持多种退避策略和自定义配置。
## 功能特点
- 基于Redis的分布式任务队列
- 支持多种退避策略（固定间隔、指数退避、线性退避等）
- 支持任务优先级
- 支持任务超时和失败处理
- 支持并发执行任务（线程池/进程池）
- 支持任务状态管理和监控
## 打包上传
```bash
python setup.py sdist bdist_wheel
twine upload dist/*
更新包
pip3 install --upgrade .
```
## 安装说明
### 使用pip安装
```bash
pip install backoff_scheduler
```
### 从源码安装
```bash
# 克隆项目
git clone https://github.com/yourusername/backoff_scheduler.git
cd backoff_scheduler
# 安装依赖
pip install -r requirements.txt
# 安装包
python setup.py install
```
## 快速开始
```python
from backoff_scheduler import TaskBackoffFramework, TaskEntity, BackoffConfig
from backoff_scheduler.common.task_entity import BackoffStrategy
# 初始化配置
config = BackoffConfig(
redis_host='localhost',
redis_port=6379,
redis_db=0,
backoff_strategy=BackoffStrategy.EXPONENTIAL,
max_retries=5,
initial_delay=1000,
max_delay=30000
)
# 初始化框架
framework = TaskBackoffFramework(config)
# 定义任务处理函数
def task_handler(task_entity, params):
# 任务处理逻辑
print(f"处理任务: {task_entity.task_id}, 参数: {params}")
# 模拟任务成功
return {"status": "success", "result": "任务处理完成"}
# 设置任务处理器
framework.set_task_handler(task_handler)
# 创建任务
params = {"data": "需要处理的数据"}
task_id = framework.create_task(params)
# 启动框架
framework.start()
# 等待任务完成
import time
time.sleep(10)
# 获取任务结果
result = framework.get_task_result(task_id)
print(f"任务结果: {result}")
# 停止框架
framework.stop()
```
## 配置说明
### BackoffConfig 参数
- `redis_host`: Redis主机地址
- `redis_port`: Redis端口
- `redis_db`: Redis数据库索引
- `backoff_strategy`: 退避策略（FIXED, EXPONENTIAL, LINEAR）
- `max_retries`: 最大重试次数
- `initial_delay`: 初始延迟时间（毫秒）
- `max_delay`: 最大延迟时间（毫秒）
- `executor_type`: 执行器类型（THREAD, PROCESS）
- `max_workers`: 最大工作线程/进程数
## 高级使用
### 自定义退避策略
```python
from backoff_scheduler.common.task_entity import BackoffStrategy
from backoff_scheduler.core.backoff_strategy import BaseBackoffStrategy
class CustomBackoffStrategy(BaseBackoffStrategy):
def calculate_delay(self, retry_count):
# 自定义退避算法
return min(1000 * (retry_count ** 2), 30000)
# 注册自定义策略
BackoffStrategy.register('CUSTOM', CustomBackoffStrategy)
# 使用自定义策略
config = BackoffConfig(
backoff_strategy=BackoffStrategy.CUSTOM,
# 其他配置...
)
```
### 任务异常处理
```python
def exception_handler(task_entity, params):
# 异常处理逻辑
print(f"任务 {task_entity.task_id} 执行失败: {params}")
# 设置异常处理器
framework.set_exception_handler(exception_handler)
```
## 项目结构
```
app/
├── common/ # 公共模块
├── conf/ # 配置文件
├── core/ # 核心功能
├── models/ # 数据模型
├── scheduler/ # 调度器
└── utils/ # 工具函数
```
## 贡献指南
1. Fork 项目
2. 创建特性分支 (`git checkout -b feature/fooBar`)
3. 提交更改 (`git commit -am 'Add some fooBar'`)
4. 推送到分支 (`git push origin feature/fooBar`)
5. 创建新的 Pull Request
## 许可证
本项目采用 MIT 许可证 - 详情请见 LICENSE 文件
## 联系方式
如有问题或建议，请联系: xinzf@ucap.com.cn
Raw data
{
"_id": null,
"home_page": "http://192.168.3.118:8082/maas/maas-backoff_scheduler/",
"name": "maas-backoff-scheduler",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "task retry backoff redis queue",
"author": "xinzhifu",
"author_email": "515361725@qq.com",
"download_url": "https://files.pythonhosted.org/packages/b7/21/5da825615d19d6b3b4d91cd14b4e8b62bc603f746fa8a0560dce1416b9c6/maas_backoff_scheduler-1.0.22.tar.gz",
"platform": null,
"description": "# \u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\n\n\u4e00\u4e2a\u57fa\u4e8eRedis\u7684\u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\uff0c\u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\u548c\u81ea\u5b9a\u4e49\u914d\u7f6e\u3002\n\n## \u529f\u80fd\u7279\u70b9\n- \u57fa\u4e8eRedis\u7684\u5206\u5e03\u5f0f\u4efb\u52a1\u961f\u5217\n- \u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\uff08\u56fa\u5b9a\u95f4\u9694\u3001\u6307\u6570\u9000\u907f\u3001\u7ebf\u6027\u9000\u907f\u7b49\uff09\n- \u652f\u6301\u4efb\u52a1\u4f18\u5148\u7ea7\n- \u652f\u6301\u4efb\u52a1\u8d85\u65f6\u548c\u5931\u8d25\u5904\u7406\n- \u652f\u6301\u5e76\u53d1\u6267\u884c\u4efb\u52a1\uff08\u7ebf\u7a0b\u6c60/\u8fdb\u7a0b\u6c60\uff09\n- \u652f\u6301\u4efb\u52a1\u72b6\u6001\u7ba1\u7406\u548c\u76d1\u63a7\n\n## \u6253\u5305\u4e0a\u4f20\n\n```bash\npython setup.py sdist bdist_wheel\n\ntwine upload dist/*\n\n\u66f4\u65b0\u5305\npip3 install --upgrade .\n```\n## \u5b89\u88c5\u8bf4\u660e\n\n### \u4f7f\u7528pip\u5b89\u88c5\n```bash\npip install backoff_scheduler\n```\n\n### \u4ece\u6e90\u7801\u5b89\u88c5\n```bash\n# \u514b\u9686\u9879\u76ee\ngit clone https://github.com/yourusername/backoff_scheduler.git\ncd backoff_scheduler\n\n# \u5b89\u88c5\u4f9d\u8d56\npip install -r requirements.txt\n\n# \u5b89\u88c5\u5305\npython setup.py install\n```\n\n## \u5feb\u901f\u5f00\u59cb\n\n```python\nfrom backoff_scheduler import TaskBackoffFramework, TaskEntity, BackoffConfig\nfrom backoff_scheduler.common.task_entity import BackoffStrategy\n\n# \u521d\u59cb\u5316\u914d\u7f6e\nconfig = BackoffConfig(\n redis_host='localhost',\n redis_port=6379,\n redis_db=0,\n backoff_strategy=BackoffStrategy.EXPONENTIAL,\n max_retries=5,\n initial_delay=1000,\n max_delay=30000\n)\n\n# \u521d\u59cb\u5316\u6846\u67b6\nframework = TaskBackoffFramework(config)\n\n# \u5b9a\u4e49\u4efb\u52a1\u5904\u7406\u51fd\u6570\ndef task_handler(task_entity, params):\n # \u4efb\u52a1\u5904\u7406\u903b\u8f91\n print(f\"\u5904\u7406\u4efb\u52a1: {task_entity.task_id}, \u53c2\u6570: {params}\")\n # \u6a21\u62df\u4efb\u52a1\u6210\u529f\n return {\"status\": \"success\", \"result\": \"\u4efb\u52a1\u5904\u7406\u5b8c\u6210\"}\n\n# \u8bbe\u7f6e\u4efb\u52a1\u5904\u7406\u5668\nframework.set_task_handler(task_handler)\n\n# \u521b\u5efa\u4efb\u52a1\nparams = {\"data\": \"\u9700\u8981\u5904\u7406\u7684\u6570\u636e\"}\n task_id = framework.create_task(params)\n\n# \u542f\u52a8\u6846\u67b6\nframework.start()\n\n# \u7b49\u5f85\u4efb\u52a1\u5b8c\u6210\nimport time\ntime.sleep(10)\n\n# \u83b7\u53d6\u4efb\u52a1\u7ed3\u679c\nresult = framework.get_task_result(task_id)\nprint(f\"\u4efb\u52a1\u7ed3\u679c: {result}\")\n\n# \u505c\u6b62\u6846\u67b6\nframework.stop()\n```\n\n## \u914d\u7f6e\u8bf4\u660e\n\n### BackoffConfig \u53c2\u6570\n- `redis_host`: Redis\u4e3b\u673a\u5730\u5740\n- `redis_port`: Redis\u7aef\u53e3\n- `redis_db`: Redis\u6570\u636e\u5e93\u7d22\u5f15\n- `backoff_strategy`: \u9000\u907f\u7b56\u7565\uff08FIXED, EXPONENTIAL, LINEAR\uff09\n- `max_retries`: \u6700\u5927\u91cd\u8bd5\u6b21\u6570\n- `initial_delay`: \u521d\u59cb\u5ef6\u8fdf\u65f6\u95f4\uff08\u6beb\u79d2\uff09\n- `max_delay`: \u6700\u5927\u5ef6\u8fdf\u65f6\u95f4\uff08\u6beb\u79d2\uff09\n- `executor_type`: \u6267\u884c\u5668\u7c7b\u578b\uff08THREAD, PROCESS\uff09\n- `max_workers`: \u6700\u5927\u5de5\u4f5c\u7ebf\u7a0b/\u8fdb\u7a0b\u6570\n\n## \u9ad8\u7ea7\u4f7f\u7528\n\n### \u81ea\u5b9a\u4e49\u9000\u907f\u7b56\u7565\n```python\nfrom backoff_scheduler.common.task_entity import BackoffStrategy\nfrom backoff_scheduler.core.backoff_strategy import BaseBackoffStrategy\n\nclass CustomBackoffStrategy(BaseBackoffStrategy):\n def calculate_delay(self, retry_count):\n # \u81ea\u5b9a\u4e49\u9000\u907f\u7b97\u6cd5\n return min(1000 * (retry_count ** 2), 30000)\n\n# \u6ce8\u518c\u81ea\u5b9a\u4e49\u7b56\u7565\nBackoffStrategy.register('CUSTOM', CustomBackoffStrategy)\n\n# \u4f7f\u7528\u81ea\u5b9a\u4e49\u7b56\u7565\nconfig = BackoffConfig(\n backoff_strategy=BackoffStrategy.CUSTOM,\n # \u5176\u4ed6\u914d\u7f6e...\n)\n```\n\n### \u4efb\u52a1\u5f02\u5e38\u5904\u7406\n```python\ndef exception_handler(task_entity, params):\n # \u5f02\u5e38\u5904\u7406\u903b\u8f91\n print(f\"\u4efb\u52a1 {task_entity.task_id} \u6267\u884c\u5931\u8d25: {params}\")\n\n# \u8bbe\u7f6e\u5f02\u5e38\u5904\u7406\u5668\nframework.set_exception_handler(exception_handler)\n```\n\n## \u9879\u76ee\u7ed3\u6784\n```\napp/\n\u251c\u2500\u2500 common/ # \u516c\u5171\u6a21\u5757\n\u251c\u2500\u2500 conf/ # \u914d\u7f6e\u6587\u4ef6\n\u251c\u2500\u2500 core/ # \u6838\u5fc3\u529f\u80fd\n\u251c\u2500\u2500 models/ # \u6570\u636e\u6a21\u578b\n\u251c\u2500\u2500 scheduler/ # \u8c03\u5ea6\u5668\n\u2514\u2500\u2500 utils/ # \u5de5\u5177\u51fd\u6570\n```\n\n## \u8d21\u732e\u6307\u5357\n1. Fork \u9879\u76ee\n2. \u521b\u5efa\u7279\u6027\u5206\u652f (`git checkout -b feature/fooBar`)\n3. \u63d0\u4ea4\u66f4\u6539 (`git commit -am 'Add some fooBar'`)\n4. \u63a8\u9001\u5230\u5206\u652f (`git push origin feature/fooBar`)\n5. \u521b\u5efa\u65b0\u7684 Pull Request\n\n## \u8bb8\u53ef\u8bc1\n\u672c\u9879\u76ee\u91c7\u7528 MIT \u8bb8\u53ef\u8bc1 - \u8be6\u60c5\u8bf7\u89c1 LICENSE \u6587\u4ef6\n\n## \u8054\u7cfb\u65b9\u5f0f\n\u5982\u6709\u95ee\u9898\u6216\u5efa\u8bae\uff0c\u8bf7\u8054\u7cfb: xinzf@ucap.com.cn\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "\u4e00\u4e2a\u57fa\u4e8eRedis\u7684\u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\uff0c\u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\u548c\u81ea\u5b9a\u4e49\u914d\u7f6e",
"version": "1.0.22",
"project_urls": {
"Bug Reports": "https://github.com/yourusername/task-backoff-scheduler/issues",
"Documentation": "https://github.com/yourusername/task-backoff-scheduler#readme",
"Homepage": "http://192.168.3.118:8082/maas/maas-backoff_scheduler/",
"Source": "https://github.com/yourusername/task-backoff-scheduler"
},
"split_keywords": [
"task",
"retry",
"backoff",
"redis",
"queue"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "a1b8f8ff640ca063c9eee30be820badda07b168e8016fba2a43fdf98b9e415ed",
"md5": "7c9e031d0981f5e41c5638c26d6ff098",
"sha256": "5ab5a04e4b44e73505d864499d3538c05e7b82a0576769741281060b6090f932"
},
"downloads": -1,
"filename": "maas_backoff_scheduler-1.0.22-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7c9e031d0981f5e41c5638c26d6ff098",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 29896,
"upload_time": "2025-08-22T10:09:39",
"upload_time_iso_8601": "2025-08-22T10:09:39.736598Z",
"url": "https://files.pythonhosted.org/packages/a1/b8/f8ff640ca063c9eee30be820badda07b168e8016fba2a43fdf98b9e415ed/maas_backoff_scheduler-1.0.22-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b7215da825615d19d6b3b4d91cd14b4e8b62bc603f746fa8a0560dce1416b9c6",
"md5": "cff25ef54b4ce8c339b6be4b35ee1083",
"sha256": "22c3b0adab6271ba90499a8a03e89fe33c3324bf17382837f56e6479f9c0592e"
},
"downloads": -1,
"filename": "maas_backoff_scheduler-1.0.22.tar.gz",
"has_sig": false,
"md5_digest": "cff25ef54b4ce8c339b6be4b35ee1083",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 26804,
"upload_time": "2025-08-22T10:09:40",
"upload_time_iso_8601": "2025-08-22T10:09:40.935947Z",
"url": "https://files.pythonhosted.org/packages/b7/21/5da825615d19d6b3b4d91cd14b4e8b62bc603f746fa8a0560dce1416b9c6/maas_backoff_scheduler-1.0.22.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-22 10:09:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "task-backoff-scheduler",
"github_not_found": true,
"lcname": "maas-backoff-scheduler"
}
JSON
