# 任务退避重试框架
一个基于Redis的任务退避重试框架，支持多种退避策略和自定义配置。
## 功能特点
- 基于Redis的分布式任务队列
- 支持多种退避策略（固定间隔、指数退避、线性退避等）
- 支持任务优先级
- 支持任务超时和失败处理
- 支持并发执行任务（线程池/进程池）
- 支持任务状态管理和监控
## 安装说明
### 使用pip安装
```bash
pip install backoff_scheduler
```
### 从源码安装
```bash
# 克隆项目
git clone https://github.com/yourusername/backoff_scheduler.git
cd backoff_scheduler
# 安装依赖
pip install -r requirements.txt
# 安装包
python setup.py install
```
## 快速开始
```python
from backoff_scheduler import TaskBackoffFramework, TaskEntity, BackoffConfig
from backoff_scheduler.common.task_entity import BackoffStrategy
# 初始化配置
config = BackoffConfig(
redis_host='localhost',
redis_port=6379,
redis_db=0,
backoff_strategy=BackoffStrategy.EXPONENTIAL,
max_retries=5,
initial_delay=1000,
max_delay=30000
)
# 初始化框架
framework = TaskBackoffFramework(config)
# 定义任务处理函数
def task_handler(task_entity, params):
# 任务处理逻辑
print(f"处理任务: {task_entity.task_id}, 参数: {params}")
# 模拟任务成功
return {"status": "success", "result": "任务处理完成"}
# 设置任务处理器
framework.set_task_handler(task_handler)
# 创建任务
params = {"data": "需要处理的数据"}
task_id = framework.create_task(params)
# 启动框架
framework.start()
# 等待任务完成
import time
time.sleep(10)
# 获取任务结果
result = framework.get_task_result(task_id)
print(f"任务结果: {result}")
# 停止框架
framework.stop()
```
## 配置说明
### BackoffConfig 参数
- `redis_host`: Redis主机地址
- `redis_port`: Redis端口
- `redis_db`: Redis数据库索引
- `backoff_strategy`: 退避策略（FIXED, EXPONENTIAL, LINEAR）
- `max_retries`: 最大重试次数
- `initial_delay`: 初始延迟时间（毫秒）
- `max_delay`: 最大延迟时间（毫秒）
- `executor_type`: 执行器类型（THREAD, PROCESS）
- `max_workers`: 最大工作线程/进程数
## 高级使用
### 自定义退避策略
```python
from backoff_scheduler.common.task_entity import BackoffStrategy
from backoff_scheduler.core.backoff_strategy import BaseBackoffStrategy
class CustomBackoffStrategy(BaseBackoffStrategy):
def calculate_delay(self, retry_count):
# 自定义退避算法
return min(1000 * (retry_count ** 2), 30000)
# 注册自定义策略
BackoffStrategy.register('CUSTOM', CustomBackoffStrategy)
# 使用自定义策略
config = BackoffConfig(
backoff_strategy=BackoffStrategy.CUSTOM,
# 其他配置...
)
```
### 任务异常处理
```python
def exception_handler(task_entity, params):
# 异常处理逻辑
print(f"任务 {task_entity.task_id} 执行失败: {params}")
# 设置异常处理器
framework.set_exception_handler(exception_handler)
```
## 项目结构
```
app/
├── common/ # 公共模块
├── conf/ # 配置文件
├── core/ # 核心功能
├── models/ # 数据模型
├── scheduler/ # 调度器
└── utils/ # 工具函数
```
## 贡献指南
1. Fork 项目
2. 创建特性分支 (`git checkout -b feature/fooBar`)
3. 提交更改 (`git commit -am 'Add some fooBar'`)
4. 推送到分支 (`git push origin feature/fooBar`)
5. 创建新的 Pull Request
## 许可证
本项目采用 MIT 许可证 - 详情请见 LICENSE 文件
## 联系方式
如有问题或建议，请联系: xinzf@ucap.com.cn
Raw data
{
"_id": null,
"home_page": "http://192.168.3.118:8082/maas/maas-backoff_scheduler/",
"name": "backoff-scheduler",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "task retry backoff redis queue",
"author": "xinzf",
"author_email": "xinzf@ucap.com.cn",
"download_url": "https://files.pythonhosted.org/packages/d4/fa/34c76d08b854e107370de9171fc0e2b909c9996c0e66c3447b659d5fdc26/backoff_scheduler-1.0.0.tar.gz",
"platform": null,
"description": "# \u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\n\n\u4e00\u4e2a\u57fa\u4e8eRedis\u7684\u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\uff0c\u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\u548c\u81ea\u5b9a\u4e49\u914d\u7f6e\u3002\n\n## \u529f\u80fd\u7279\u70b9\n- \u57fa\u4e8eRedis\u7684\u5206\u5e03\u5f0f\u4efb\u52a1\u961f\u5217\n- \u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\uff08\u56fa\u5b9a\u95f4\u9694\u3001\u6307\u6570\u9000\u907f\u3001\u7ebf\u6027\u9000\u907f\u7b49\uff09\n- \u652f\u6301\u4efb\u52a1\u4f18\u5148\u7ea7\n- \u652f\u6301\u4efb\u52a1\u8d85\u65f6\u548c\u5931\u8d25\u5904\u7406\n- \u652f\u6301\u5e76\u53d1\u6267\u884c\u4efb\u52a1\uff08\u7ebf\u7a0b\u6c60/\u8fdb\u7a0b\u6c60\uff09\n- \u652f\u6301\u4efb\u52a1\u72b6\u6001\u7ba1\u7406\u548c\u76d1\u63a7\n\n## \u5b89\u88c5\u8bf4\u660e\n\n### \u4f7f\u7528pip\u5b89\u88c5\n```bash\npip install backoff_scheduler\n```\n\n### \u4ece\u6e90\u7801\u5b89\u88c5\n```bash\n# \u514b\u9686\u9879\u76ee\ngit clone https://github.com/yourusername/backoff_scheduler.git\ncd backoff_scheduler\n\n# \u5b89\u88c5\u4f9d\u8d56\npip install -r requirements.txt\n\n# \u5b89\u88c5\u5305\npython setup.py install\n```\n\n## \u5feb\u901f\u5f00\u59cb\n\n```python\nfrom backoff_scheduler import TaskBackoffFramework, TaskEntity, BackoffConfig\nfrom backoff_scheduler.common.task_entity import BackoffStrategy\n\n# \u521d\u59cb\u5316\u914d\u7f6e\nconfig = BackoffConfig(\n redis_host='localhost',\n redis_port=6379,\n redis_db=0,\n backoff_strategy=BackoffStrategy.EXPONENTIAL,\n max_retries=5,\n initial_delay=1000,\n max_delay=30000\n)\n\n# \u521d\u59cb\u5316\u6846\u67b6\nframework = TaskBackoffFramework(config)\n\n# \u5b9a\u4e49\u4efb\u52a1\u5904\u7406\u51fd\u6570\ndef task_handler(task_entity, params):\n # \u4efb\u52a1\u5904\u7406\u903b\u8f91\n print(f\"\u5904\u7406\u4efb\u52a1: {task_entity.task_id}, \u53c2\u6570: {params}\")\n # \u6a21\u62df\u4efb\u52a1\u6210\u529f\n return {\"status\": \"success\", \"result\": \"\u4efb\u52a1\u5904\u7406\u5b8c\u6210\"}\n\n# \u8bbe\u7f6e\u4efb\u52a1\u5904\u7406\u5668\nframework.set_task_handler(task_handler)\n\n# \u521b\u5efa\u4efb\u52a1\nparams = {\"data\": \"\u9700\u8981\u5904\u7406\u7684\u6570\u636e\"}\n task_id = framework.create_task(params)\n\n# \u542f\u52a8\u6846\u67b6\nframework.start()\n\n# \u7b49\u5f85\u4efb\u52a1\u5b8c\u6210\nimport time\ntime.sleep(10)\n\n# \u83b7\u53d6\u4efb\u52a1\u7ed3\u679c\nresult = framework.get_task_result(task_id)\nprint(f\"\u4efb\u52a1\u7ed3\u679c: {result}\")\n\n# \u505c\u6b62\u6846\u67b6\nframework.stop()\n```\n\n## \u914d\u7f6e\u8bf4\u660e\n\n### BackoffConfig \u53c2\u6570\n- `redis_host`: Redis\u4e3b\u673a\u5730\u5740\n- `redis_port`: Redis\u7aef\u53e3\n- `redis_db`: Redis\u6570\u636e\u5e93\u7d22\u5f15\n- `backoff_strategy`: \u9000\u907f\u7b56\u7565\uff08FIXED, EXPONENTIAL, LINEAR\uff09\n- `max_retries`: \u6700\u5927\u91cd\u8bd5\u6b21\u6570\n- `initial_delay`: \u521d\u59cb\u5ef6\u8fdf\u65f6\u95f4\uff08\u6beb\u79d2\uff09\n- `max_delay`: \u6700\u5927\u5ef6\u8fdf\u65f6\u95f4\uff08\u6beb\u79d2\uff09\n- `executor_type`: \u6267\u884c\u5668\u7c7b\u578b\uff08THREAD, PROCESS\uff09\n- `max_workers`: \u6700\u5927\u5de5\u4f5c\u7ebf\u7a0b/\u8fdb\u7a0b\u6570\n\n## \u9ad8\u7ea7\u4f7f\u7528\n\n### \u81ea\u5b9a\u4e49\u9000\u907f\u7b56\u7565\n```python\nfrom backoff_scheduler.common.task_entity import BackoffStrategy\nfrom backoff_scheduler.core.backoff_strategy import BaseBackoffStrategy\n\nclass CustomBackoffStrategy(BaseBackoffStrategy):\n def calculate_delay(self, retry_count):\n # \u81ea\u5b9a\u4e49\u9000\u907f\u7b97\u6cd5\n return min(1000 * (retry_count ** 2), 30000)\n\n# \u6ce8\u518c\u81ea\u5b9a\u4e49\u7b56\u7565\nBackoffStrategy.register('CUSTOM', CustomBackoffStrategy)\n\n# \u4f7f\u7528\u81ea\u5b9a\u4e49\u7b56\u7565\nconfig = BackoffConfig(\n backoff_strategy=BackoffStrategy.CUSTOM,\n # \u5176\u4ed6\u914d\u7f6e...\n)\n```\n\n### \u4efb\u52a1\u5f02\u5e38\u5904\u7406\n```python\ndef exception_handler(task_entity, params):\n # \u5f02\u5e38\u5904\u7406\u903b\u8f91\n print(f\"\u4efb\u52a1 {task_entity.task_id} \u6267\u884c\u5931\u8d25: {params}\")\n\n# \u8bbe\u7f6e\u5f02\u5e38\u5904\u7406\u5668\nframework.set_exception_handler(exception_handler)\n```\n\n## \u9879\u76ee\u7ed3\u6784\n```\napp/\n\u251c\u2500\u2500 common/ # \u516c\u5171\u6a21\u5757\n\u251c\u2500\u2500 conf/ # \u914d\u7f6e\u6587\u4ef6\n\u251c\u2500\u2500 core/ # \u6838\u5fc3\u529f\u80fd\n\u251c\u2500\u2500 models/ # \u6570\u636e\u6a21\u578b\n\u251c\u2500\u2500 scheduler/ # \u8c03\u5ea6\u5668\n\u2514\u2500\u2500 utils/ # \u5de5\u5177\u51fd\u6570\n```\n\n## \u8d21\u732e\u6307\u5357\n1. Fork \u9879\u76ee\n2. \u521b\u5efa\u7279\u6027\u5206\u652f (`git checkout -b feature/fooBar`)\n3. \u63d0\u4ea4\u66f4\u6539 (`git commit -am 'Add some fooBar'`)\n4. \u63a8\u9001\u5230\u5206\u652f (`git push origin feature/fooBar`)\n5. \u521b\u5efa\u65b0\u7684 Pull Request\n\n## \u8bb8\u53ef\u8bc1\n\u672c\u9879\u76ee\u91c7\u7528 MIT \u8bb8\u53ef\u8bc1 - \u8be6\u60c5\u8bf7\u89c1 LICENSE \u6587\u4ef6\n\n## \u8054\u7cfb\u65b9\u5f0f\n\u5982\u6709\u95ee\u9898\u6216\u5efa\u8bae\uff0c\u8bf7\u8054\u7cfb: xinzf@ucap.com.cn\n",
"bugtrack_url": null,
"license": null,
"summary": "\u4e00\u4e2a\u57fa\u4e8eRedis\u7684\u4efb\u52a1\u9000\u907f\u91cd\u8bd5\u6846\u67b6\uff0c\u652f\u6301\u591a\u79cd\u9000\u907f\u7b56\u7565\u548c\u81ea\u5b9a\u4e49\u914d\u7f6e",
"version": "1.0.0",
"project_urls": {
"Bug Reports": "https://github.com/yourusername/backoff_scheduler/issues",
"Documentation": "https://github.com/yourusername/backoff_scheduler#readme",
"Homepage": "http://192.168.3.118:8082/maas/maas-backoff_scheduler/",
"Source": "https://github.com/yourusername/backoff_scheduler"
},
"split_keywords": [
"task",
"retry",
"backoff",
"redis",
"queue"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ab032c8ad8f46b2ea2bd762238f36b456cc3d27f54e26b5ebbd1325127c8366a",
"md5": "4c86e5a5c8bd895ef4f8457ba01a36bb",
"sha256": "b72f6e9a186174b124a98c967318624120739e0a3b2126965f5339d49a0aee82"
},
"downloads": -1,
"filename": "backoff_scheduler-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4c86e5a5c8bd895ef4f8457ba01a36bb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 4504,
"upload_time": "2025-08-22T03:28:05",
"upload_time_iso_8601": "2025-08-22T03:28:05.993525Z",
"url": "https://files.pythonhosted.org/packages/ab/03/2c8ad8f46b2ea2bd762238f36b456cc3d27f54e26b5ebbd1325127c8366a/backoff_scheduler-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d4fa34c76d08b854e107370de9171fc0e2b909c9996c0e66c3447b659d5fdc26",
"md5": "9ea64e56a6628d88c2380112fd536bd7",
"sha256": "103d6a19788d13a8d79360569343f5f063ddf17e39e8afa650e04a1ad078cbac"
},
"downloads": -1,
"filename": "backoff_scheduler-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "9ea64e56a6628d88c2380112fd536bd7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 7801,
"upload_time": "2025-08-22T03:28:07",
"upload_time_iso_8601": "2025-08-22T03:28:07.573310Z",
"url": "https://files.pythonhosted.org/packages/d4/fa/34c76d08b854e107370de9171fc0e2b909c9996c0e66c3447b659d5fdc26/backoff_scheduler-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-22 03:28:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "backoff_scheduler",
"github_not_found": true,
"lcname": "backoff-scheduler"
}
JSON
