# xxl-jobs 的python客户端实现
<p align="center">
<a href="https://pypi.org/project/pyxxl" target="_blank">
<img src="https://img.shields.io/pypi/v/pyxxl?color=%2334D058&label=pypi%20package" alt="Package version">
</a>
<a href="https://pypi.org/project/pyxxl" target="_blank">
<img src="https://img.shields.io/pypi/pyversions/pyxxl.svg?color=%2334D058" alt="Supported Python versions">
</a>
<a href="https://pypi.org/project/pyxxl" target="_blank">
<img src="https://img.shields.io/codecov/c/github/fcfangcc/pyxxl?color=%2334D058" alt="Coverage">
</a>
</p>
使用pyxxl可以方便的把Python写的方法注册到[XXL-JOB](https://github.com/xuxueli/xxl-job)中,使用XXL-JOB-ADMIN管理Python定时任务和周期任务
实现原理：通过XXL-JOB提供的RESTful API接口进行对接
<font color="#dd0000">注意！！！如果用同步的方法，请查看下面同步任务注意事项。</font>
## 已经支持的功能
* 执行器注册到job-admin
* task注册，类似于flask路由装饰器的用法
* 任务的管理（支持在界面上取消，发起等操作，任务完成后会回调admin）
* 所有阻塞策略的支持
* 异步支持（推荐）
* job-admin上查看日志
## 适配的XXL-JOB版本
**2.4.0**,**2.3.0**,**2.2.0**
如遇到不兼容的情况请issue告诉我XXL-JOB版本和对应的问题我会尽量适配
## 如何使用
```shell
pip install pyxxl
# 如果日志需要写入redis
pip install "pyxxl[redis]"
# 如果需要从.env加载配置
pip install "pyxxl[dotenv]"
# 安装所有功能
pip install "pyxxl[all]"
```
```python
import asyncio
from pyxxl import ExecutorConfig, PyxxlRunner
config = ExecutorConfig(
xxl_admin_baseurl="http://localhost:8080/xxl-job-admin/api/",
executor_app_name="xxl-job-executor-sample",
executor_host="172.17.0.1",
)
app = PyxxlRunner(config)
@app.register(name="demoJobHandler")
async def test_task():
await asyncio.sleep(5)
return "成功..."
# 如果你代码里面没有实现全异步，请使用同步函数，不然会阻塞其他任务
@app.register(name="xxxxx")
def test_task3():
return "成功3"
app.run_executor()
```
更多示例和接口文档请参考 [PYXXL文档](https://fcfangcc.github.io/pyxxl/example/) ，具体代码在example文件夹下面
如果executor服务无法直连xxl-admin，请参考[PYXXL配置](https://fcfangcc.github.io/pyxxl/apis/config/)修改executor_listen_host
## 监控指标
```shell
pip install "pyxxl[metrics]"
```
安装metrics扩展后，执行器会自动加载prometheus的指标监控功能
访问地址为: http://executor_listen_host:executor_listen_port/metrics
## 同步任务注意事项
同步任务会放到线程池中运行，无法正确接受cancel信号和timeout配置
如果需要使用同步任务并且无法控制（预料）里面执行时间，又需要进行超时和cancel功能的，需要接受pyxxl发出的cancel_event，当该cancel_event被设置时需要中断任务，下面是一个案例:
```python
...
@app.register(name="sync_func")
def sync_loop_demo():
# 如果同步任务里面有循环，为了支持cancel操作，必须每次都判断g.cancel_event.
task_params_list = []
while not g.cancel_event.is_set() and task_parasm_list:
params = task_params_list.pop()
time.sleep(3)
return "ok"
# 如下代码会造成线程池里的线程被永远占用，timeout cancel全部不生效
@app.register(name="sync_func2")
def sync_loop_demo2():
while True:
time.sleep(3) # 模拟你运行的任务
return "ok"
```
## 其他
* 由于种种3.9之后才加入的语法与特性，减少开发与适配成本，计划后续版本不再适配Python3.9以下版本，0.3.0最后一个支持Python3.8的版本
## 已知问题
1. 界面上显示的执行时间其实是任务回调的时间，而不是真正开始的时间.这是XXL-JOB的bug，pyxxl这边已经传了正确的执行时间过去，XXL-JOB没有按预期解析直接取了当前时间
## 开发人员
下面是开发人员如何快捷的搭建开发调试环境
### 启动xxl的调度中心
```shell
./init_dev_env.sh
```
### 启动执行器
```shell
# if you need. set venv in project.
# poetry config virtualenvs.in-project true
poetry install --all-extras
# 修改app.py中相关的配置信息,然后启动
poetry run python example/executor_app.py
```
Raw data
{
"_id": null,
"home_page": "https://github.com/fcfangcc/pyxxl",
"name": "pyxxl",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "XXL",
"author": "fcfangcc",
"author_email": "swjfc22@live.com",
"download_url": "https://files.pythonhosted.org/packages/81/fe/6486ad7f88b74c8be0b58abfef86aa72117743512334523372b566f4a9c0/pyxxl-0.3.5.tar.gz",
"platform": null,
"description": "# xxl-jobs \u7684python\u5ba2\u6237\u7aef\u5b9e\u73b0\n\n<p align=\"center\">\n<a href=\"https://pypi.org/project/pyxxl\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/v/pyxxl?color=%2334D058&label=pypi%20package\" alt=\"Package version\">\n</a>\n<a href=\"https://pypi.org/project/pyxxl\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/pyversions/pyxxl.svg?color=%2334D058\" alt=\"Supported Python versions\">\n</a>\n<a href=\"https://pypi.org/project/pyxxl\" target=\"_blank\">\n <img src=\"https://img.shields.io/codecov/c/github/fcfangcc/pyxxl?color=%2334D058\" alt=\"Coverage\">\n</a>\n</p>\n\n\u4f7f\u7528pyxxl\u53ef\u4ee5\u65b9\u4fbf\u7684\u628aPython\u5199\u7684\u65b9\u6cd5\u6ce8\u518c\u5230[XXL-JOB](https://github.com/xuxueli/xxl-job)\u4e2d,\u4f7f\u7528XXL-JOB-ADMIN\u7ba1\u7406Python\u5b9a\u65f6\u4efb\u52a1\u548c\u5468\u671f\u4efb\u52a1\n\n\u5b9e\u73b0\u539f\u7406\uff1a\u901a\u8fc7XXL-JOB\u63d0\u4f9b\u7684RESTful API\u63a5\u53e3\u8fdb\u884c\u5bf9\u63a5\n\n<font color=\"#dd0000\">\u6ce8\u610f\uff01\uff01\uff01\u5982\u679c\u7528\u540c\u6b65\u7684\u65b9\u6cd5\uff0c\u8bf7\u67e5\u770b\u4e0b\u9762\u540c\u6b65\u4efb\u52a1\u6ce8\u610f\u4e8b\u9879\u3002</font>\n\n## \u5df2\u7ecf\u652f\u6301\u7684\u529f\u80fd\n\n* \u6267\u884c\u5668\u6ce8\u518c\u5230job-admin\n* task\u6ce8\u518c\uff0c\u7c7b\u4f3c\u4e8eflask\u8def\u7531\u88c5\u9970\u5668\u7684\u7528\u6cd5\n* \u4efb\u52a1\u7684\u7ba1\u7406\uff08\u652f\u6301\u5728\u754c\u9762\u4e0a\u53d6\u6d88\uff0c\u53d1\u8d77\u7b49\u64cd\u4f5c\uff0c\u4efb\u52a1\u5b8c\u6210\u540e\u4f1a\u56de\u8c03admin\uff09\n* \u6240\u6709\u963b\u585e\u7b56\u7565\u7684\u652f\u6301\n* \u5f02\u6b65\u652f\u6301\uff08\u63a8\u8350\uff09\n* job-admin\u4e0a\u67e5\u770b\u65e5\u5fd7\n\n## \u9002\u914d\u7684XXL-JOB\u7248\u672c\n\n**2.4.0**,**2.3.0**,**2.2.0**\n\n\u5982\u9047\u5230\u4e0d\u517c\u5bb9\u7684\u60c5\u51b5\u8bf7issue\u544a\u8bc9\u6211XXL-JOB\u7248\u672c\u548c\u5bf9\u5e94\u7684\u95ee\u9898\u6211\u4f1a\u5c3d\u91cf\u9002\u914d\n\n## \u5982\u4f55\u4f7f\u7528\n\n```shell\npip install pyxxl\n\n# \u5982\u679c\u65e5\u5fd7\u9700\u8981\u5199\u5165redis\npip install \"pyxxl[redis]\"\n\n# \u5982\u679c\u9700\u8981\u4ece.env\u52a0\u8f7d\u914d\u7f6e\npip install \"pyxxl[dotenv]\"\n\n# \u5b89\u88c5\u6240\u6709\u529f\u80fd\npip install \"pyxxl[all]\"\n```\n\n```python\nimport asyncio\n\nfrom pyxxl import ExecutorConfig, PyxxlRunner\n\nconfig = ExecutorConfig(\n xxl_admin_baseurl=\"http://localhost:8080/xxl-job-admin/api/\",\n executor_app_name=\"xxl-job-executor-sample\",\n executor_host=\"172.17.0.1\",\n)\n\napp = PyxxlRunner(config)\n\n@app.register(name=\"demoJobHandler\")\nasync def test_task():\n await asyncio.sleep(5)\n return \"\u6210\u529f...\"\n\n# \u5982\u679c\u4f60\u4ee3\u7801\u91cc\u9762\u6ca1\u6709\u5b9e\u73b0\u5168\u5f02\u6b65\uff0c\u8bf7\u4f7f\u7528\u540c\u6b65\u51fd\u6570\uff0c\u4e0d\u7136\u4f1a\u963b\u585e\u5176\u4ed6\u4efb\u52a1\n@app.register(name=\"xxxxx\")\ndef test_task3():\n return \"\u6210\u529f3\"\n\n\napp.run_executor()\n```\n\n\n\u66f4\u591a\u793a\u4f8b\u548c\u63a5\u53e3\u6587\u6863\u8bf7\u53c2\u8003 [PYXXL\u6587\u6863](https://fcfangcc.github.io/pyxxl/example/) \uff0c\u5177\u4f53\u4ee3\u7801\u5728example\u6587\u4ef6\u5939\u4e0b\u9762\n\n\u5982\u679cexecutor\u670d\u52a1\u65e0\u6cd5\u76f4\u8fdexxl-admin\uff0c\u8bf7\u53c2\u8003[PYXXL\u914d\u7f6e](https://fcfangcc.github.io/pyxxl/apis/config/)\u4fee\u6539executor_listen_host\n\n## \u76d1\u63a7\u6307\u6807\n\n```shell\npip install \"pyxxl[metrics]\"\n```\n\n\u5b89\u88c5metrics\u6269\u5c55\u540e\uff0c\u6267\u884c\u5668\u4f1a\u81ea\u52a8\u52a0\u8f7dprometheus\u7684\u6307\u6807\u76d1\u63a7\u529f\u80fd\n\n\u8bbf\u95ee\u5730\u5740\u4e3a: http://executor_listen_host:executor_listen_port/metrics\n\n## \u540c\u6b65\u4efb\u52a1\u6ce8\u610f\u4e8b\u9879\n\u540c\u6b65\u4efb\u52a1\u4f1a\u653e\u5230\u7ebf\u7a0b\u6c60\u4e2d\u8fd0\u884c\uff0c\u65e0\u6cd5\u6b63\u786e\u63a5\u53d7cancel\u4fe1\u53f7\u548ctimeout\u914d\u7f6e\n\n\u5982\u679c\u9700\u8981\u4f7f\u7528\u540c\u6b65\u4efb\u52a1\u5e76\u4e14\u65e0\u6cd5\u63a7\u5236\uff08\u9884\u6599\uff09\u91cc\u9762\u6267\u884c\u65f6\u95f4\uff0c\u53c8\u9700\u8981\u8fdb\u884c\u8d85\u65f6\u548ccancel\u529f\u80fd\u7684\uff0c\u9700\u8981\u63a5\u53d7pyxxl\u53d1\u51fa\u7684cancel_event\uff0c\u5f53\u8be5cancel_event\u88ab\u8bbe\u7f6e\u65f6\u9700\u8981\u4e2d\u65ad\u4efb\u52a1\uff0c\u4e0b\u9762\u662f\u4e00\u4e2a\u6848\u4f8b:\n\n```python\n...\n\n@app.register(name=\"sync_func\")\ndef sync_loop_demo():\n # \u5982\u679c\u540c\u6b65\u4efb\u52a1\u91cc\u9762\u6709\u5faa\u73af\uff0c\u4e3a\u4e86\u652f\u6301cancel\u64cd\u4f5c\uff0c\u5fc5\u987b\u6bcf\u6b21\u90fd\u5224\u65adg.cancel_event.\n task_params_list = []\n while not g.cancel_event.is_set() and task_parasm_list:\n params = task_params_list.pop()\n time.sleep(3)\n return \"ok\"\n\n# \u5982\u4e0b\u4ee3\u7801\u4f1a\u9020\u6210\u7ebf\u7a0b\u6c60\u91cc\u7684\u7ebf\u7a0b\u88ab\u6c38\u8fdc\u5360\u7528\uff0ctimeout cancel\u5168\u90e8\u4e0d\u751f\u6548\n@app.register(name=\"sync_func2\")\ndef sync_loop_demo2():\n while True:\n time.sleep(3) # \u6a21\u62df\u4f60\u8fd0\u884c\u7684\u4efb\u52a1\n return \"ok\"\n\n```\n\n## \u5176\u4ed6\n\n* \u7531\u4e8e\u79cd\u79cd3.9\u4e4b\u540e\u624d\u52a0\u5165\u7684\u8bed\u6cd5\u4e0e\u7279\u6027\uff0c\u51cf\u5c11\u5f00\u53d1\u4e0e\u9002\u914d\u6210\u672c\uff0c\u8ba1\u5212\u540e\u7eed\u7248\u672c\u4e0d\u518d\u9002\u914dPython3.9\u4ee5\u4e0b\u7248\u672c\uff0c0.3.0\u6700\u540e\u4e00\u4e2a\u652f\u6301Python3.8\u7684\u7248\u672c\n\n## \u5df2\u77e5\u95ee\u9898\n\n1. \u754c\u9762\u4e0a\u663e\u793a\u7684\u6267\u884c\u65f6\u95f4\u5176\u5b9e\u662f\u4efb\u52a1\u56de\u8c03\u7684\u65f6\u95f4\uff0c\u800c\u4e0d\u662f\u771f\u6b63\u5f00\u59cb\u7684\u65f6\u95f4.\u8fd9\u662fXXL-JOB\u7684bug\uff0cpyxxl\u8fd9\u8fb9\u5df2\u7ecf\u4f20\u4e86\u6b63\u786e\u7684\u6267\u884c\u65f6\u95f4\u8fc7\u53bb\uff0cXXL-JOB\u6ca1\u6709\u6309\u9884\u671f\u89e3\u6790\u76f4\u63a5\u53d6\u4e86\u5f53\u524d\u65f6\u95f4\n\n## \u5f00\u53d1\u4eba\u5458\n\u4e0b\u9762\u662f\u5f00\u53d1\u4eba\u5458\u5982\u4f55\u5feb\u6377\u7684\u642d\u5efa\u5f00\u53d1\u8c03\u8bd5\u73af\u5883\n\n### \u542f\u52a8xxl\u7684\u8c03\u5ea6\u4e2d\u5fc3\n\n```shell\n./init_dev_env.sh\n```\n\n\n### \u542f\u52a8\u6267\u884c\u5668\n\n\n```shell\n# if you need. set venv in project.\n# poetry config virtualenvs.in-project true\npoetry install --all-extras\n# \u4fee\u6539app.py\u4e2d\u76f8\u5173\u7684\u914d\u7f6e\u4fe1\u606f,\u7136\u540e\u542f\u52a8\npoetry run python example/executor_app.py\n```\n\n",
"bugtrack_url": null,
"license": "GPL-3.0-only",
"summary": "A Python executor for XXL-jobs",
"version": "0.3.5",
"project_urls": {
"Homepage": "https://github.com/fcfangcc/pyxxl",
"Repository": "https://github.com/fcfangcc/pyxxl"
},
"split_keywords": [
"xxl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3d615d0d36f0cd519eae0784e4dabe5b2e69754fd227665405b6054989133e8f",
"md5": "1e79b2ded194ed006d814bd4e9e15bcc",
"sha256": "499f4883d8c3f826933709ce95742d283251803a1f5b2a2845779bb2272abe94"
},
"downloads": -1,
"filename": "pyxxl-0.3.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1e79b2ded194ed006d814bd4e9e15bcc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 45630,
"upload_time": "2024-04-30T06:58:14",
"upload_time_iso_8601": "2024-04-30T06:58:14.824386Z",
"url": "https://files.pythonhosted.org/packages/3d/61/5d0d36f0cd519eae0784e4dabe5b2e69754fd227665405b6054989133e8f/pyxxl-0.3.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "81fe6486ad7f88b74c8be0b58abfef86aa72117743512334523372b566f4a9c0",
"md5": "8230524b3c3b100d6581be05c1a83037",
"sha256": "9ef77536c602e16e1f501ecfbe93fa11a32e8e6f7a11cc6e9db2d97e19092bd8"
},
"downloads": -1,
"filename": "pyxxl-0.3.5.tar.gz",
"has_sig": false,
"md5_digest": "8230524b3c3b100d6581be05c1a83037",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 38182,
"upload_time": "2024-04-30T06:58:16",
"upload_time_iso_8601": "2024-04-30T06:58:16.376297Z",
"url": "https://files.pythonhosted.org/packages/81/fe/6486ad7f88b74c8be0b58abfef86aa72117743512334523372b566f4a9c0/pyxxl-0.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-30 06:58:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fcfangcc",
"github_project": "pyxxl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyxxl"
}
JSON
