| Name | hssp JSON |
| Version |
0.4.18
JSON |
| download |
| home_page | None |
| Summary | 一个简单快速的异步爬虫框架 |
| upload_time | 2025-08-05 16:45:51 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
异步
爬虫
爬虫框架
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# HSSP 爬虫框架
[](https://deepwiki.com/x-haose/hssp)
一个基于python asyncio开发的爬虫框架 (开发中)
## 作者
- [@昊色居士](https://github.com/x-haose)
## 特性
- 使用scrapy框架的选择器 [parsel](https://github.com/scrapy/parsel) 作为内置网页选择器
- 基于tenacity的自动异常重试
- 基于fake-useragent的可选随机UA
- 可选的多种下载器: [httpx](https://github.com/encode/httpx)、[aiohttp](https://github.com/aio-libs/aiohttp)、[requests](https://github.com/psf/requests)、[curl-cffi](https://github.com/lexiforest/curl_cffi)、[requests-go](https://github.com/wangluozhe/requests-go)
- 请求前、响应后、重试后监听
## 路线
- 增加其他解析器
- 在情求过程中临时更换下载器：比如net初始化时使用的是httpx下载器，其中一个情求要临时切换至 `DrissionPage`, 其他的依旧是httpx
- 支持 `DrissionPage`、`playwright` 浏览器渲染的下载器
- 下载器支持更多配置项及自定义项
- 编写详细使用文档
## 安装
###
使用 pip 安装 hssp
```bash
pip install hssp
```
###
使用 uv 安装 hssp
```bash
uv add hssp
```
## 支持
如需支持，请发送电子邮件至 xhrtxh@gmail.com。
## 开发测试
项目使用`uv`管理依赖，需先安装uv
```bash
rye sync
```
Raw data
{
"_id": null,
"home_page": null,
"name": "hssp",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "\u5f02\u6b65, \u722c\u866b, \u722c\u866b\u6846\u67b6",
"author": null,
"author_email": "\u660a\u8272\u5c45\u58eb <xhrtxh@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/1b/49/bcd2d8e6e79b53e8dd89882e0d0763d252c58cae7f724a8b353836ad4da7/hssp-0.4.18.tar.gz",
"platform": null,
"description": "# HSSP \u722c\u866b\u6846\u67b6\n\n[](https://deepwiki.com/x-haose/hssp)\n\n\u4e00\u4e2a\u57fa\u4e8epython asyncio\u5f00\u53d1\u7684\u722c\u866b\u6846\u67b6 (\u5f00\u53d1\u4e2d)\n\n## \u4f5c\u8005\n\n- [@\u660a\u8272\u5c45\u58eb](https://github.com/x-haose)\n\n## \u7279\u6027\n\n- \u4f7f\u7528scrapy\u6846\u67b6\u7684\u9009\u62e9\u5668 [parsel](https://github.com/scrapy/parsel) \u4f5c\u4e3a\u5185\u7f6e\u7f51\u9875\u9009\u62e9\u5668\n- \u57fa\u4e8etenacity\u7684\u81ea\u52a8\u5f02\u5e38\u91cd\u8bd5\n- \u57fa\u4e8efake-useragent\u7684\u53ef\u9009\u968f\u673aUA\n- \u53ef\u9009\u7684\u591a\u79cd\u4e0b\u8f7d\u5668: [httpx](https://github.com/encode/httpx)\u3001[aiohttp](https://github.com/aio-libs/aiohttp)\u3001[requests](https://github.com/psf/requests)\u3001[curl-cffi](https://github.com/lexiforest/curl_cffi)\u3001[requests-go](https://github.com/wangluozhe/requests-go)\n- \u8bf7\u6c42\u524d\u3001\u54cd\u5e94\u540e\u3001\u91cd\u8bd5\u540e\u76d1\u542c\n\n## \u8def\u7ebf\n\n- \u589e\u52a0\u5176\u4ed6\u89e3\u6790\u5668\n- \u5728\u60c5\u6c42\u8fc7\u7a0b\u4e2d\u4e34\u65f6\u66f4\u6362\u4e0b\u8f7d\u5668\uff1a\u6bd4\u5982net\u521d\u59cb\u5316\u65f6\u4f7f\u7528\u7684\u662fhttpx\u4e0b\u8f7d\u5668\uff0c\u5176\u4e2d\u4e00\u4e2a\u60c5\u6c42\u8981\u4e34\u65f6\u5207\u6362\u81f3 `DrissionPage`, \u5176\u4ed6\u7684\u4f9d\u65e7\u662fhttpx\n- \u652f\u6301 `DrissionPage`\u3001`playwright` \u6d4f\u89c8\u5668\u6e32\u67d3\u7684\u4e0b\u8f7d\u5668\n- \u4e0b\u8f7d\u5668\u652f\u6301\u66f4\u591a\u914d\u7f6e\u9879\u53ca\u81ea\u5b9a\u4e49\u9879\n- \u7f16\u5199\u8be6\u7ec6\u4f7f\u7528\u6587\u6863\n\n## \u5b89\u88c5\n\n###\n\n\u4f7f\u7528 pip \u5b89\u88c5 hssp\n\n```bash\npip install hssp\n```\n\n###\n\n\u4f7f\u7528 uv \u5b89\u88c5 hssp\n\n```bash\nuv add hssp\n```\n\n## \u652f\u6301\n\n\u5982\u9700\u652f\u6301\uff0c\u8bf7\u53d1\u9001\u7535\u5b50\u90ae\u4ef6\u81f3 xhrtxh@gmail.com\u3002\n\n## \u5f00\u53d1\u6d4b\u8bd5\n\n\u9879\u76ee\u4f7f\u7528`uv`\u7ba1\u7406\u4f9d\u8d56\uff0c\u9700\u5148\u5b89\u88c5uv\n\n```bash\n rye sync\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "\u4e00\u4e2a\u7b80\u5355\u5feb\u901f\u7684\u5f02\u6b65\u722c\u866b\u6846\u67b6",
"version": "0.4.18",
"project_urls": {
"documentation": "https://github.com/x-haose/hssp",
"homepage": "https://github.com/x-haose/hssp",
"repository": "https://github.com/x-haose/hssp"
},
"split_keywords": [
"\u5f02\u6b65",
" \u722c\u866b",
" \u722c\u866b\u6846\u67b6"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "cdc0e9ecead4df168d3c4db493cb00fa3fb6591bb388ce805e366ef72e67d1e0",
"md5": "a29a79b0c5ed712f6725ca34ea459ee9",
"sha256": "95c7fe09d1e1611c11bd73a80093fa4945dd8188b015e1c0c8ef28b903799517"
},
"downloads": -1,
"filename": "hssp-0.4.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a29a79b0c5ed712f6725ca34ea459ee9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 23049,
"upload_time": "2025-08-05T16:45:50",
"upload_time_iso_8601": "2025-08-05T16:45:50.132413Z",
"url": "https://files.pythonhosted.org/packages/cd/c0/e9ecead4df168d3c4db493cb00fa3fb6591bb388ce805e366ef72e67d1e0/hssp-0.4.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1b49bcd2d8e6e79b53e8dd89882e0d0763d252c58cae7f724a8b353836ad4da7",
"md5": "cae04cd1006d7fcf434fe2b0c2afcdb6",
"sha256": "5032ecfb90d8a40476080da435b8d1934c66490296252bd9d83ec685d7271fdf"
},
"downloads": -1,
"filename": "hssp-0.4.18.tar.gz",
"has_sig": false,
"md5_digest": "cae04cd1006d7fcf434fe2b0c2afcdb6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 16996,
"upload_time": "2025-08-05T16:45:51",
"upload_time_iso_8601": "2025-08-05T16:45:51.510276Z",
"url": "https://files.pythonhosted.org/packages/1b/49/bcd2d8e6e79b53e8dd89882e0d0763d252c58cae7f724a8b353836ad4da7/hssp-0.4.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-05 16:45:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "x-haose",
"github_project": "hssp",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "hssp"
}
