AioSpider-zly


NameAioSpider-zly JSON
Version 2.0.0 PyPI version JSON
download
home_pagehttps://github.com/zly717216/AioSpider
Summary高并发异步爬虫框架
upload_time2023-12-26 08:10:41
maintainerzly
docs_urlNone
authorzly717216
requires_python>=3.7
licenseMIT
keywords 异步网页抓取 网络爬虫 异步爬虫 python 异步抓取 数据提取 网络数据挖掘 异步爬虫框架 网络爬取工具 数据采集 数据挖掘工具
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            ## License

This project is licensed under the terms of the MIT license. See the [LICENSE](LICENSE) file for details.

让爬虫更简单，让爬虫更高效，让爬虫更智能

## 快速开始

### 1 命令行

aioSpider 系统语法：aioSpider [action] [-argv] [--option]

#### 1.1 查看帮助

```bash
aioSpider -h
```

#### 1.2 查看版本

```bash
aioSpider -v
```

#### 1.3 创建项目

```bash
aioSpider create -p <project>
```

#### 1.4 创建爬虫

```bash
aioSpider create -s <name> --u <url> --en <spider_en_name> --s <source> --t <target> --h <help>
```

#### 1.5 `sql` 表结构转 `model`

```bash
aioSpider make -m --i <path> [--o <path>]
```

#### 1.6 生成爬虫 `bat` 启动脚本

```bash
aioSpider make -b [--i <path>] [--o <path>]
```

#### 1.7 启动 `aioSpider Server`

```bash
aioSpider server run
```

#### 1.8 停止 `aioSpider Server`

```bash
aioSpider server stop
```

#### 1.9 测试 `IP` 带宽 

```bash
aioSpider test proxy [-p <proxy>] [--d <timeout>]
```

### 1.10 适配浏览器环境

```bash
aioSpider install
```

### 1.11 启动redis服务器

```bash
aioSpider redis start
```

### 1.12 关闭redis服务器

```bash
aioSpider redis stop [-p <port>]
```

### 2 使用步骤

（1）创建项目

```bash
aioSpider create -p myproject
```

（2）进入项目根路径

```bash
cd myproject
```

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/zly717216/AioSpider",
    "name": "AioSpider-zly",
    "maintainer": "zly",
    "docs_url": null,
    "requires_python": ">=3.7",
    "maintainer_email": "",
    "keywords": "\u5f02\u6b65\u7f51\u9875\u6293\u53d6,\u7f51\u7edc\u722c\u866b,\u5f02\u6b65\u722c\u866b,Python \u5f02\u6b65\u6293\u53d6,\u6570\u636e\u63d0\u53d6,\u7f51\u7edc\u6570\u636e\u6316\u6398,\u5f02\u6b65\u722c\u866b\u6846\u67b6,\u7f51\u7edc\u722c\u53d6\u5de5\u5177,\u6570\u636e\u91c7\u96c6,\u6570\u636e\u6316\u6398\u5de5\u5177",
    "author": "zly717216",
    "author_email": "zly717216@qq.com",
    "download_url": "https://files.pythonhosted.org/packages/33/af/74b5ea8fe96790eae3a02395dd0a17cc17b00249cfed23f8e133d02b0507/AioSpider-zly-2.0.0.tar.gz",
    "platform": null,
    "description": "## License\r\n\r\nThis project is licensed under the terms of the MIT license. See the [LICENSE](LICENSE) file for details.\r\n\r\n\u8ba9\u722c\u866b\u66f4\u7b80\u5355\uff0c\u8ba9\u722c\u866b\u66f4\u9ad8\u6548\uff0c\u8ba9\u722c\u866b\u66f4\u667a\u80fd\r\n\r\n## \u5feb\u901f\u5f00\u59cb\r\n\r\n### 1 \u547d\u4ee4\u884c\r\n\r\naioSpider \u7cfb\u7edf\u8bed\u6cd5\uff1aaioSpider [action] [-argv] [--option]\r\n\r\n#### 1.1 \u67e5\u770b\u5e2e\u52a9\r\n\r\n```bash\r\naioSpider -h\r\n```\r\n\r\n#### 1.2 \u67e5\u770b\u7248\u672c\r\n\r\n```bash\r\naioSpider -v\r\n```\r\n\r\n#### 1.3 \u521b\u5efa\u9879\u76ee\r\n\r\n```bash\r\naioSpider create -p <project>\r\n```\r\n\r\n#### 1.4 \u521b\u5efa\u722c\u866b\r\n\r\n```bash\r\naioSpider create -s <name> --u <url> --en <spider_en_name> --s <source> --t <target> --h <help>\r\n```\r\n\r\n#### 1.5 `sql` \u8868\u7ed3\u6784\u8f6c `model`\r\n\r\n```bash\r\naioSpider make -m --i <path> [--o <path>]\r\n```\r\n\r\n#### 1.6 \u751f\u6210\u722c\u866b `bat` \u542f\u52a8\u811a\u672c\r\n\r\n```bash\r\naioSpider make -b [--i <path>] [--o <path>]\r\n```\r\n\r\n#### 1.7 \u542f\u52a8 `aioSpider Server`\r\n\r\n```bash\r\naioSpider server run\r\n```\r\n\r\n#### 1.8 \u505c\u6b62 `aioSpider Server`\r\n\r\n```bash\r\naioSpider server stop\r\n```\r\n\r\n#### 1.9 \u6d4b\u8bd5 `IP` \u5e26\u5bbd \r\n\r\n```bash\r\naioSpider test proxy [-p <proxy>] [--d <timeout>]\r\n```\r\n\r\n### 1.10 \u9002\u914d\u6d4f\u89c8\u5668\u73af\u5883\r\n\r\n```bash\r\naioSpider install\r\n```\r\n\r\n### 1.11 \u542f\u52a8redis\u670d\u52a1\u5668\r\n\r\n```bash\r\naioSpider redis start\r\n```\r\n\r\n### 1.12 \u5173\u95edredis\u670d\u52a1\u5668\r\n\r\n```bash\r\naioSpider redis stop [-p <port>]\r\n```\r\n\r\n### 2 \u4f7f\u7528\u6b65\u9aa4\r\n\r\n\uff081\uff09\u521b\u5efa\u9879\u76ee\r\n\r\n```bash\r\naioSpider create -p myproject\r\n```\r\n\r\n\uff082\uff09\u8fdb\u5165\u9879\u76ee\u6839\u8def\u5f84\r\n\r\n```bash\r\ncd myproject\r\n```\r\n\r\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "\u9ad8\u5e76\u53d1\u5f02\u6b65\u722c\u866b\u6846\u67b6",
    "version": "2.0.0",
    "project_urls": {
        "Homepage": "https://github.com/zly717216/AioSpider"
    },
    "split_keywords": [
        "\u5f02\u6b65\u7f51\u9875\u6293\u53d6",
        "\u7f51\u7edc\u722c\u866b",
        "\u5f02\u6b65\u722c\u866b",
        "python \u5f02\u6b65\u6293\u53d6",
        "\u6570\u636e\u63d0\u53d6",
        "\u7f51\u7edc\u6570\u636e\u6316\u6398",
        "\u5f02\u6b65\u722c\u866b\u6846\u67b6",
        "\u7f51\u7edc\u722c\u53d6\u5de5\u5177",
        "\u6570\u636e\u91c7\u96c6",
        "\u6570\u636e\u6316\u6398\u5de5\u5177"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "12b193b1146e07489b581890cb16c5e7e82694130b5636d4d9ec81b6cb0f7867",
                "md5": "56f5269d5868530f8a0ab7c1af970494",
                "sha256": "9bf0fe6eff6367033e14b1f8d39fc26c377103f3f0c4a9d9431d74a1f3f81a32"
            },
            "downloads": -1,
            "filename": "AioSpider_zly-2.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "56f5269d5868530f8a0ab7c1af970494",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.7",
            "size": 233433,
            "upload_time": "2023-12-26T08:11:07",
            "upload_time_iso_8601": "2023-12-26T08:11:07.398539Z",
            "url": "https://files.pythonhosted.org/packages/12/b1/93b1146e07489b581890cb16c5e7e82694130b5636d4d9ec81b6cb0f7867/AioSpider_zly-2.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "33af74b5ea8fe96790eae3a02395dd0a17cc17b00249cfed23f8e133d02b0507",
                "md5": "a3e226710b69bf4178a373ff698ab32d",
                "sha256": "292e45bd2467c2f2bc74107c6237138d9806f2ea3595ba672da5a1ed1f48178e"
            },
            "downloads": -1,
            "filename": "AioSpider-zly-2.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "a3e226710b69bf4178a373ff698ab32d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.7",
            "size": 190089,
            "upload_time": "2023-12-26T08:10:41",
            "upload_time_iso_8601": "2023-12-26T08:10:41.074815Z",
            "url": "https://files.pythonhosted.org/packages/33/af/74b5ea8fe96790eae3a02395dd0a17cc17b00249cfed23f8e133d02b0507/AioSpider-zly-2.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-12-26 08:10:41",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "zly717216",
    "github_project": "AioSpider",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "aiospider-zly"
}
zly717216
Elapsed time: 0.23225s