RunSpiders
^^^^^^^^^^^
|Python3|
A python library contains many predefined powerful web crawlers.
**Attention**: this package probably can't work properly because of the correlated webs' updates.
If this situation happens, just fix it on your own.
Installation
>>>>>>>>>>>>>
.. code:: bash
pip install RunSpiders
Requirements
>>>>>>>>>>>>>
.. code:: python
from RunSpiders import Checker
checker = Checker()
checker.main()
1. [calibre](https://calibre-ebook.com/download) for converting ebook
[Mac安装Calibre电子书应用](https://blog.csdn.net/u012955829/article/details/99240602)
.. code::bash
vim ~/.bash_profile
export EBOOK_PATH=/Applications/calibre.app/Contents/MacOS
export PATH=$PATH:$EBOOK_PATH
2. [MAC 系统安装FFmpeg](https://www.jianshu.com/p/73441acf7815)
Examples
>>>>>>>>>>>>>
novel
::::::::::::::::
Please install calibre and add `ebook-convert` to environment variables.
.. code:: python
from RunSpiders import WebFictionSpider
output = "F:/ebooks"
spider = WebFictionSpider(output)
s.download(book="诛仙")
s.download(author="云天空")
s.download_books(["秘巫之主", "极品家丁"])
movie
::::::::::::::::
Please install ffmpeg and add it to environment variables.
.. code:: python
from RunSpiders.video.base.m3u8 import M3U8Spider
spider = M3U8Spider(output="F\movies")
spider.download_movies([(m3u8_url, file_name), ...])
.. |Python3| image:: https://img.shields.io/badge/python-3-red.svg
FAQ
>>>>>>>>>>>>>
IPython报错
::::::::::::::::
- 问题：ImportError: cannot import name 'generator_to_async_generator'
- 解决方案：``pip install ipython prompt_toolkit --upgrade``
Raw data
{
"_id": null,
"home_page": "https://github.com/Ijustwantyouhappy/RunSpiders",
"name": "RunSpiders",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": "",
"keywords": "spiders",
"author": "Ijustwantyouhappy",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/88/3f/a2a3014fc37b194fcfffe30d338922eb94cd5053937bcc77c24abd093108/RunSpiders-1.0.8.tar.gz",
"platform": null,
"description": "RunSpiders\n^^^^^^^^^^^\n\n|Python3|\n\nA python library contains many predefined powerful web crawlers.\n\n**Attention**: this package probably can't work properly because of the correlated webs' updates.\nIf this situation happens, just fix it on your own.\n\nInstallation\n>>>>>>>>>>>>>\n\n.. code:: bash\n\n pip install RunSpiders\n\nRequirements\n>>>>>>>>>>>>>\n\n.. code:: python\n\n from RunSpiders import Checker\n\n checker = Checker()\n checker.main()\n\n1. [calibre](https://calibre-ebook.com/download) for converting ebook\n[Mac\u5b89\u88c5Calibre\u7535\u5b50\u4e66\u5e94\u7528](https://blog.csdn.net/u012955829/article/details/99240602)\n\n.. code::bash\n\n vim ~/.bash_profile\n export EBOOK_PATH=/Applications/calibre.app/Contents/MacOS\n export PATH=$PATH:$EBOOK_PATH\n\n2. [MAC \u7cfb\u7edf\u5b89\u88c5FFmpeg](https://www.jianshu.com/p/73441acf7815)\n\nExamples\n>>>>>>>>>>>>>\n\nnovel\n::::::::::::::::\nPlease install calibre and add `ebook-convert` to environment variables.\n\n.. code:: python\n\n from RunSpiders import WebFictionSpider\n\n output = \"F:/ebooks\"\n spider = WebFictionSpider(output)\n\n s.download(book=\"\u8bdb\u4ed9\")\n s.download(author=\"\u4e91\u5929\u7a7a\")\n s.download_books([\"\u79d8\u5deb\u4e4b\u4e3b\", \"\u6781\u54c1\u5bb6\u4e01\"])\n\nmovie\n::::::::::::::::\nPlease install ffmpeg and add it to environment variables.\n\n.. code:: python\n\n from RunSpiders.video.base.m3u8 import M3U8Spider\n\n spider = M3U8Spider(output=\"F\\movies\")\n spider.download_movies([(m3u8_url, file_name), ...])\n\n\n.. |Python3| image:: https://img.shields.io/badge/python-3-red.svg\n\n\nFAQ\n>>>>>>>>>>>>>\nIPython\u62a5\u9519\n::::::::::::::::\n- \u95ee\u9898\uff1aImportError: cannot import name 'generator_to_async_generator'\n- \u89e3\u51b3\u65b9\u6848\uff1a``pip install ipython prompt_toolkit --upgrade``\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A python library contains many powerful web crawlers.",
"version": "1.0.8",
"project_urls": {
"GitHub": "https://github.com/Ijustwantyouhappy/RunSpiders",
"Homepage": "https://github.com/Ijustwantyouhappy/RunSpiders",
"Source": "https://github.com/Ijustwantyouhappy/RunSpiders"
},
"split_keywords": [
"spiders"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "86baa9a8608bd651e7381b20fa64cec993f40f843fe7fffa4ad98c3a8fd2601d",
"md5": "f128b5732640fcce2f4a468ef7935822",
"sha256": "8c93f89533f990fd2c78cd553453d46dae18087b7f1efc46ec110030e026e9b5"
},
"downloads": -1,
"filename": "RunSpiders-1.0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f128b5732640fcce2f4a468ef7935822",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 29295,
"upload_time": "2024-03-10T18:24:24",
"upload_time_iso_8601": "2024-03-10T18:24:24.470002Z",
"url": "https://files.pythonhosted.org/packages/86/ba/a9a8608bd651e7381b20fa64cec993f40f843fe7fffa4ad98c3a8fd2601d/RunSpiders-1.0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "883fa2a3014fc37b194fcfffe30d338922eb94cd5053937bcc77c24abd093108",
"md5": "4a3d61fefa38a374016db4546905752c",
"sha256": "78d227796ab8bb69d0c330577af9a5835a3d651cd2ff9150c44dd94f3a9a5067"
},
"downloads": -1,
"filename": "RunSpiders-1.0.8.tar.gz",
"has_sig": false,
"md5_digest": "4a3d61fefa38a374016db4546905752c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 24351,
"upload_time": "2024-03-10T18:24:26",
"upload_time_iso_8601": "2024-03-10T18:24:26.192790Z",
"url": "https://files.pythonhosted.org/packages/88/3f/a2a3014fc37b194fcfffe30d338922eb94cd5053937bcc77c24abd093108/RunSpiders-1.0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-10 18:24:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Ijustwantyouhappy",
"github_project": "RunSpiders",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "runspiders"
}
JSON
