| Name | zimscraperlib JSON |
| Version |
4.0.0
JSON |
| download |
| home_page | None |
| Summary | Collection of python tools to re-use common code across scrapers |
| upload_time | 2024-08-05 09:34:14 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | <3.13,>=3.8 |
| license | GPL-3.0-or-later |
| keywords |
offline
openzim
zim
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
zimscraperlib
=============
[](https://github.com/openzim/python-scraperlib/actions?query=branch%3Amain)
[](https://www.codefactor.io/repository/github/openzim/python-scraperlib)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://pypi.org/project/zimscraperlib/)
[](https://pypi.org/project/zimscraperlib)
[](https://codecov.io/gh/openzim/python-scraperlib)
Collection of python code to re-use across python-based scrapers
# Usage
* This library is meant to be installed via PyPI ([`zimscraperlib`](https://pypi.org/project/zimscraperlib/)).
* Make sure to reference it using a version code as the API is subject to frequent changes.
* API should remain the same only within the same *minor* version.
Example usage:
``` pip
zimscraperlib>=1.1,<1.2
```
# Dependencies
* libmagic
* wget
* libzim (auto-installed, not available on Windows)
* Pillow
* FFmpeg
* gifsicle (>=1.92)
## macOS
```sh
brew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg gifsicle
```
## Linux
```sh
sudo apt install libmagic1 wget ffmpeg \
libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \
libharfbuzz-dev libfribidi-dev libxcb1-dev gifsicle
```
## Alpine
```
apk add ffmpeg gifsicle libmagic wget libjpeg
```
# Contribution
This project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).
This project has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/docs/Policy.md) **v1.0.2**.
```shell
pip install hatch
pip install ".[dev]"
pre-commit install
# For tests
invoke coverage
```
# Users
Non-exhaustive list of scrapers using it (check status when updating API):
* [openzim/freecodecamp](https://github.com/openzim/freecodecamp)
* [openzim/gutenberg](https://github.com/openzim/gutenberg)
* [openzim/ifixit](https://github.com/openzim/ifixit)
* [openzim/kolibri](https://github.com/openzim/kolibri)
* [openzim/nautilus](https://github.com/openzim/nautilus)
* [openzim/nautilus](https://github.com/openzim/nautilus)
* [openzim/openedx](https://github.com/openzim/openedx)
* [openzim/sotoki](https://github.com/openzim/sotoki)
* [openzim/ted](https://github.com/openzim/ted)
* [openzim/warc2zim](https://github.com/openzim/warc2zim)
* [openzim/wikihow](https://github.com/openzim/wikihow)
* [openzim/youtube](https://github.com/openzim/youtube)
Raw data
{
"_id": null,
"home_page": null,
"name": "zimscraperlib",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": "offline, openzim, zim",
"author": null,
"author_email": "openZIM <dev@openzim.org>",
"download_url": "https://files.pythonhosted.org/packages/28/be/389179082da6d0b4784ba4afbc6fc48d69429955e302665fb48ed485538e/zimscraperlib-4.0.0.tar.gz",
"platform": null,
"description": "zimscraperlib\n=============\n\n[](https://github.com/openzim/python-scraperlib/actions?query=branch%3Amain)\n[](https://www.codefactor.io/repository/github/openzim/python-scraperlib)\n[](https://www.gnu.org/licenses/gpl-3.0)\n[](https://pypi.org/project/zimscraperlib/)\n[](https://pypi.org/project/zimscraperlib)\n[](https://codecov.io/gh/openzim/python-scraperlib)\n\nCollection of python code to re-use across python-based scrapers\n\n# Usage\n\n* This library is meant to be installed via PyPI ([`zimscraperlib`](https://pypi.org/project/zimscraperlib/)).\n* Make sure to reference it using a version code as the API is subject to frequent changes.\n* API should remain the same only within the same *minor* version.\n\nExample usage:\n\n``` pip\nzimscraperlib>=1.1,<1.2\n```\n\n# Dependencies\n\n* libmagic\n* wget\n* libzim (auto-installed, not available on Windows)\n* Pillow\n* FFmpeg\n* gifsicle (>=1.92)\n\n## macOS\n\n```sh\nbrew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg gifsicle\n```\n\n## Linux\n\n```sh\nsudo apt install libmagic1 wget ffmpeg \\\n libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \\\n libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \\\n libharfbuzz-dev libfribidi-dev libxcb1-dev gifsicle\n```\n\n## Alpine\n```\napk add ffmpeg gifsicle libmagic wget libjpeg\n```\n\n# Contribution\n\nThis project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).\n\nThis project has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/docs/Policy.md) **v1.0.2**.\n\n```shell\npip install hatch\npip install \".[dev]\"\npre-commit install\n# For tests\ninvoke coverage\n```\n\n# Users\n\nNon-exhaustive list of scrapers using it (check status when updating API):\n\n* [openzim/freecodecamp](https://github.com/openzim/freecodecamp)\n* [openzim/gutenberg](https://github.com/openzim/gutenberg)\n* [openzim/ifixit](https://github.com/openzim/ifixit)\n* [openzim/kolibri](https://github.com/openzim/kolibri)\n* [openzim/nautilus](https://github.com/openzim/nautilus)\n* [openzim/nautilus](https://github.com/openzim/nautilus)\n* [openzim/openedx](https://github.com/openzim/openedx)\n* [openzim/sotoki](https://github.com/openzim/sotoki)\n* [openzim/ted](https://github.com/openzim/ted)\n* [openzim/warc2zim](https://github.com/openzim/warc2zim)\n* [openzim/wikihow](https://github.com/openzim/wikihow)\n* [openzim/youtube](https://github.com/openzim/youtube)\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Collection of python tools to re-use common code across scrapers",
"version": "4.0.0",
"project_urls": {
"Donate": "https://www.kiwix.org/en/support-us/",
"Homepage": "https://github.com/openzim/python-scraperlib"
},
"split_keywords": [
"offline",
" openzim",
" zim"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4be05710b8aa2e0f1d788f5979dae98640dc6f000b17ce8fc739da56cf9c2085",
"md5": "1276a0dc9a40f1eda76ab0a86d8dd24e",
"sha256": "7a0e3c3aa9d33aec225555aaa069f29f00a5af2d4ab767822d614c70adbd5718"
},
"downloads": -1,
"filename": "zimscraperlib-4.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1276a0dc9a40f1eda76ab0a86d8dd24e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 63892,
"upload_time": "2024-08-05T09:34:12",
"upload_time_iso_8601": "2024-08-05T09:34:12.773030Z",
"url": "https://files.pythonhosted.org/packages/4b/e0/5710b8aa2e0f1d788f5979dae98640dc6f000b17ce8fc739da56cf9c2085/zimscraperlib-4.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "28be389179082da6d0b4784ba4afbc6fc48d69429955e302665fb48ed485538e",
"md5": "02ad3de851bc511a97b9d9ca342e3bc6",
"sha256": "5570a9c7efbe117ff27cc20f3fcaa4ed5d305e580a20399ab60375f742dc4281"
},
"downloads": -1,
"filename": "zimscraperlib-4.0.0.tar.gz",
"has_sig": false,
"md5_digest": "02ad3de851bc511a97b9d9ca342e3bc6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8",
"size": 6493613,
"upload_time": "2024-08-05T09:34:14",
"upload_time_iso_8601": "2024-08-05T09:34:14.343256Z",
"url": "https://files.pythonhosted.org/packages/28/be/389179082da6d0b4784ba4afbc6fc48d69429955e302665fb48ed485538e/zimscraperlib-4.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-05 09:34:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openzim",
"github_project": "python-scraperlib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "zimscraperlib"
}
