| Name | zimscraperlib JSON |
| Version |
3.3.2
JSON |
| download |
| home_page | None |
| Summary | Collection of python tools to re-use common code across scrapers |
| upload_time | 2024-03-25 09:16:49 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | <3.13,>=3.8 |
| license | GPL-3.0-or-later |
| keywords |
offline
openzim
zim
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
zimscraperlib
=============
[](https://github.com/openzim/python-scraperlib/actions?query=branch%3Amain)
[](https://www.codefactor.io/repository/github/openzim/python-scraperlib)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://pypi.org/project/zimscraperlib/)
[](https://pypi.org/project/zimscraperlib)
[](https://codecov.io/gh/openzim/python-scraperlib)
Collection of python code to re-use across python-based scrapers
# Usage
* This library is meant to be installed via PyPI ([`zimscraperlib`](https://pypi.org/project/zimscraperlib/)).
* Make sure to reference it using a version code as the API is subject to frequent changes.
* API should remain the same only within the same *minor* version.
Example usage:
``` pip
zimscraperlib>=1.1,<1.2
```
# Dependencies
* libmagic
* wget
* libzim (auto-installed, not available on Windows)
* Pillow
* FFmpeg
* gifsicle (>=1.92)
## macOS
```sh
brew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg gifsicle
```
## Linux
```sh
sudo apt install libmagic1 wget ffmpeg \
libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \
libharfbuzz-dev libfribidi-dev libxcb1-dev gifsicle
```
## Alpine
```
apk add ffmpeg gifsicle libmagic wget libjpeg
```
**Nota:** i18n features do not work on Alpine, see https://github.com/openzim/python-scraperlib/issues/134 ; there is one corresponding test which is failing.
# Contribution
This project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing)
```shell
pip install hatch
pip install ".[dev]"
pre-commit install
# For tests
invoke coverage
```
# Users
Non-exhaustive list of scrapers using it (check status when updating API):
* [openzim/freecodecamp](https://github.com/openzim/freecodecamp)
* [openzim/gutenberg](https://github.com/openzim/gutenberg)
* [openzim/ifixit](https://github.com/openzim/ifixit)
* [openzim/kolibri](https://github.com/openzim/kolibri)
* [openzim/nautilus](https://github.com/openzim/nautilus)
* [openzim/nautilus](https://github.com/openzim/nautilus)
* [openzim/openedx](https://github.com/openzim/openedx)
* [openzim/sotoki](https://github.com/openzim/sotoki)
* [openzim/ted](https://github.com/openzim/ted)
* [openzim/warc2zim](https://github.com/openzim/warc2zim)
* [openzim/wikihow](https://github.com/openzim/wikihow)
* [openzim/youtube](https://github.com/openzim/youtube)
Raw data
{
"_id": null,
"home_page": null,
"name": "zimscraperlib",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": "offline, openzim, zim",
"author": null,
"author_email": "openZIM <dev@openzim.org>",
"download_url": "https://files.pythonhosted.org/packages/a6/6a/66ac953ef074dbfc8be83154680d521e504e89b0f93d512b289e05b31d3d/zimscraperlib-3.3.2.tar.gz",
"platform": null,
"description": "zimscraperlib\n=============\n\n[](https://github.com/openzim/python-scraperlib/actions?query=branch%3Amain)\n[](https://www.codefactor.io/repository/github/openzim/python-scraperlib)\n[](https://www.gnu.org/licenses/gpl-3.0)\n[](https://pypi.org/project/zimscraperlib/)\n[](https://pypi.org/project/zimscraperlib)\n[](https://codecov.io/gh/openzim/python-scraperlib)\n\nCollection of python code to re-use across python-based scrapers\n\n# Usage\n\n* This library is meant to be installed via PyPI ([`zimscraperlib`](https://pypi.org/project/zimscraperlib/)).\n* Make sure to reference it using a version code as the API is subject to frequent changes.\n* API should remain the same only within the same *minor* version.\n\nExample usage:\n\n``` pip\nzimscraperlib>=1.1,<1.2\n```\n\n# Dependencies\n\n* libmagic\n* wget\n* libzim (auto-installed, not available on Windows)\n* Pillow\n* FFmpeg\n* gifsicle (>=1.92)\n\n## macOS\n\n```sh\nbrew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg gifsicle\n```\n\n## Linux\n\n```sh\nsudo apt install libmagic1 wget ffmpeg \\\n libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \\\n libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \\\n libharfbuzz-dev libfribidi-dev libxcb1-dev gifsicle\n```\n\n## Alpine\n```\napk add ffmpeg gifsicle libmagic wget libjpeg\n```\n\n**Nota:** i18n features do not work on Alpine, see https://github.com/openzim/python-scraperlib/issues/134 ; there is one corresponding test which is failing.\n\n# Contribution\n\nThis project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing)\n\n```shell\npip install hatch\npip install \".[dev]\"\npre-commit install\n# For tests\ninvoke coverage\n```\n\n# Users\n\nNon-exhaustive list of scrapers using it (check status when updating API):\n\n* [openzim/freecodecamp](https://github.com/openzim/freecodecamp)\n* [openzim/gutenberg](https://github.com/openzim/gutenberg)\n* [openzim/ifixit](https://github.com/openzim/ifixit)\n* [openzim/kolibri](https://github.com/openzim/kolibri)\n* [openzim/nautilus](https://github.com/openzim/nautilus)\n* [openzim/nautilus](https://github.com/openzim/nautilus)\n* [openzim/openedx](https://github.com/openzim/openedx)\n* [openzim/sotoki](https://github.com/openzim/sotoki)\n* [openzim/ted](https://github.com/openzim/ted)\n* [openzim/warc2zim](https://github.com/openzim/warc2zim)\n* [openzim/wikihow](https://github.com/openzim/wikihow)\n* [openzim/youtube](https://github.com/openzim/youtube)\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Collection of python tools to re-use common code across scrapers",
"version": "3.3.2",
"project_urls": {
"Donate": "https://www.kiwix.org/en/support-us/",
"Homepage": "https://github.com/openzim/python-scraperlib"
},
"split_keywords": [
"offline",
" openzim",
" zim"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c2addb97e3eb1869d3b7d87cb3f71538fdb7053899807b72c733b9fa5b7cc9b1",
"md5": "7f0a9f00427495837857f7c23ea4ee5f",
"sha256": "103eed5c6433c6786fb4cf8341841d8fdb35dbc29d65047a6e1c54cf95c684bc"
},
"downloads": -1,
"filename": "zimscraperlib-3.3.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f0a9f00427495837857f7c23ea4ee5f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 58951,
"upload_time": "2024-03-25T09:16:46",
"upload_time_iso_8601": "2024-03-25T09:16:46.805871Z",
"url": "https://files.pythonhosted.org/packages/c2/ad/db97e3eb1869d3b7d87cb3f71538fdb7053899807b72c733b9fa5b7cc9b1/zimscraperlib-3.3.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a66a66ac953ef074dbfc8be83154680d521e504e89b0f93d512b289e05b31d3d",
"md5": "6fab03680e7d07ef1a5ba94961d838fa",
"sha256": "4e8e5ff5c512895feb20d6276de947c52a2ed5ceed9bbde47602e13fe2e51d5e"
},
"downloads": -1,
"filename": "zimscraperlib-3.3.2.tar.gz",
"has_sig": false,
"md5_digest": "6fab03680e7d07ef1a5ba94961d838fa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8",
"size": 3196860,
"upload_time": "2024-03-25T09:16:49",
"upload_time_iso_8601": "2024-03-25T09:16:49.111182Z",
"url": "https://files.pythonhosted.org/packages/a6/6a/66ac953ef074dbfc8be83154680d521e504e89b0f93d512b289e05b31d3d/zimscraperlib-3.3.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-25 09:16:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openzim",
"github_project": "python-scraperlib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "zimscraperlib"
}
