# Scrapy Extensions
[](https://pypi.python.org/pypi/scrapy-extensions/)
[](https://pypi.python.org/pypi/scrapy-extensions/)
[](https://pypi.python.org/pypi/scrapy-extensions/)
[](https://github.com/woltapp/wolt-python-package-cookiecutter)
---
**Documentation**: [https://MarkusShepherd.github.io/scrapy-extensions](https://MarkusShepherd.github.io/scrapy-extensions)
**Source Code**: [https://github.com/MarkusShepherd/scrapy-extensions](https://github.com/MarkusShepherd/scrapy-extensions)
**PyPI**: [https://pypi.org/project/scrapy-extensions/](https://pypi.org/project/scrapy-extensions/)
---
A collection of Scrapy extensions and other utilities
## Installation
```sh
pip install scrapy-extensions
```
## Development
* Clone this repository
* Requirements:
* [Poetry](https://python-poetry.org/)
* Python 3.8+
* Create a virtual environment and install the dependencies
```sh
poetry install
```
* Activate the virtual environment
```sh
poetry shell
```
### Testing
```sh
pytest
```
### Documentation
The documentation is automatically generated from the content of the [docs directory](https://github.com/MarkusShepherd/scrapy-extensions/tree/master/docs) and from the docstrings
of the public signatures of the source code. The documentation is updated and published as a [Github Pages page](https://pages.github.com/) automatically as part each release.
### Releasing
Trigger the [Draft release workflow](https://github.com/MarkusShepherd/scrapy-extensions/actions/workflows/draft_release.yml)
(press _Run workflow_). This will update the changelog & version and create a GitHub release which is in _Draft_ state.
Find the draft release from the
[GitHub releases](https://github.com/MarkusShepherd/scrapy-extensions/releases) and publish it. When
a release is published, it'll trigger [release](https://github.com/MarkusShepherd/scrapy-extensions/blob/master/.github/workflows/release.yml) workflow which creates PyPI
release and deploys updated documentation.
### Pre-commit
Pre-commit hooks run all the auto-formatting (`ruff format`), linters (e.g. `ruff` and `mypy`), and other quality
checks to make sure the changeset is in good shape before a commit/push happens.
You can install the hooks with (runs for each commit):
```sh
pre-commit install
```
Or if you want them to run only for each push:
```sh
pre-commit install -t pre-push
```
Or if you want e.g. want to run all checks manually for all files:
```sh
pre-commit run --all-files
```
---
This project was generated using the [wolt-python-package-cookiecutter](https://github.com/woltapp/wolt-python-package-cookiecutter) template.
Raw data
{
"_id": null,
"home_page": "https://MarkusShepherd.github.io/scrapy-extensions",
"name": "scrapy-extensions",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9.0",
"maintainer_email": null,
"keywords": null,
"author": "Markus Shepherd",
"author_email": "markus@recommend.games",
"download_url": "https://files.pythonhosted.org/packages/e2/de/0bac8773915d910ac8415673b6c2b17d240c5f922a779c5899005e813e8b/scrapy_extensions-1.0.2.tar.gz",
"platform": null,
"description": "# Scrapy Extensions\n\n[](https://pypi.python.org/pypi/scrapy-extensions/)\n[](https://pypi.python.org/pypi/scrapy-extensions/)\n[](https://pypi.python.org/pypi/scrapy-extensions/)\n[](https://github.com/woltapp/wolt-python-package-cookiecutter)\n\n\n---\n\n**Documentation**: [https://MarkusShepherd.github.io/scrapy-extensions](https://MarkusShepherd.github.io/scrapy-extensions)\n\n**Source Code**: [https://github.com/MarkusShepherd/scrapy-extensions](https://github.com/MarkusShepherd/scrapy-extensions)\n\n**PyPI**: [https://pypi.org/project/scrapy-extensions/](https://pypi.org/project/scrapy-extensions/)\n\n---\n\nA collection of Scrapy extensions and other utilities\n\n## Installation\n\n```sh\npip install scrapy-extensions\n```\n\n## Development\n\n* Clone this repository\n* Requirements:\n * [Poetry](https://python-poetry.org/)\n * Python 3.8+\n* Create a virtual environment and install the dependencies\n\n```sh\npoetry install\n```\n\n* Activate the virtual environment\n\n```sh\npoetry shell\n```\n\n### Testing\n\n```sh\npytest\n```\n\n### Documentation\n\nThe documentation is automatically generated from the content of the [docs directory](https://github.com/MarkusShepherd/scrapy-extensions/tree/master/docs) and from the docstrings\n of the public signatures of the source code. The documentation is updated and published as a [Github Pages page](https://pages.github.com/) automatically as part each release.\n\n### Releasing\n\nTrigger the [Draft release workflow](https://github.com/MarkusShepherd/scrapy-extensions/actions/workflows/draft_release.yml)\n(press _Run workflow_). This will update the changelog & version and create a GitHub release which is in _Draft_ state.\n\nFind the draft release from the\n[GitHub releases](https://github.com/MarkusShepherd/scrapy-extensions/releases) and publish it. When\n a release is published, it'll trigger [release](https://github.com/MarkusShepherd/scrapy-extensions/blob/master/.github/workflows/release.yml) workflow which creates PyPI\n release and deploys updated documentation.\n\n### Pre-commit\n\nPre-commit hooks run all the auto-formatting (`ruff format`), linters (e.g. `ruff` and `mypy`), and other quality\n checks to make sure the changeset is in good shape before a commit/push happens.\n\nYou can install the hooks with (runs for each commit):\n\n```sh\npre-commit install\n```\n\nOr if you want them to run only for each push:\n\n```sh\npre-commit install -t pre-push\n```\n\nOr if you want e.g. want to run all checks manually for all files:\n\n```sh\npre-commit run --all-files\n```\n\n---\n\nThis project was generated using the [wolt-python-package-cookiecutter](https://github.com/woltapp/wolt-python-package-cookiecutter) template.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A collection of Scrapy extensions and other utilities",
"version": "1.0.2",
"project_urls": {
"Documentation": "https://MarkusShepherd.github.io/scrapy-extensions",
"Homepage": "https://MarkusShepherd.github.io/scrapy-extensions",
"Repository": "https://github.com/MarkusShepherd/scrapy-extensions"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5ac41f7ad4d32d126e0108c215f68247198bb1b606d62269cee279498f59e02b",
"md5": "ed9b25cfe013bb3079dcb4ba447ee461",
"sha256": "bdd2d931e67af8ceb9bf71c5d1230ad93b7587ec0d0ebd3794f392c4cd5e72a1"
},
"downloads": -1,
"filename": "scrapy_extensions-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ed9b25cfe013bb3079dcb4ba447ee461",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9.0",
"size": 8972,
"upload_time": "2024-11-18T21:36:39",
"upload_time_iso_8601": "2024-11-18T21:36:39.479375Z",
"url": "https://files.pythonhosted.org/packages/5a/c4/1f7ad4d32d126e0108c215f68247198bb1b606d62269cee279498f59e02b/scrapy_extensions-1.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e2de0bac8773915d910ac8415673b6c2b17d240c5f922a779c5899005e813e8b",
"md5": "866b41f13ebe3f400518d8baf7743c91",
"sha256": "1a29d9d250dc5a6a89f1cb64e7b795766e69c9400da22c6aef9c6ba2da282e64"
},
"downloads": -1,
"filename": "scrapy_extensions-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "866b41f13ebe3f400518d8baf7743c91",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9.0",
"size": 8553,
"upload_time": "2024-11-18T21:36:40",
"upload_time_iso_8601": "2024-11-18T21:36:40.576184Z",
"url": "https://files.pythonhosted.org/packages/e2/de/0bac8773915d910ac8415673b6c2b17d240c5f922a779c5899005e813e8b/scrapy_extensions-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 21:36:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MarkusShepherd",
"github_project": "scrapy-extensions",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "scrapy-extensions"
}
JSON
