| Name | imgscraper JSON |
| Version |
0.2.0
JSON |
| download |
| home_page | |
| Summary | Scans web pages for images |
| upload_time | 2023-10-28 19:57:04 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2023 Dawid Szaniawski Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
api
automation
json
web
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://github.com/dawid-szaniawski/imgscraper/actions/workflows/tox.yml)
[](https://pypi.org/project/imgscraper/)
[](https://pypi.org/project/imgscraper/)
[](https://github.com/dawid-szaniawski/imgscraper/blob/master/LICENSE)
[](https://github.com/psf/black)
[](https://github.com/astral-sh/ruff)
[](https://codecov.io/github/dawid-szaniawski/imgscraper)
[](https://www.codefactor.io/repository/github/dawid-szaniawski/imgscraper)
# imgscraper: yet another web scraper
imgscraper is a simple library that allows you to retrieve image information from meme sites.
## Installation
To install _imgscraper_, you can use pip:
```bash
pip install imgscraper
```
_ImgScraper_ supports Python 3.10+
## Usage
```python
from imgscraper.scraper_constructor import create_scraper
img_scraper = create_scraper(
website_url="https://imagocms.webludus.pl/",
container_class="image-holder",
pagination_class="pagination"
)
img_scraper.start_sync()
print(img_scraper.synchronization_data)
```
Output:
```python
[
Image(
source="https://imagocms.webludus.pl/images/01/",
url_address="https://imagocms.webludus.pl/img/01.jpg",
title="String"
),
Image(
source="https://imagocms.webludus.pl/images/02/",
url_address="https://imagocms.webludus.pl/img/02.jpg",
title="String"
)
]
```
## Pages to scan and scraper
The user can specify how many subpages should be scraped and what tool the application should use.
```python
from imgscraper.scraper_constructor import create_scraper
img_scraper = create_scraper(
website_url="https://imagocms.webludus.pl/",
container_class="image-holder",
pagination_class="pagination",
pages_to_scan=1,
scraper="bs4"
)
```
## Last sync data
When starting the synchronization process, the user can provide data from the last synchronization (img.src).
If the application encounters a provided image, the process is terminated. All previously synced images are available.
```python
scraper.start_sync(
(
"https://imagocms.webludus.pl/img/01.jpg",
"https://imagocms.webludus.pl/img/02.jpg",
)
)
```
## Image Object
The Image object provides the ``.as_dict()`` method to turn it into a dictionary.
```python
img = Image(
source="https://imagocms.webludus.pl/images/01/",
url_address="https://imagocms.webludus.pl/img/01.jpg",
title="String"
).as_dict()
```
Output:
```python
img = {
"source": "https://imagocms.webludus.pl/images/01/",
"url_address": "https://imagocms.webludus.pl/img/01.jpg",
"title": "String"
}
```
Raw data
{
"_id": null,
"home_page": "",
"name": "imgscraper",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "api,automation,json,web",
"author": "",
"author_email": "Dawid Szaniawski <webluduspl@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/9c/cf/ae9b5c9ccbfdbb6b8df77ccf3644ec94a86d507591a260a659a97e32dd0a/imgscraper-0.2.0.tar.gz",
"platform": null,
"description": "[](https://github.com/dawid-szaniawski/imgscraper/actions/workflows/tox.yml)\n[](https://pypi.org/project/imgscraper/)\n[](https://pypi.org/project/imgscraper/)\n[](https://github.com/dawid-szaniawski/imgscraper/blob/master/LICENSE)\n[](https://github.com/psf/black)\n[](https://github.com/astral-sh/ruff)\n[](https://codecov.io/github/dawid-szaniawski/imgscraper)\n[](https://www.codefactor.io/repository/github/dawid-szaniawski/imgscraper)\n\n# imgscraper: yet another web scraper\n\nimgscraper is a simple library that allows you to retrieve image information from meme sites.\n\n## Installation\n\nTo install _imgscraper_, you can use pip:\n\n```bash\npip install imgscraper\n```\n\n_ImgScraper_ supports Python 3.10+\n\n\n## Usage\n\n```python\nfrom imgscraper.scraper_constructor import create_scraper\n\nimg_scraper = create_scraper(\n website_url=\"https://imagocms.webludus.pl/\",\n container_class=\"image-holder\",\n pagination_class=\"pagination\"\n)\n\nimg_scraper.start_sync()\n\nprint(img_scraper.synchronization_data)\n```\n\nOutput:\n\n```python\n[\n Image(\n source=\"https://imagocms.webludus.pl/images/01/\",\n url_address=\"https://imagocms.webludus.pl/img/01.jpg\",\n title=\"String\"\n ),\n Image(\n source=\"https://imagocms.webludus.pl/images/02/\",\n url_address=\"https://imagocms.webludus.pl/img/02.jpg\",\n title=\"String\"\n )\n]\n```\n\n## Pages to scan and scraper\n\nThe user can specify how many subpages should be scraped and what tool the application should use.\n\n```python\nfrom imgscraper.scraper_constructor import create_scraper\n\nimg_scraper = create_scraper(\n website_url=\"https://imagocms.webludus.pl/\",\n container_class=\"image-holder\",\n pagination_class=\"pagination\",\n pages_to_scan=1,\n scraper=\"bs4\"\n)\n```\n\n## Last sync data\n\nWhen starting the synchronization process, the user can provide data from the last synchronization (img.src).\nIf the application encounters a provided image, the process is terminated. All previously synced images are available.\n\n```python\nscraper.start_sync(\n (\n \"https://imagocms.webludus.pl/img/01.jpg\",\n \"https://imagocms.webludus.pl/img/02.jpg\",\n )\n)\n```\n\n## Image Object\n\nThe Image object provides the ``.as_dict()`` method to turn it into a dictionary.\n\n```python\nimg = Image(\n source=\"https://imagocms.webludus.pl/images/01/\",\n url_address=\"https://imagocms.webludus.pl/img/01.jpg\",\n title=\"String\"\n).as_dict()\n```\n\nOutput:\n\n```python\nimg = {\n \"source\": \"https://imagocms.webludus.pl/images/01/\",\n \"url_address\": \"https://imagocms.webludus.pl/img/01.jpg\",\n \"title\": \"String\"\n}\n```\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 Dawid Szaniawski Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Scans web pages for images",
"version": "0.2.0",
"project_urls": {
"Source": "https://github.com/dawid-szaniawski/imgscraper"
},
"split_keywords": [
"api",
"automation",
"json",
"web"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dd94b7b2c7cd27851f7fbc7fa616a3b3b86945e3d8bf480679e5ec372dd1e773",
"md5": "d7ce682431db3a720f99490da1697412",
"sha256": "c59ef16e78c1a47d768be7e3b4190ac939622f32149d1529eaa935418e8812e1"
},
"downloads": -1,
"filename": "imgscraper-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d7ce682431db3a720f99490da1697412",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 10412,
"upload_time": "2023-10-28T19:57:02",
"upload_time_iso_8601": "2023-10-28T19:57:02.723524Z",
"url": "https://files.pythonhosted.org/packages/dd/94/b7b2c7cd27851f7fbc7fa616a3b3b86945e3d8bf480679e5ec372dd1e773/imgscraper-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9ccfae9b5c9ccbfdbb6b8df77ccf3644ec94a86d507591a260a659a97e32dd0a",
"md5": "2dce65c9994970ae1cb8d97b76cf6f32",
"sha256": "ab6e75bc4a954d5af281e9a9fd3f5137f08bba4b77ca53864b525c0d6977d4dc"
},
"downloads": -1,
"filename": "imgscraper-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "2dce65c9994970ae1cb8d97b76cf6f32",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 10810,
"upload_time": "2023-10-28T19:57:04",
"upload_time_iso_8601": "2023-10-28T19:57:04.077014Z",
"url": "https://files.pythonhosted.org/packages/9c/cf/ae9b5c9ccbfdbb6b8df77ccf3644ec94a86d507591a260a659a97e32dd0a/imgscraper-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-28 19:57:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dawid-szaniawski",
"github_project": "imgscraper",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "imgscraper"
}
