| Name | extract-favicon JSON |
| Version |
0.5.0
JSON |
| download |
| home_page | None |
| Summary | Find and extract the favicon of any website |
| upload_time | 2024-12-21 22:45:43 |
| maintainer | None |
| docs_url | None |
| author | Alex Mili |
| requires_python | >=3.9 |
| license | MIT License Copyright (c) 2016 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Extract Favicon
---
**Documentation**: <a href="https://alexmili.github.io/extract_favicon" target="_blank">https://alexmili.github.io/extract_favicon</a>
**Source Code**: <a href="https://github.com/alexmili/extract_favicon" target="_blank">https://github.com/alexmili/extract_favicon</a>
---
**Extract Favicon** is designed to easily retrieve favicons from any website. Built atop robust `reachable` and `BeautifulSoup`, it aims to deliver accurate and efficient favicon extraction for web scraping and data analysis workflows.
Key features include:
* **Automatic Extraction**: Detects multiple favicon references like `<link>`, `<meta>` and inline base64-encoded icons.
* **Smart Fallbacks**: When explicit icons aren’t defined, it checks standard fallback routes (like `favicon.ico`) to provide consistent results even on sites without standard declarations.
* **Size Guessing**: Dynamically determines favicon dimensions, even for images lacking explicit size information, by partially downloading and parsing their headers.
* **Base64 Support**: Easily handles inline data URLs, decoding base64-encoded images and validating them on-the-fly.
* **Availability Checks**: Validates each favicon’s URL, following redirects and marking icons as reachable or not.
* **DuckDuckGo Support**: Downloads Favicon directly from DuckDuckGo's public favicon API.
* **Google Support**: Downloads Favicon directly from Google's public favicon API.
* **Custom Strategy**: Sets the order in which the different available techniques are used to retrieve the best favicon.
* **Generate Favicon**: Generate a default SVG favicon when none are available.
* **Get Best Favicon**: Easily gets the best Favicon available, generate one if none are found.
* **Async Support**: Offers asynchronous methods (via `asyncio`) to efficiently handle multiple favicon extractions concurrently, enhancing overall performance when dealing with numerous URLs.
## Installation
Create and activate a virtual environment and then install `extract_favicon`:
```console
$ pip install extract_favicon
```
## Usage
### Extracting Favicons from HTML
The `from_html` function allows you to parse a given HTML string and extract all favicons referenced within it. It looks for common `<link>` and `<meta>` tags that reference icons (e.g., `icon`, `shortcut icon`, `apple-touch-icon`, etc.). If `include_fallbacks` is set to `True`, it will also check standard fallback paths like `favicon.ico` when no icons are explicitly defined.
**Example:**
```python
html_content = """
<!DOCTYPE html>
<html>
<head>
<link rel="icon" href="https://example.com/favicon.ico" />
<link rel="apple-touch-icon" href="/apple-touch-icon.png">
</head>
<body>
<p>Sample page</p>
</body>
</html>
"""
favicons = from_html(html_content, root_url="https://example.com", include_fallbacks=True)
for favicon in favicons:
print(favicon.url, favicon.width, favicon.height)
```
### Extracting Favicons from a URL
If you only have a URL and want to directly extract favicons, `from_url` fetches the page, parses it, and returns a set of `Favicon` objects. It uses `Reachable` internally to check if the URL is accessible. If `include_fallbacks` is True, fallback icons (like `/favicon.ico`) are also considered.
```python
favicons = from_url("https://example.com", include_fallbacks=True)
for favicon in favicons:
print(favicon.url, favicon.format, favicon.width, favicon.height)
```
### Downloading Favicons
Depending on the mode, you can choose to download:
* "all": Download all favicons.
* "biggest": Download only the largest favicon (by area).
* "smallest": Download only the smallest favicon.
If `include_unknown` is False, favicons without known dimensions are skipped. The `sort` option sorts the returned favicons by size, and `sleep_time` controls how long to wait between requests to avoid rate limits.
The result is a list of `RealFavicon` objects, which contain additional information like the loaded image or raw SVG data.
```python
favicons = from_url("https://example.com")
real_favicons = download(favicons, mode="all", sort="DESC")
for real_favicon in real_favicons:
print(real_favicon.url.url, real_favicon.valid, real_favicon.width, real_favicon.height)
```
### Checking Favicon Availability
Sends a HEAD request for each favicon URL to determine if it’s reachable. If the favicon has been redirected, it updates the URL accordingly. It also sets the reachable attribute on each Favicon. The `sleep_time` parameter lets you pause between checks to reduce the load on the target server.
```python
favicons = from_url("https://example.com")
checked_favicons = check_availability(favicons)
for favicon in checked_favicons:
print(favicon.url, favicon.reachable)
```
### Guessing Favicon Sizes
If some extracted favicons don’t have their dimensions specified, `guess_missing_sizes` can attempt to determine their width and height. For base64-encoded favicons (data URLs), setting `load_base64_img` to `True` allows the function to decode and load the image in memory to get its size. For external images, it partially downloads the image to guess its dimensions without retrieving the entire file.
```python
favicons = from_url("https://example.com")
# Some favicons may not have width/height info
favicons_with_sizes = guess_missing_sizes(favicons, load_base64_img=True)
for favicon in favicons_with_sizes:
print(favicon.url, favicon.width, favicon.height)
```
### Generating a Favicon
The generate_favicon function builds a simple placeholder favicon in SVG format based on the first letter of the domain. It’s useful when other methods fail or if you need a fallback icon quickly.
```python
placeholder_favicon = generate_favicon("https://example.com")
# The Favicon object contains the SVG data as if it were a real icon.
print("Generated favicon URL:", placeholder_favicon.url)
```
### Get the Best Favicon Available
The `get_best_favicon` function tries multiple techniques in a specified order to find the best possible favicon. By default, the order is:
* `content`: Attempts to extract favicons from HTML or directly from the URL.
* `duckduckgo`: Fetches a favicon from DuckDuckGo if the first step fails.
* `google`: Retrieves a favicon from Google if the previous steps fails.
* `generate`: Generates a placeholder if no other method is successful.
The function returns the first valid favicon found or None if none is discovered.
```python
best_icon = get_best_favicon("https://example.com")
if best_icon:
print("Best favicon URL:", best_icon.url)
print("Favicon dimensions:", best_icon.width, "x", best_icon.height)
else:
print("No valid favicon found for this URL.")
```
## Dependencies
When you install `extract_favicon` it comes with the following dependencies:
* <a href="https://www.crummy.com/software/BeautifulSoup" target="_blank"><code>BeautifulSoup</code></a> - to parse HTML content.
* <a href="https://github.com/python-pillow/Pillow" target="_blank"><code>Pillow</code></a> - to load images to get real size once downloaded and to guess image size based on its streamed headers.
* <a href="https://github.com/alexmili/reachable" target="_blank"><code>Reachable</code></a> - to check availability of favicons' URLs, download content and handle redirects, HTTP errors and some simple anti-bot protections.
* <a href="https://github.com/tiran/defusedxml" target="_blank"><code>DefusedXML</code></a> - to parse and check validity of SVG files.
* <a href="https://github.com/john-kurkowski/tldextract" target="_blank"><code>TLDextract</code></a> - to parse and extract domain information from URL.
## Inspiration
This library is an extension of the [favicon](https://github.com/scottwernervt/favicon/) package.
## License
This project is licensed under the terms of the MIT license.
Raw data
{
"_id": null,
"home_page": null,
"name": "extract-favicon",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Alex Mili",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/c3/1a/a95276f0ceaffd25665c90afdf19a14ce2ba8f4b44877ab476c90f5e7e6a/extract_favicon-0.5.0.tar.gz",
"platform": null,
"description": "# Extract Favicon\n\n---\n\n**Documentation**: <a href=\"https://alexmili.github.io/extract_favicon\" target=\"_blank\">https://alexmili.github.io/extract_favicon</a>\n\n**Source Code**: <a href=\"https://github.com/alexmili/extract_favicon\" target=\"_blank\">https://github.com/alexmili/extract_favicon</a>\n\n---\n\n**Extract Favicon** is designed to easily retrieve favicons from any website. Built atop robust `reachable` and `BeautifulSoup`, it aims to deliver accurate and efficient favicon extraction for web scraping and data analysis workflows.\n\nKey features include:\n\n* **Automatic Extraction**: Detects multiple favicon references like `<link>`, `<meta>` and inline base64-encoded icons.\n* **Smart Fallbacks**: When explicit icons aren\u2019t defined, it checks standard fallback routes (like `favicon.ico`) to provide consistent results even on sites without standard declarations.\n* **Size Guessing**: Dynamically determines favicon dimensions, even for images lacking explicit size information, by partially downloading and parsing their headers.\n* **Base64 Support**: Easily handles inline data URLs, decoding base64-encoded images and validating them on-the-fly.\n* **Availability Checks**: Validates each favicon\u2019s URL, following redirects and marking icons as reachable or not.\n* **DuckDuckGo Support**: Downloads Favicon directly from DuckDuckGo's public favicon API.\n* **Google Support**: Downloads Favicon directly from Google's public favicon API.\n* **Custom Strategy**: Sets the order in which the different available techniques are used to retrieve the best favicon.\n* **Generate Favicon**: Generate a default SVG favicon when none are available.\n* **Get Best Favicon**: Easily gets the best Favicon available, generate one if none are found.\n* **Async Support**: Offers asynchronous methods (via `asyncio`) to efficiently handle multiple favicon extractions concurrently, enhancing overall performance when dealing with numerous URLs.\n\n## Installation\n\nCreate and activate a virtual environment and then install `extract_favicon`:\n\n```console\n$ pip install extract_favicon\n```\n\n## Usage\n\n\n### Extracting Favicons from HTML\n\nThe `from_html` function allows you to parse a given HTML string and extract all favicons referenced within it. It looks for common `<link>` and `<meta>` tags that reference icons (e.g., `icon`, `shortcut icon`, `apple-touch-icon`, etc.). If `include_fallbacks` is set to `True`, it will also check standard fallback paths like `favicon.ico` when no icons are explicitly defined.\n\n**Example:**\n```python\nhtml_content = \"\"\"\n<!DOCTYPE html>\n<html>\n<head>\n <link rel=\"icon\" href=\"https://example.com/favicon.ico\" />\n <link rel=\"apple-touch-icon\" href=\"/apple-touch-icon.png\">\n</head>\n<body>\n <p>Sample page</p>\n</body>\n</html>\n\"\"\"\n\nfavicons = from_html(html_content, root_url=\"https://example.com\", include_fallbacks=True)\nfor favicon in favicons:\n print(favicon.url, favicon.width, favicon.height)\n```\n\n### Extracting Favicons from a URL\n\nIf you only have a URL and want to directly extract favicons, `from_url` fetches the page, parses it, and returns a set of `Favicon` objects. It uses `Reachable` internally to check if the URL is accessible. If `include_fallbacks` is True, fallback icons (like `/favicon.ico`) are also considered.\n\n```python\nfavicons = from_url(\"https://example.com\", include_fallbacks=True)\nfor favicon in favicons:\n print(favicon.url, favicon.format, favicon.width, favicon.height)\n```\n\n### Downloading Favicons\n\nDepending on the mode, you can choose to download:\n\n* \"all\": Download all favicons.\n* \"biggest\": Download only the largest favicon (by area).\n* \"smallest\": Download only the smallest favicon.\n\nIf `include_unknown` is False, favicons without known dimensions are skipped. The `sort` option sorts the returned favicons by size, and `sleep_time` controls how long to wait between requests to avoid rate limits.\n\nThe result is a list of `RealFavicon` objects, which contain additional information like the loaded image or raw SVG data.\n\n```python\nfavicons = from_url(\"https://example.com\")\nreal_favicons = download(favicons, mode=\"all\", sort=\"DESC\")\n\nfor real_favicon in real_favicons:\n print(real_favicon.url.url, real_favicon.valid, real_favicon.width, real_favicon.height)\n```\n\n### Checking Favicon Availability\n\nSends a HEAD request for each favicon URL to determine if it\u2019s reachable. If the favicon has been redirected, it updates the URL accordingly. It also sets the reachable attribute on each Favicon. The `sleep_time` parameter lets you pause between checks to reduce the load on the target server.\n\n```python\nfavicons = from_url(\"https://example.com\")\nchecked_favicons = check_availability(favicons)\n\nfor favicon in checked_favicons:\n print(favicon.url, favicon.reachable)\n```\n\n### Guessing Favicon Sizes\n\nIf some extracted favicons don\u2019t have their dimensions specified, `guess_missing_sizes` can attempt to determine their width and height. For base64-encoded favicons (data URLs), setting `load_base64_img` to `True` allows the function to decode and load the image in memory to get its size. For external images, it partially downloads the image to guess its dimensions without retrieving the entire file.\n\n```python\nfavicons = from_url(\"https://example.com\")\n# Some favicons may not have width/height info\nfavicons_with_sizes = guess_missing_sizes(favicons, load_base64_img=True)\n\nfor favicon in favicons_with_sizes:\n print(favicon.url, favicon.width, favicon.height)\n```\n\n### Generating a Favicon\n\nThe generate_favicon function builds a simple placeholder favicon in SVG format based on the first letter of the domain. It\u2019s useful when other methods fail or if you need a fallback icon quickly.\n\n```python\nplaceholder_favicon = generate_favicon(\"https://example.com\")\n\n# The Favicon object contains the SVG data as if it were a real icon.\nprint(\"Generated favicon URL:\", placeholder_favicon.url)\n```\n\n### Get the Best Favicon Available\n\nThe `get_best_favicon` function tries multiple techniques in a specified order to find the best possible favicon. By default, the order is:\n\n* `content`: Attempts to extract favicons from HTML or directly from the URL.\n* `duckduckgo`: Fetches a favicon from DuckDuckGo if the first step fails.\n* `google`: Retrieves a favicon from Google if the previous steps fails.\n* `generate`: Generates a placeholder if no other method is successful.\n\nThe function returns the first valid favicon found or None if none is discovered.\n\n```python\nbest_icon = get_best_favicon(\"https://example.com\")\n\nif best_icon:\n print(\"Best favicon URL:\", best_icon.url)\n print(\"Favicon dimensions:\", best_icon.width, \"x\", best_icon.height)\nelse:\n print(\"No valid favicon found for this URL.\")\n```\n\n## Dependencies\n\nWhen you install `extract_favicon` it comes with the following dependencies:\n\n* <a href=\"https://www.crummy.com/software/BeautifulSoup\" target=\"_blank\"><code>BeautifulSoup</code></a> - to parse HTML content.\n* <a href=\"https://github.com/python-pillow/Pillow\" target=\"_blank\"><code>Pillow</code></a> - to load images to get real size once downloaded and to guess image size based on its streamed headers.\n* <a href=\"https://github.com/alexmili/reachable\" target=\"_blank\"><code>Reachable</code></a> - to check availability of favicons' URLs, download content and handle redirects, HTTP errors and some simple anti-bot protections.\n* <a href=\"https://github.com/tiran/defusedxml\" target=\"_blank\"><code>DefusedXML</code></a> - to parse and check validity of SVG files.\n* <a href=\"https://github.com/john-kurkowski/tldextract\" target=\"_blank\"><code>TLDextract</code></a> - to parse and extract domain information from URL.\n\n## Inspiration\nThis library is an extension of the [favicon](https://github.com/scottwernervt/favicon/) package.\n\n## License\n\nThis project is licensed under the terms of the MIT license.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2016 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Find and extract the favicon of any website",
"version": "0.5.0",
"project_urls": {
"Documentation": "https://alexmili.github.io/extract_favicon",
"Homepage": "https://alexmili.github.io/extract_favicon",
"Issues": "https://github.com/AlexMili/Extract_Favicon/issues",
"Repository": "https://github.com/AlexMili/Extract_Favicon"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "46be012a2132aa88ccaf88149d475134899207739903bbb0099f4dd7c6145152",
"md5": "cc4c30948532b5ff3ee79c030d1214b2",
"sha256": "6f1ce9754506ee096cd2f1c0cdb0760a0d7fb3496317166469684117eea57b58"
},
"downloads": -1,
"filename": "extract_favicon-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc4c30948532b5ff3ee79c030d1214b2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 18603,
"upload_time": "2024-12-21T22:45:41",
"upload_time_iso_8601": "2024-12-21T22:45:41.698358Z",
"url": "https://files.pythonhosted.org/packages/46/be/012a2132aa88ccaf88149d475134899207739903bbb0099f4dd7c6145152/extract_favicon-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c31aa95276f0ceaffd25665c90afdf19a14ce2ba8f4b44877ab476c90f5e7e6a",
"md5": "78a84f2c75aebf12966907495d7a9fe1",
"sha256": "fbd9607c15538e2925b6ee2c9ef06139cf15c4ad564878e5e38600bd4acc5d09"
},
"downloads": -1,
"filename": "extract_favicon-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "78a84f2c75aebf12966907495d7a9fe1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 20644,
"upload_time": "2024-12-21T22:45:43",
"upload_time_iso_8601": "2024-12-21T22:45:43.088558Z",
"url": "https://files.pythonhosted.org/packages/c3/1a/a95276f0ceaffd25665c90afdf19a14ce2ba8f4b44877ab476c90f5e7e6a/extract_favicon-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-21 22:45:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AlexMili",
"github_project": "Extract_Favicon",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "extract-favicon"
}
