**web_utils.py Library: Providing a Unified Solution for Web Requests**
===============================
[English Documentation](README.md) | [中文文档](README_CN.md)
> **Note:**
>If there is a conflict between the Chinese and English documents, please prioritize the English document.
In the digital age, data has never been more important. Every day, developers, data scientists, and business analysts are striving to retrieve, analyze, and leverage data from the web. However, there is currently a wide variety of web request libraries, each with its unique interface and usage methods. This not only makes learning and switching costly but also introduces complexity and instability into the code.
The birth of the **WebFetcher** library stems from the desire for a unified, efficient, and straightforward solution for web requests. Whether you are using requests, urllib, or Selenium under the hood, **WebFetcher** provides you with a unified and concise interface. This means you can freely switch backend technologies without altering any core code, ensuring code readability and maintainability.
But this is just our first step. We believe that community strength is essential to making this library truly influential and widely used. Whether you are a beginner or an experienced developer, we sincerely invite you to participate in this project. You can contribute not only by submitting code and improving features but also by sharing your usage experience and suggestions to help us. Only through cooperation can **WebFetcher** truly become an essential tool in every developer's toolbox.
If you are passionate about web development and have an unwavering pursuit of innovation and excellence, please join us! Let's work together to create a powerful, flexible, and user-friendly web request library that serves developers worldwide.
## Installation
You can install the package using pip:
```bash
pip install webmix
```
or
```bash
pip3 install webmix
```
Alternatively, you can clone the repository from GitHub and install it manually:
```bash
git clone https://github.com/ng-fukgin/webmix
cd webmix
python setup.py install
```
#### Using the `WebFetcher` Class
### Using `requests` as the Backend
To use the `WebFetcher` class with `requests` as the backend, you can follow these steps:
```
from webmix.web_utils import WebFetcher
fetcher = WebFetcher(backend='requests')
response = fetcher.get("https://example.com")
print(response)
```
### Using `urllib` as the Backend
To use the `WebFetcher` class with `urllib` as the backend, you can follow these steps:
```
fetcher = WebFetcher(backend='urllib')
response = fetcher.get("https://example.com")
print(response)
```
### Using `selenium` as the Backend
To use the `WebFetcher` class with `selenium` as the backend, you can follow these steps:
```
fetcher = WebFetcher(backend='selenium')
response = fetcher.get("https://example.com")
print(response)
```
# Selenium Helper Methods
```
fetcher.scroll_to_bottom()
fetcher.fill_form_by_id("myFormId", "Some Value")
fetcher.click_button_by_id("myButtonId")
script_output = fetcher.execute_script("return document.title;")
print(script_output)
fetcher.close() # close the `WebDriver` session
```
Note that using Selenium may require additional setup, such as installing the appropriate browser driver.
Raw data
{
"_id": null,
"home_page": "https://github.com/ng-fukgin/webmix",
"name": "webmix",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "webmix,\u7f51\u9875\u6293\u53d6,\u81ea\u52a8\u5316,requests,urllib,selenium,web scraping,automation",
"author": "wfj",
"author_email": "wfj.0000@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a1/00/af05c7ece4227526571f06a02b150e0c57cbe35f092329bb9197c9ffebf4/webmix-0.0.4.tar.gz",
"platform": "any",
"description": "\r\n**web_utils.py Library: Providing a Unified Solution for Web Requests**\r\n===============================\r\n[English Documentation](README.md) | [\u4e2d\u6587\u6587\u6863](README_CN.md)\r\n> **Note:**\r\n>If there is a conflict between the Chinese and English documents, please prioritize the English document.\r\n\r\n\r\n\r\n\r\nIn the digital age, data has never been more important. Every day, developers, data scientists, and business analysts are striving to retrieve, analyze, and leverage data from the web. However, there is currently a wide variety of web request libraries, each with its unique interface and usage methods. This not only makes learning and switching costly but also introduces complexity and instability into the code.\r\n\r\nThe birth of the **WebFetcher** library stems from the desire for a unified, efficient, and straightforward solution for web requests. Whether you are using requests, urllib, or Selenium under the hood, **WebFetcher** provides you with a unified and concise interface. This means you can freely switch backend technologies without altering any core code, ensuring code readability and maintainability.\r\n\r\nBut this is just our first step. We believe that community strength is essential to making this library truly influential and widely used. Whether you are a beginner or an experienced developer, we sincerely invite you to participate in this project. You can contribute not only by submitting code and improving features but also by sharing your usage experience and suggestions to help us. Only through cooperation can **WebFetcher** truly become an essential tool in every developer's toolbox.\r\n\r\nIf you are passionate about web development and have an unwavering pursuit of innovation and excellence, please join us! Let's work together to create a powerful, flexible, and user-friendly web request library that serves developers worldwide.\r\n\r\n## Installation\r\n\r\nYou can install the package using pip:\r\n\r\n```bash\r\npip install webmix\r\n```\r\nor \r\n```bash\r\npip3 install webmix\r\n```\r\n \r\nAlternatively, you can clone the repository from GitHub and install it manually:\r\n\r\n```bash\r\ngit clone https://github.com/ng-fukgin/webmix\r\ncd webmix\r\npython setup.py install\r\n```\r\n\r\n\r\n\r\n#### Using the `WebFetcher` Class\r\n\r\n### Using `requests` as the Backend\r\n\r\nTo use the `WebFetcher` class with `requests` as the backend, you can follow these steps:\r\n```\r\nfrom webmix.web_utils import WebFetcher\r\n\r\nfetcher = WebFetcher(backend='requests')\r\nresponse = fetcher.get(\"https://example.com\")\r\nprint(response)\r\n\r\n```\r\n\r\n### Using `urllib` as the Backend\r\n\r\nTo use the `WebFetcher` class with `urllib` as the backend, you can follow these steps:\r\n\r\n```\r\nfetcher = WebFetcher(backend='urllib')\r\nresponse = fetcher.get(\"https://example.com\")\r\nprint(response)\r\n```\r\n### Using `selenium` as the Backend\r\n\r\nTo use the `WebFetcher` class with `selenium` as the backend, you can follow these steps:\r\n\r\n```\r\nfetcher = WebFetcher(backend='selenium')\r\nresponse = fetcher.get(\"https://example.com\")\r\nprint(response)\r\n```\r\n# Selenium Helper Methods\r\n```\r\nfetcher.scroll_to_bottom()\r\nfetcher.fill_form_by_id(\"myFormId\", \"Some Value\")\r\nfetcher.click_button_by_id(\"myButtonId\")\r\nscript_output = fetcher.execute_script(\"return document.title;\")\r\nprint(script_output)\r\n\r\nfetcher.close() # close the `WebDriver` session\r\n```\r\nNote that using Selenium may require additional setup, such as installing the appropriate browser driver.\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A set of utilities for web scraping and automation, supporting multiple backends: requests, urllib, and selenium.",
"version": "0.0.4",
"project_urls": {
"Bug Tracker": "https://github.com/ng-fukgin/webmix/issues",
"Documentation": "https://github.com/ng-fukgin/webmix/wiki",
"Homepage": "https://github.com/ng-fukgin/webmix",
"Source Code": "https://github.com/ng-fukgin/webmix"
},
"split_keywords": [
"webmix",
"\u7f51\u9875\u6293\u53d6",
"\u81ea\u52a8\u5316",
"requests",
"urllib",
"selenium",
"web scraping",
"automation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d1a9261bfba471d40d05aedcf9abb7713d1e9c3ad552efc3398de0ffa7583364",
"md5": "fb160e4176a146f01e91bc08248f53cb",
"sha256": "8216238586e1634855314a6990ce05b8591ac54d409d114d07569047a4a354dc"
},
"downloads": -1,
"filename": "webmix-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fb160e4176a146f01e91bc08248f53cb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 4674,
"upload_time": "2023-10-12T03:24:12",
"upload_time_iso_8601": "2023-10-12T03:24:12.298295Z",
"url": "https://files.pythonhosted.org/packages/d1/a9/261bfba471d40d05aedcf9abb7713d1e9c3ad552efc3398de0ffa7583364/webmix-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a100af05c7ece4227526571f06a02b150e0c57cbe35f092329bb9197c9ffebf4",
"md5": "ee4edd4e378777e1f2a1e0fed18ed9f7",
"sha256": "ec02791d4c0f83e5df3431315b2ddfd78e397f6440b65fdc2c0dfdde90057149"
},
"downloads": -1,
"filename": "webmix-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "ee4edd4e378777e1f2a1e0fed18ed9f7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 4627,
"upload_time": "2023-10-12T03:24:14",
"upload_time_iso_8601": "2023-10-12T03:24:14.730860Z",
"url": "https://files.pythonhosted.org/packages/a1/00/af05c7ece4227526571f06a02b150e0c57cbe35f092329bb9197c9ffebf4/webmix-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-12 03:24:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ng-fukgin",
"github_project": "webmix",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "webmix"
}
JSON
