| Name | async-rotating-proxy JSON |
| Version |
0.0.6
JSON |
| download |
| home_page | |
| Summary | Run an API on your local machine that reroutes traffic through rotating proxies. Useful for pyppeteer proxy rotation. |
| upload_time | 2023-05-08 00:28:56 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >3.10 |
| license | |
| keywords |
fastapi
proxy
pyppeteer
rotating proxy
scraping
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# async-rotating-proxy
Run an API on your local machine that reroutes traffic through rotating proxies.
- Excellent for scaling your web scraping
- Designed for page-by-page dynamic proxy switching with pyppeteer
## Installation
`pip install async_rotating_proxy`
## Usage
```py
from async_rotating_proxy import ProxyAPI
import pyppeteer
proxies = [
ip:port,
ip:port,
ip:port
]
with ProxyAPI(proxies) as api:
url = api.format_url("http://checkip.dyndns.org")
browser = await pyppeteer.launch()
page = await browser.newPage()
await page.goto(url)
```
## Purpose
The article referenced below sums it up pretty well:
"The chrome browser does not support fain-grained proxy configuration out of the box. Therefore, the following use cases are not possible when using puppeteer in combination with Google Chrome:
Using different proxies for different tabs/windows
Switching proxies without restarting the browser
This is a bit annoying, because restarting the entire browser is an expensive operation in terms of computational resources. The chrome restart takes up to two seconds (depending on the system). We ideally want to switch proxies whenever the need arises without restarting the entire chrome process. This is a common requirement when scraping websites in scale."
## Reference
https://incolumitas.com/2020/12/20/dynamically-changing-puppeteer-proxies/
Raw data
{
"_id": null,
"home_page": "",
"name": "async-rotating-proxy",
"maintainer": "",
"docs_url": null,
"requires_python": ">3.10",
"maintainer_email": "",
"keywords": "fastapi,proxy,pyppeteer,rotating proxy,scraping",
"author": "",
"author_email": "Greg Bizup <g.bizup@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/16/19/9b3d92bd304e4a4b1815d143708bd76d3767068103b8019c0319a36930a0/async_rotating_proxy-0.0.6.tar.gz",
"platform": null,
"description": "# async-rotating-proxy\nRun an API on your local machine that reroutes traffic through rotating proxies.\n\n- Excellent for scaling your web scraping\n- Designed for page-by-page dynamic proxy switching with pyppeteer\n\n## Installation\n`pip install async_rotating_proxy`\n\n## Usage\n```py\nfrom async_rotating_proxy import ProxyAPI\nimport pyppeteer\n\nproxies = [\n ip:port,\n ip:port,\n ip:port\n]\n\nwith ProxyAPI(proxies) as api:\n url = api.format_url(\"http://checkip.dyndns.org\")\n\n browser = await pyppeteer.launch()\n page = await browser.newPage()\n await page.goto(url)\n```\n\n\n## Purpose\nThe article referenced below sums it up pretty well:\n\"The chrome browser does not support fain-grained proxy configuration out of the box. Therefore, the following use cases are not possible when using puppeteer in combination with Google Chrome:\n\n Using different proxies for different tabs/windows\n Switching proxies without restarting the browser\n\nThis is a bit annoying, because restarting the entire browser is an expensive operation in terms of computational resources. The chrome restart takes up to two seconds (depending on the system). We ideally want to switch proxies whenever the need arises without restarting the entire chrome process. This is a common requirement when scraping websites in scale.\"\n\n## Reference\nhttps://incolumitas.com/2020/12/20/dynamically-changing-puppeteer-proxies/\n",
"bugtrack_url": null,
"license": "",
"summary": "Run an API on your local machine that reroutes traffic through rotating proxies. Useful for pyppeteer proxy rotation.",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://github.com/gbiz123/async-rotating-proxy"
},
"split_keywords": [
"fastapi",
"proxy",
"pyppeteer",
"rotating proxy",
"scraping"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e8742c2bfa1ed152f37616540be567b1b935f4cf8a854bc78e24e07239e827f4",
"md5": "64c171c990dc134c5f19fc39b454e21b",
"sha256": "b3f14cfa61c08b37adce84d1989d9780891d938e2b4d47ce881503551938a961"
},
"downloads": -1,
"filename": "async_rotating_proxy-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "64c171c990dc134c5f19fc39b454e21b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">3.10",
"size": 5096,
"upload_time": "2023-05-08T00:28:55",
"upload_time_iso_8601": "2023-05-08T00:28:55.018593Z",
"url": "https://files.pythonhosted.org/packages/e8/74/2c2bfa1ed152f37616540be567b1b935f4cf8a854bc78e24e07239e827f4/async_rotating_proxy-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "16199b3d92bd304e4a4b1815d143708bd76d3767068103b8019c0319a36930a0",
"md5": "beb1810c5d4b0109783c73777e483a2a",
"sha256": "30dcd391ce4a884a53321352a8e8258126432019ab54dc82bde0a3b6b49ef6de"
},
"downloads": -1,
"filename": "async_rotating_proxy-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "beb1810c5d4b0109783c73777e483a2a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">3.10",
"size": 3947,
"upload_time": "2023-05-08T00:28:56",
"upload_time_iso_8601": "2023-05-08T00:28:56.963371Z",
"url": "https://files.pythonhosted.org/packages/16/19/9b3d92bd304e4a4b1815d143708bd76d3767068103b8019c0319a36930a0/async_rotating_proxy-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-08 00:28:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "gbiz123",
"github_project": "async-rotating-proxy",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "async-rotating-proxy"
}
