| Name | proxyproviders JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | A unified interface for different proxy providers |
| upload_time | 2025-02-18 03:09:33 |
| maintainer | None |
| docs_url | None |
| author | David Teather |
| requires_python | >=3.9 |
| license | MIT License
Copyright (c) 2025 David Teather
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE. |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ProxyProviders
The Unified Python Proxy API for managing proxies with support for BrightData, WebShare, and more.
[](https://www.linkedin.com/in/davidteather/) [](https://github.com/sponsors/davidteather) [](https://github.com/davidteather/proxyproviders/releases) [](https://github.com/davidteather/proxyproviders/blob/main/LICENSE) [](https://pypi.org/project/proxyproviders/)  [](https://discord.gg/yyPhbfma6f)
## Table of Contents
- [Documentation](#documentation)
- [Getting Started](#getting-started)
- [How to Support The Project](#how-to-support-the-project)
- [Installing](#installing)
- [Quick Start Guide](#quick-start-guide)
- [Examples](https://github.com/davidteather/proxyproviders/tree/main/examples)
## Documentation
You can find the full documentation [here](https://davidteather.github.io/proxyproviders)
**Note:** If you want to learn how to web scrape websites check my [free and open-source course for learning everything web scraping](https://github.com/davidteather/everything-web-scraping)
### How to Support The Project
- Star the repo 😎
- Consider [sponsoring](https://github.com/sponsors/davidteather) me on GitHub
- Send me an email or a [LinkedIn](https://www.linkedin.com/in/davidteather/) message about what you're doing with it :D
- Submit PRs for issues or any new providers/features :)
## Getting Started
To get started using this package follow the instructions below.
### Installation
This package is installable via pip.
```sh
python -m pip install proxyproviders
```
## Quick Start Guide
Here's a quick bit of code to get you started! There's also a few [examples in the folder](https://github.com/davidteather/proxyproviders/tree/main/examples).
**Note:** If you want to learn how to web scrape websites check my [free and open-source course for learning everything web scraping](https://github.com/davidteather/everything-web-scraping)
### Choose a Proxy Provider
If you already haven't, choose which proxy provider to use. You can find a [list in the documentation](https://davidteather.github.io/proxyproviders/#proxyproviders-supported-providers). After choosing, look at the documentation around the specific provider you've choosen. Where needed, we've laid out steps to get api keys in the documentation. These steps will vary slightly by provider. For this example I'll be using the [Webshare Provider](https://davidteather.github.io/proxyproviders/#proxyproviders.providers.webshare.Webshare) because it's both easy to setup and they give you 10 free data center proxies to test out.
You can create an account on webshare [here](https://www.webshare.io/?referral_code=3x5812idzzzp) (affiliate link), then head into the [API tab](https://dashboard.webshare.io/userapi/keys) on the side and generate a new token. Keep this API token safe and don't post it publically.
### Basic Example
After you can list out your proxies with
```py
from proxyproviders import Webshare
proxy_provider = Webshare(api_key="your-api-key")
proxies = proxy_provider.list_proxies()
print(proxies)
```
Each provider has their own custom options, the `Webshare` class lets you specify url params according to their [api spec](https://apidocs.webshare.io/proxy-list/list#parameters), here's an example which will only return proxies that are based in the US.
```py
proxy_provider = Webshare(api_key="your-api-key", params={"country_code_in": "US"})
```
### Using ProxyConfig
For any shared logic across all types of proxy providers, we use the `ProxyConfig` data class to configure them. The full docs for [ProxyConfig are here](https://davidteather.github.io/proxyproviders/#proxyproviders.proxy_provider.ProxyConfig). In this example we will configure it to use a shorter `refresh_interval` than default.
```py
from proxyproviders import Webshare, ProxyConfig
import time
config = ProxyConfig(refresh_interval=3)
ws = Webshare(api_key="your-api-token", config=config)
proxies = ws.list_proxies() # calls API
ws.list_proxies() # cached
time.sleep(5)
ws.list_proxies() # calls API since it's more than 3s later
```
### Function Using Generic ProxyProvider
Since all proxy providers implement the same interface, we can make a function that allows us to easily swap out and utilize different providers. This is the main appeal of having a unified interface. It allows other modules to be provider agnostic, like my [TikTokAPI](https://github.com/davidteather/TikTok-Api) package.
```py
from proxyproviders import Webshare, BrightData, ProxyProvider, ProxyConfig
def some_function(provider: ProxyProvider):
proxies = provider.list_proxies()
print(proxies)
webshare = Webshare(api_key="your_api_key")
brightdata = BrightData(api_key="your_api_key", zone="my_zone")
some_function(webshare)
some_function(brightdata)
```
Here's a more meaningful example that takes the `Proxy` class and uses it to create a python requests http proxy.
```py
from proxyproviders import Webshare, BrightData, ProxyProvider
import requests
import os
def request_with_proxy(provider: ProxyProvider):
requests_proxy = None
if provider:
proxies = provider.list_proxies()
requests_proxy = {
"http": f"http://{proxies[0].username}:{proxies[0].password}@{proxies[0].proxy_address}:{proxies[0].port}",
"https": f"http://{proxies[0].username}:{proxies[0].password}@{proxies[0].proxy_address}:{proxies[0].port}",
}
r = requests.get("https://httpbin.org/ip", proxies=requests_proxy)
return r.json()
webshare = Webshare(api_key="your_api_key")
brightdata = BrightData(api_key="your_api_key", zone="your_zone")
print(f"Your IP: {request_with_proxy(None)}")
print(f"Webshare: {request_with_proxy(webshare)}")
print(f"BrightData: {request_with_proxy(brightdata)}")
```
### Making Your Own Proxy Provider
Here's a skeleton of how you can make your very own `ProxyProvider` class. You'll need to implemenet all the required functions of the `ProxyProvider` which may be more than what's here at the time of writing.
If you do find yourself making one of these, consider contributing it back to the repository so everyone can use them :D
```py
from proxyproviders import ProxyProvider, ProxyConfig, Proxy
from typing import List, Optional
class MyProxyProvider(ProxyProvider):
def __init__(self, config: Optional[ProxyConfig] = None):
super().__init__(config=config)
def _fetch_proxies(self):
proxies: List[Proxy] = []
for i in range(10):
# TODO: your real proxy fetching logic
# There are required fields on the Proxy class, be sure that these are filled out properly
# especially if you're using it with another library.
proxy = Proxy(
id=str(i),
username="username",
password="password",
proxy_address="192.168.1.1",
port=80,
)
proxies.append(proxy)
return proxies
def some_function(provider: ProxyProvider):
proxies = provider.list_proxies()
for proxy in proxies:
print(proxy)
provider = MyProxyProvider()
some_function(provider) # calls the function with the provider
```
Raw data
{
"_id": null,
"home_page": null,
"name": "proxyproviders",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "David Teather",
"author_email": "contact.davidteather@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d4/fc/b99446e02cfacd948673dc177ac27d35ba29ebbb0ce3083efd6c4d60e04c/proxyproviders-0.1.1.tar.gz",
"platform": null,
"description": "# ProxyProviders\n\nThe Unified Python Proxy API for managing proxies with support for BrightData, WebShare, and more.\n\n[](https://www.linkedin.com/in/davidteather/) [](https://github.com/sponsors/davidteather) [](https://github.com/davidteather/proxyproviders/releases) [](https://github.com/davidteather/proxyproviders/blob/main/LICENSE) [](https://pypi.org/project/proxyproviders/)  [](https://discord.gg/yyPhbfma6f)\n\n## Table of Contents\n\n- [Documentation](#documentation)\n- [Getting Started](#getting-started)\n - [How to Support The Project](#how-to-support-the-project)\n - [Installing](#installing)\n- [Quick Start Guide](#quick-start-guide)\n - [Examples](https://github.com/davidteather/proxyproviders/tree/main/examples)\n\n## Documentation\n\nYou can find the full documentation [here](https://davidteather.github.io/proxyproviders)\n\n**Note:** If you want to learn how to web scrape websites check my [free and open-source course for learning everything web scraping](https://github.com/davidteather/everything-web-scraping)\n\n### How to Support The Project\n\n- Star the repo \ud83d\ude0e\n- Consider [sponsoring](https://github.com/sponsors/davidteather) me on GitHub\n- Send me an email or a [LinkedIn](https://www.linkedin.com/in/davidteather/) message about what you're doing with it :D \n- Submit PRs for issues or any new providers/features :)\n\n## Getting Started\n\nTo get started using this package follow the instructions below.\n\n### Installation\n\nThis package is installable via pip.\n```sh\npython -m pip install proxyproviders\n```\n\n## Quick Start Guide\n\nHere's a quick bit of code to get you started! There's also a few [examples in the folder](https://github.com/davidteather/proxyproviders/tree/main/examples). \n\n**Note:** If you want to learn how to web scrape websites check my [free and open-source course for learning everything web scraping](https://github.com/davidteather/everything-web-scraping)\n\n### Choose a Proxy Provider\n\nIf you already haven't, choose which proxy provider to use. You can find a [list in the documentation](https://davidteather.github.io/proxyproviders/#proxyproviders-supported-providers). After choosing, look at the documentation around the specific provider you've choosen. Where needed, we've laid out steps to get api keys in the documentation. These steps will vary slightly by provider. For this example I'll be using the [Webshare Provider](https://davidteather.github.io/proxyproviders/#proxyproviders.providers.webshare.Webshare) because it's both easy to setup and they give you 10 free data center proxies to test out.\n\nYou can create an account on webshare [here](https://www.webshare.io/?referral_code=3x5812idzzzp) (affiliate link), then head into the [API tab](https://dashboard.webshare.io/userapi/keys) on the side and generate a new token. Keep this API token safe and don't post it publically.\n\n### Basic Example\n\nAfter you can list out your proxies with\n```py\nfrom proxyproviders import Webshare\n\nproxy_provider = Webshare(api_key=\"your-api-key\")\n\nproxies = proxy_provider.list_proxies()\n\nprint(proxies)\n```\n\nEach provider has their own custom options, the `Webshare` class lets you specify url params according to their [api spec](https://apidocs.webshare.io/proxy-list/list#parameters), here's an example which will only return proxies that are based in the US.\n\n```py\nproxy_provider = Webshare(api_key=\"your-api-key\", params={\"country_code_in\": \"US\"})\n```\n\n### Using ProxyConfig\n\nFor any shared logic across all types of proxy providers, we use the `ProxyConfig` data class to configure them. The full docs for [ProxyConfig are here](https://davidteather.github.io/proxyproviders/#proxyproviders.proxy_provider.ProxyConfig). In this example we will configure it to use a shorter `refresh_interval` than default.\n\n```py\nfrom proxyproviders import Webshare, ProxyConfig\nimport time\n\nconfig = ProxyConfig(refresh_interval=3)\nws = Webshare(api_key=\"your-api-token\", config=config)\n\nproxies = ws.list_proxies() # calls API \n\nws.list_proxies() # cached\n\ntime.sleep(5)\nws.list_proxies() # calls API since it's more than 3s later\n```\n\n### Function Using Generic ProxyProvider\n\nSince all proxy providers implement the same interface, we can make a function that allows us to easily swap out and utilize different providers. This is the main appeal of having a unified interface. It allows other modules to be provider agnostic, like my [TikTokAPI](https://github.com/davidteather/TikTok-Api) package.\n\n```py\nfrom proxyproviders import Webshare, BrightData, ProxyProvider, ProxyConfig\n\ndef some_function(provider: ProxyProvider):\n proxies = provider.list_proxies()\n print(proxies)\n\nwebshare = Webshare(api_key=\"your_api_key\")\nbrightdata = BrightData(api_key=\"your_api_key\", zone=\"my_zone\")\n\nsome_function(webshare)\nsome_function(brightdata)\n```\n\nHere's a more meaningful example that takes the `Proxy` class and uses it to create a python requests http proxy.\n\n```py\nfrom proxyproviders import Webshare, BrightData, ProxyProvider\nimport requests\nimport os\n\n\ndef request_with_proxy(provider: ProxyProvider):\n requests_proxy = None\n\n if provider:\n proxies = provider.list_proxies()\n\n requests_proxy = {\n \"http\": f\"http://{proxies[0].username}:{proxies[0].password}@{proxies[0].proxy_address}:{proxies[0].port}\",\n \"https\": f\"http://{proxies[0].username}:{proxies[0].password}@{proxies[0].proxy_address}:{proxies[0].port}\",\n }\n\n r = requests.get(\"https://httpbin.org/ip\", proxies=requests_proxy)\n return r.json()\n\nwebshare = Webshare(api_key=\"your_api_key\")\nbrightdata = BrightData(api_key=\"your_api_key\", zone=\"your_zone\")\n\nprint(f\"Your IP: {request_with_proxy(None)}\")\nprint(f\"Webshare: {request_with_proxy(webshare)}\")\nprint(f\"BrightData: {request_with_proxy(brightdata)}\")\n```\n\n### Making Your Own Proxy Provider\n\nHere's a skeleton of how you can make your very own `ProxyProvider` class. You'll need to implemenet all the required functions of the `ProxyProvider` which may be more than what's here at the time of writing.\n\nIf you do find yourself making one of these, consider contributing it back to the repository so everyone can use them :D\n\n```py\nfrom proxyproviders import ProxyProvider, ProxyConfig, Proxy\nfrom typing import List, Optional\n\nclass MyProxyProvider(ProxyProvider):\n def __init__(self, config: Optional[ProxyConfig] = None):\n super().__init__(config=config)\n\n def _fetch_proxies(self):\n proxies: List[Proxy] = []\n\n for i in range(10):\n # TODO: your real proxy fetching logic\n\n # There are required fields on the Proxy class, be sure that these are filled out properly\n # especially if you're using it with another library.\n proxy = Proxy(\n id=str(i),\n username=\"username\",\n password=\"password\",\n proxy_address=\"192.168.1.1\",\n port=80,\n )\n\n proxies.append(proxy)\n\n return proxies\n\ndef some_function(provider: ProxyProvider):\n proxies = provider.list_proxies()\n for proxy in proxies:\n print(proxy)\n\nprovider = MyProxyProvider()\nsome_function(provider) # calls the function with the provider\n```\n\n\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2025 David Teather\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.",
"summary": "A unified interface for different proxy providers",
"version": "0.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "4c06ed5f5039a77f3d56baf8c6ef7a21a979c916af2448e33a91884994c85f7f",
"md5": "b89f70f7cca1a3967bdddca37ee91285",
"sha256": "e052138c44e09a0d55c16d13ae706f47ae391458765f18e4fb7bb4ba84125563"
},
"downloads": -1,
"filename": "proxyproviders-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b89f70f7cca1a3967bdddca37ee91285",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 12605,
"upload_time": "2025-02-18T03:09:29",
"upload_time_iso_8601": "2025-02-18T03:09:29.878251Z",
"url": "https://files.pythonhosted.org/packages/4c/06/ed5f5039a77f3d56baf8c6ef7a21a979c916af2448e33a91884994c85f7f/proxyproviders-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d4fcb99446e02cfacd948673dc177ac27d35ba29ebbb0ce3083efd6c4d60e04c",
"md5": "5ca73c222aeb79682aa7cc588bc683d0",
"sha256": "d6acd1e5f4ffdc37bdf4dd91e909d1e1d759da694b017f84f80c73918e455714"
},
"downloads": -1,
"filename": "proxyproviders-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "5ca73c222aeb79682aa7cc588bc683d0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 11794,
"upload_time": "2025-02-18T03:09:33",
"upload_time_iso_8601": "2025-02-18T03:09:33.672493Z",
"url": "https://files.pythonhosted.org/packages/d4/fc/b99446e02cfacd948673dc177ac27d35ba29ebbb0ce3083efd6c4d60e04c/proxyproviders-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-18 03:09:33",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "proxyproviders"
}
