# Botasaurus Proxy Authentication
Botasaurus Proxy Authentication provides SSL support for authenticated proxies.
Proxy providers like BrightData, IPRoyal, and others typically provide authenticated proxies in the format "http://username:password@proxy-provider-domain:port". For example, "http://greyninja:awesomepassword@geo.iproyal.com:12321".
However, if you use an authenticated proxy with a library like seleniumwire to scrape a Cloudflare protected website like G2.com, you will surely be blocked because you are using a non-SSL connection.
To verify this, run the following code:
First, install the necessary packages:
```bash
python -m pip install selenium_wire chromedriver_autoinstaller
```
Then, execute this Python script:
```python
from seleniumwire import webdriver
from chromedriver_autoinstaller import install
# Define the proxy
proxy_options = {
'proxy': {
'http': 'http://username:password@proxy-provider-domain:port', # TODO: Replace with your own proxy
'https': 'http://username:password@proxy-provider-domain:port', # TODO: Replace with your own proxy
}
}
# Install and set up the driver
driver_path = install()
driver = webdriver.Chrome(driver_path, seleniumwire_options=proxy_options)
# Navigate to the desired URL
driver.get("https://ipinfo.io/")
# Prompt for user input
input("Press Enter to exit...")
# Clean up
driver.quit()
```
You will definetely encounter a block by Cloudflare:
However, using proxies with botasaurus_proxy_authentication prevents this issue. See the difference by running the following code:
First, install the necessary packages:
```bash
python -m pip install botasaurus
```
Then, execute this Python script:
```python
from botasaurus import *
@browser(proxy="http://username:password@proxy-provider-domain:port") # TODO: Replace with your own proxy
def scrape_heading_task(driver: AntiDetectDriver, data):
driver.get("https://ipinfo.io/")
driver.prompt()
scrape_heading_task()
```
Result:
NOTE: To run the code above, you will need Node.js installed.
## Usage with Botasaurus
```python
from botasaurus import *
@browser(proxy="http://username:password@proxy-provider-domain:port") # TODO: Replace with your own proxy
def visit_ipinfo(driver: AntiDetectDriver, data):
driver.get("https://ipinfo.io/")
driver.prompt()
visit_ipinfo()
```
## Usage with Selenium
```python
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from chromedriver_autoinstaller import install
from botasaurus_proxy_authentication import add_proxy_options
# Define the proxy settings
proxy = 'http://username:password@proxy-provider-domain:port' # TODO: Replace with your own proxy
# Set Chrome options
chrome_options = Options()
add_proxy_options(chrome_options, proxy)
# Install and set up the driver
driver_path = install()
driver = webdriver.Chrome(driver_path, options=chrome_options)
# Navigate to the desired URL
driver.get("https://ipinfo.io/")
# Prompt for user input
input("Press Enter to exit...")
# Clean up
driver.quit()
```
## Botasaurus
We encourage you to learn about [Botasaurus](https://github.com/omkarcloud/botasaurus). The All-in-One Web Scraping Framework with Anti-Detection, Parallelization, Asynchronous, and Caching Superpowers.
## Thanks
- Kudos to the Apify Team for creating `proxy-chain` library. The implementation of SSL-based Proxy Authentication wouldn't be possible without their groundbreaking work on `proxy-chain`.
## Love It? [Star It! ⭐](https://github.com/omkarcloud/botasaurus)
Become one of our amazing stargazers by giving us a star ⭐ on GitHub!
It's just one click, but it means the world to me.
[](https://github.com/omkarcloud/botasaurus-proxy-authentication/stargazers)
## Made with ❤️ in Bharat 🇮🇳 - Vande Mataram
Raw data
{
"_id": null,
"home_page": "https://github.com/omkarcloud/botasaurus-proxy-authentication",
"name": "botasaurus-proxy-authentication",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "seleniumwire proxy authentication, proxy authentication",
"author": "Chetan Jain",
"author_email": "chetan@omkar.cloud",
"download_url": "https://files.pythonhosted.org/packages/1e/f0/9a9de72d09666b9e3c66cfac26a9c41fd78cab643d75e044dc73560cc601/botasaurus_proxy_authentication-1.0.16.tar.gz",
"platform": null,
"description": "# Botasaurus Proxy Authentication\r\n\r\nBotasaurus Proxy Authentication provides SSL support for authenticated proxies. \r\n\r\nProxy providers like BrightData, IPRoyal, and others typically provide authenticated proxies in the format \"http://username:password@proxy-provider-domain:port\". For example, \"http://greyninja:awesomepassword@geo.iproyal.com:12321\".\r\n\r\nHowever, if you use an authenticated proxy with a library like seleniumwire to scrape a Cloudflare protected website like G2.com, you will surely be blocked because you are using a non-SSL connection. \r\n\r\nTo verify this, run the following code:\r\n\r\nFirst, install the necessary packages:\r\n```bash \r\npython -m pip install selenium_wire chromedriver_autoinstaller\r\n```\r\n\r\nThen, execute this Python script:\r\n```python\r\nfrom seleniumwire import webdriver\r\nfrom chromedriver_autoinstaller import install\r\n\r\n# Define the proxy\r\nproxy_options = {\r\n 'proxy': {\r\n 'http': 'http://username:password@proxy-provider-domain:port', # TODO: Replace with your own proxy\r\n 'https': 'http://username:password@proxy-provider-domain:port', # TODO: Replace with your own proxy\r\n }\r\n}\r\n\r\n# Install and set up the driver\r\ndriver_path = install()\r\ndriver = webdriver.Chrome(driver_path, seleniumwire_options=proxy_options)\r\n\r\n# Navigate to the desired URL\r\ndriver.get(\"https://ipinfo.io/\")\r\n\r\n# Prompt for user input\r\ninput(\"Press Enter to exit...\")\r\n\r\n# Clean up\r\ndriver.quit()\r\n```\r\n\r\nYou will definetely encounter a block by Cloudflare:\r\n\r\n\r\n\r\nHowever, using proxies with botasaurus_proxy_authentication prevents this issue. See the difference by running the following code:\r\n\r\nFirst, install the necessary packages:\r\n```bash \r\npython -m pip install botasaurus\r\n```\r\n\r\nThen, execute this Python script:\r\n\r\n```python\r\nfrom botasaurus import *\r\n\r\n@browser(proxy=\"http://username:password@proxy-provider-domain:port\") # TODO: Replace with your own proxy \r\ndef scrape_heading_task(driver: AntiDetectDriver, data):\r\n driver.get(\"https://ipinfo.io/\")\r\n driver.prompt()\r\nscrape_heading_task() \r\n``` \r\n\r\nResult: \r\n\r\n\r\nNOTE: To run the code above, you will need Node.js installed.\r\n\r\n\r\n## Usage with Botasaurus \r\n\r\n```python\r\nfrom botasaurus import *\r\n\r\n@browser(proxy=\"http://username:password@proxy-provider-domain:port\") # TODO: Replace with your own proxy \r\ndef visit_ipinfo(driver: AntiDetectDriver, data):\r\n driver.get(\"https://ipinfo.io/\")\r\n driver.prompt()\r\nvisit_ipinfo() \r\n``` \r\n\r\n## Usage with Selenium \r\n\r\n```python\r\nfrom selenium import webdriver\r\nfrom selenium.webdriver.chrome.options import Options\r\nfrom chromedriver_autoinstaller import install\r\nfrom botasaurus_proxy_authentication import add_proxy_options\r\n\r\n# Define the proxy settings\r\nproxy = 'http://username:password@proxy-provider-domain:port' # TODO: Replace with your own proxy\r\n\r\n# Set Chrome options\r\nchrome_options = Options()\r\nadd_proxy_options(chrome_options, proxy)\r\n\r\n# Install and set up the driver\r\ndriver_path = install()\r\ndriver = webdriver.Chrome(driver_path, options=chrome_options)\r\n\r\n# Navigate to the desired URL\r\ndriver.get(\"https://ipinfo.io/\")\r\n\r\n# Prompt for user input\r\ninput(\"Press Enter to exit...\")\r\n\r\n# Clean up\r\ndriver.quit()\r\n``` \r\n\r\n## Botasaurus\r\n\r\nWe encourage you to learn about [Botasaurus](https://github.com/omkarcloud/botasaurus). The All-in-One Web Scraping Framework with Anti-Detection, Parallelization, Asynchronous, and Caching Superpowers.\r\n\r\n## Thanks\r\n\r\n- Kudos to the Apify Team for creating `proxy-chain` library. The implementation of SSL-based Proxy Authentication wouldn't be possible without their groundbreaking work on `proxy-chain`.\r\n\r\n## Love It? [Star It! \u2b50](https://github.com/omkarcloud/botasaurus)\r\n\r\nBecome one of our amazing stargazers by giving us a star \u2b50 on GitHub!\r\n\r\nIt's just one click, but it means the world to me.\r\n\r\n[](https://github.com/omkarcloud/botasaurus-proxy-authentication/stargazers)\r\n\r\n## Made with \u2764\ufe0f in Bharat \ud83c\uddee\ud83c\uddf3 - Vande Mataram\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Proxy Server with support for SSL, proxy authentication and upstream proxy.",
"version": "1.0.16",
"project_urls": {
"Homepage": "https://github.com/omkarcloud/botasaurus-proxy-authentication"
},
"split_keywords": [
"seleniumwire proxy authentication",
" proxy authentication"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1ef09a9de72d09666b9e3c66cfac26a9c41fd78cab643d75e044dc73560cc601",
"md5": "b36ac7229932a8911498afc45950a2ba",
"sha256": "4a7b8bf030acd018288e3e67f771e542f84ac871d1d78c9daf7ad77118bebb8c"
},
"downloads": -1,
"filename": "botasaurus_proxy_authentication-1.0.16.tar.gz",
"has_sig": false,
"md5_digest": "b36ac7229932a8911498afc45950a2ba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 4293,
"upload_time": "2024-05-11T10:41:12",
"upload_time_iso_8601": "2024-05-11T10:41:12.979466Z",
"url": "https://files.pythonhosted.org/packages/1e/f0/9a9de72d09666b9e3c66cfac26a9c41fd78cab643d75e044dc73560cc601/botasaurus_proxy_authentication-1.0.16.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-11 10:41:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "omkarcloud",
"github_project": "botasaurus-proxy-authentication",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "botasaurus-proxy-authentication"
}
JSON
