# ShadowServer
`shadowserver` is an asynchronous HTTP proxy server library using `aiohttp`, designed to forward requests from clients to a target server. It handles HTTP and WebSocket connections, provides CORS support, and offers flexible SSL verification options, making it ideal for backend service proxying and server simulations in testing environments.
## Features
- **HTTP Proxying**: Supports forwarding HTTP requests to a specified target.
- **CORS Support**: Automatically adds headers to enable cross-origin requests.
- **WebSocket Support**: Forwards WebSocket connections between client and target server.
- **Flexible SSL Verification**: Allows disabling SSL verification for outgoing HTTPS requests.
- **Route Support**: Ability to append a custom route to the proxy server URL.
- **Asynchronous Design**: Built with `aiohttp` for handling multiple concurrent requests efficiently.
---
## Installation
You can install `shadowserver` via pip:
```bash
pip install shadowserver
```
---
## Usage
Below is a basic example of how to set up and run `shadowserver`.
### Basic Example
```python
from shadowserver import ShadowServer
import asyncio
async def main():
# Initialize the server with the target URL and optional settings
proxy = ShadowServer(
target_base_url="https://example.com",
timeout=30,
max_conn=100,
open_on_browser=True,
verify_ssl=True
)
# Start the server
await proxy.start_server(host="127.0.0.1", port=8080)
# Run the server
asyncio.run(main())
```
### Disabling SSL Verification
To disable SSL verification for outgoing HTTPS requests, pass `verify_ssl=False` during initialization:
```python
proxy = ShadowServer(
target_base_url="https://example.com",
verify_ssl=False # Disables SSL verification for HTTPS requests
)
```
This can be useful for development environments where the target server uses a self-signed certificate.
---
## ShadowServer URL Redirection
The `ShadowServer` class includes an optional redirect feature to automatically redirect requests from the base URL to a specified URL. This is enabled by passing a `redirect_url` and setting `redirects=True`.
### Parameters
- **redirect_url**: `str`
The URL to redirect to when the base URL (`/`) is accessed.
- **redirects**: `bool`
Enables or disables the redirect from the base URL to `redirect_url`.
### Example Usage
Here are some examples showing how to configure the `ShadowServer` with URL redirection.
#### Example 1: Redirecting Requests from Base URL to Another URL
In this example, requests to the base URL (`/`) will be redirected to the URL specified in `redirect_url`:
```python
from shadowserver import ShadowServer
import asyncio
BASE_URL = "https://example.com/api"
REDIRECT_URL = "https://example.com/home"
server = ShadowServer(
target_base_url=BASE_URL,
redirect_url=REDIRECT_URL,
redirects=True
)
asyncio.run(server.start_server(
host="127.0.0.1",
port=3000
))
```
- Requests to `http://127.0.0.1:3000/` will automatically redirect to `https://example.com/home`.
- All other requests (e.g., `http://127.0.0.1:3000/some/path`) will be proxied to `https://example.com/api/some/path`.
#### Example 2: Disabling Redirection
To use `ShadowServer` as a proxy without redirection, omit `redirect_url` and set `redirects=False`:
```python
from shadowserver import ShadowServer
import asyncio
BASE_URL = "https://example.com/api"
server = ShadowServer(
target_base_url=BASE_URL,
redirects=False # Disables redirection
)
asyncio.run(server.start_server(
host="127.0.0.1",
port=3000
))
```
---
## Advanced Configuration
### Setting a Custom Route
You can specify a custom route that will be appended to the base URL. This is useful when you want the server to be accessible via a specific route.
```python
proxy = ShadowServer(
target_base_url="https://example.com",
route="/customroute"
)
asyncio.run(proxy.start_server(host="127.0.0.1", port=8080))
```
### Setting Timeout and Maximum Connections
To configure custom timeouts and connection limits during initialization:
```python
proxy = ShadowServer(target_base_url="https://example.com", timeout=60, max_conn=200)
```
This example sets a 60-second timeout and allows up to 200 concurrent connections.
---
## API Reference
### ShadowServer
The main class for setting up and running the proxy server.
```python
class ShadowServer:
def __init__(self, target_base_url, timeout=30, max_conn=100)
```
- **Parameters**:
- `target_base_url` (str): The base URL to which all proxied requests are forwarded.
- `timeout` (int, optional): Timeout in seconds for requests to the target server. Default is `30`.
- `max_conn` (int, optional): Maximum number of concurrent connections. Default is `100`.
- `redirect_url` (str, optional): URL for redirecting requests from the base URL.
- `redirects` (bool, optional): If `True`, enables redirection to `redirect_url`. Default is `False`.
- `open_on_browser` (bool, optional): Automatically opens the server URL in a browser when started. Default is `True`.
- `verify_ssl` (bool, optional): If `False`, disables SSL verification. Default is `True`.
- `route` (str, optional): Appends a custom route to the server URL.
#### Methods
1. **`start_server`**
```python
async def start_server(self, host='127.0.0.1', port=8080)
```
Starts the proxy server.
- **Parameters**:
- `host` (str, optional): The host IP on which the server runs. Default is `'127.0.0.1'`.
- `port` (int, optional): The port on which the server listens. Default is `8080`.
2. **`close`**
```python
async def close(self)
```
Closes the server session and frees up resources.
---
## Request Handling
The `ShadowServer` proxy server processes requests as follows:
1. **handle_request**: Forwards HTTP and HTTPS requests to the target server and returns the response to the client.
2. **handle_websocket**: Forwards WebSocket connections to the target server.
3. **build_response**: Builds the response, applies custom headers (such as CORS), and sends it to the client.
### Example of Proxying a GET Request
Once the server is running, you can make a GET request to any endpoint available on the target server:
```bash
curl http://127.0.0.1:8080/api/resource
```
This request will be proxied to `https://example.com/api/resource`.
### WebSocket Proxying
The proxy supports WebSocket connections. You can connect to the WebSocket server via the proxy as shown below:
```python
import websockets
import asyncio
async def connect():
uri = "ws://127.0.0.1:8080/socket"
async with websockets.connect(uri) as websocket:
await websocket.send("Hello, World!")
response = await websocket.recv()
print(response)
asyncio.run(connect())
```
---
## Troubleshooting
### CORS Errors
If you encounter CORS issues, ensure that the client request headers include the correct `Origin`.
### SSL Verification Errors
To disable SSL verification, set `verify_ssl=False` when initializing the server.
Raw data
{
"_id": null,
"home_page": "https://github.com/benkimz/shadowserver",
"name": "shadowserver",
"maintainer": "benkimz",
"docs_url": null,
"requires_python": null,
"maintainer_email": "benkim3619@gmail.com",
"keywords": "proxy, server, proxy-server, http, https, aiohttp, asyncio",
"author": "benkimz",
"author_email": "benkim3619@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/44/c8/d1b04a2793cf505a512d502991bccf1796102c36572ea6e1b64487809c11/shadowserver-0.2.4.tar.gz",
"platform": "any",
"description": "# ShadowServer\r\n\r\n`shadowserver` is an asynchronous HTTP proxy server library using `aiohttp`, designed to forward requests from clients to a target server. It handles HTTP and WebSocket connections, provides CORS support, and offers flexible SSL verification options, making it ideal for backend service proxying and server simulations in testing environments.\r\n\r\n## Features\r\n\r\n- **HTTP Proxying**: Supports forwarding HTTP requests to a specified target.\r\n- **CORS Support**: Automatically adds headers to enable cross-origin requests.\r\n- **WebSocket Support**: Forwards WebSocket connections between client and target server.\r\n- **Flexible SSL Verification**: Allows disabling SSL verification for outgoing HTTPS requests.\r\n- **Route Support**: Ability to append a custom route to the proxy server URL.\r\n- **Asynchronous Design**: Built with `aiohttp` for handling multiple concurrent requests efficiently.\r\n\r\n---\r\n\r\n## Installation\r\n\r\nYou can install `shadowserver` via pip:\r\n\r\n```bash\r\npip install shadowserver\r\n```\r\n\r\n---\r\n\r\n## Usage\r\n\r\nBelow is a basic example of how to set up and run `shadowserver`.\r\n\r\n### Basic Example\r\n\r\n```python\r\nfrom shadowserver import ShadowServer\r\nimport asyncio\r\n\r\nasync def main():\r\n # Initialize the server with the target URL and optional settings\r\n proxy = ShadowServer(\r\n target_base_url=\"https://example.com\",\r\n timeout=30,\r\n max_conn=100,\r\n open_on_browser=True,\r\n verify_ssl=True\r\n )\r\n\r\n # Start the server\r\n await proxy.start_server(host=\"127.0.0.1\", port=8080)\r\n\r\n# Run the server\r\nasyncio.run(main())\r\n```\r\n\r\n### Disabling SSL Verification\r\n\r\nTo disable SSL verification for outgoing HTTPS requests, pass `verify_ssl=False` during initialization:\r\n\r\n```python\r\nproxy = ShadowServer(\r\n target_base_url=\"https://example.com\",\r\n verify_ssl=False # Disables SSL verification for HTTPS requests\r\n)\r\n```\r\n\r\nThis can be useful for development environments where the target server uses a self-signed certificate.\r\n\r\n---\r\n\r\n## ShadowServer URL Redirection\r\n\r\nThe `ShadowServer` class includes an optional redirect feature to automatically redirect requests from the base URL to a specified URL. This is enabled by passing a `redirect_url` and setting `redirects=True`.\r\n\r\n### Parameters\r\n\r\n- **redirect_url**: `str` \r\n The URL to redirect to when the base URL (`/`) is accessed.\r\n- **redirects**: `bool` \r\n Enables or disables the redirect from the base URL to `redirect_url`.\r\n\r\n### Example Usage\r\n\r\nHere are some examples showing how to configure the `ShadowServer` with URL redirection.\r\n\r\n#### Example 1: Redirecting Requests from Base URL to Another URL\r\n\r\nIn this example, requests to the base URL (`/`) will be redirected to the URL specified in `redirect_url`:\r\n\r\n```python\r\nfrom shadowserver import ShadowServer\r\nimport asyncio\r\n\r\nBASE_URL = \"https://example.com/api\"\r\nREDIRECT_URL = \"https://example.com/home\"\r\n\r\nserver = ShadowServer(\r\n target_base_url=BASE_URL,\r\n redirect_url=REDIRECT_URL,\r\n redirects=True\r\n)\r\n\r\nasyncio.run(server.start_server(\r\n host=\"127.0.0.1\",\r\n port=3000\r\n))\r\n```\r\n\r\n- Requests to `http://127.0.0.1:3000/` will automatically redirect to `https://example.com/home`.\r\n- All other requests (e.g., `http://127.0.0.1:3000/some/path`) will be proxied to `https://example.com/api/some/path`.\r\n\r\n#### Example 2: Disabling Redirection\r\n\r\nTo use `ShadowServer` as a proxy without redirection, omit `redirect_url` and set `redirects=False`:\r\n\r\n```python\r\nfrom shadowserver import ShadowServer\r\nimport asyncio\r\n\r\nBASE_URL = \"https://example.com/api\"\r\n\r\nserver = ShadowServer(\r\n target_base_url=BASE_URL,\r\n redirects=False # Disables redirection\r\n)\r\n\r\nasyncio.run(server.start_server(\r\n host=\"127.0.0.1\",\r\n port=3000\r\n))\r\n```\r\n\r\n---\r\n\r\n## Advanced Configuration\r\n\r\n### Setting a Custom Route\r\n\r\nYou can specify a custom route that will be appended to the base URL. This is useful when you want the server to be accessible via a specific route.\r\n\r\n```python\r\nproxy = ShadowServer(\r\n target_base_url=\"https://example.com\",\r\n route=\"/customroute\"\r\n)\r\n\r\nasyncio.run(proxy.start_server(host=\"127.0.0.1\", port=8080))\r\n```\r\n\r\n### Setting Timeout and Maximum Connections\r\n\r\nTo configure custom timeouts and connection limits during initialization:\r\n\r\n```python\r\nproxy = ShadowServer(target_base_url=\"https://example.com\", timeout=60, max_conn=200)\r\n```\r\n\r\nThis example sets a 60-second timeout and allows up to 200 concurrent connections.\r\n\r\n---\r\n\r\n## API Reference\r\n\r\n### ShadowServer\r\n\r\nThe main class for setting up and running the proxy server.\r\n\r\n```python\r\nclass ShadowServer:\r\n def __init__(self, target_base_url, timeout=30, max_conn=100)\r\n```\r\n\r\n- **Parameters**:\r\n - `target_base_url` (str): The base URL to which all proxied requests are forwarded.\r\n - `timeout` (int, optional): Timeout in seconds for requests to the target server. Default is `30`.\r\n - `max_conn` (int, optional): Maximum number of concurrent connections. Default is `100`.\r\n - `redirect_url` (str, optional): URL for redirecting requests from the base URL.\r\n - `redirects` (bool, optional): If `True`, enables redirection to `redirect_url`. Default is `False`.\r\n - `open_on_browser` (bool, optional): Automatically opens the server URL in a browser when started. Default is `True`.\r\n - `verify_ssl` (bool, optional): If `False`, disables SSL verification. Default is `True`.\r\n - `route` (str, optional): Appends a custom route to the server URL.\r\n\r\n#### Methods\r\n\r\n1. **`start_server`**\r\n\r\n ```python\r\n async def start_server(self, host='127.0.0.1', port=8080)\r\n ```\r\n\r\n Starts the proxy server.\r\n\r\n - **Parameters**:\r\n\r\n - `host` (str, optional): The host IP on which the server runs. Default is `'127.0.0.1'`.\r\n - `port` (int, optional): The port on which the server listens. Default is `8080`.\r\n\r\n2. **`close`**\r\n\r\n ```python\r\n async def close(self)\r\n ```\r\n\r\n Closes the server session and frees up resources.\r\n\r\n---\r\n\r\n## Request Handling\r\n\r\nThe `ShadowServer` proxy server processes requests as follows:\r\n\r\n1. **handle_request**: Forwards HTTP and HTTPS requests to the target server and returns the response to the client.\r\n2. **handle_websocket**: Forwards WebSocket connections to the target server.\r\n3. **build_response**: Builds the response, applies custom headers (such as CORS), and sends it to the client.\r\n\r\n### Example of Proxying a GET Request\r\n\r\nOnce the server is running, you can make a GET request to any endpoint available on the target server:\r\n\r\n```bash\r\ncurl http://127.0.0.1:8080/api/resource\r\n```\r\n\r\nThis request will be proxied to `https://example.com/api/resource`.\r\n\r\n### WebSocket Proxying\r\n\r\nThe proxy supports WebSocket connections. You can connect to the WebSocket server via the proxy as shown below:\r\n\r\n```python\r\nimport websockets\r\nimport asyncio\r\n\r\nasync def connect():\r\n uri = \"ws://127.0.0.1:8080/socket\"\r\n async with websockets.connect(uri) as websocket:\r\n await websocket.send(\"Hello, World!\")\r\n response = await websocket.recv()\r\n print(response)\r\n\r\nasyncio.run(connect())\r\n```\r\n\r\n---\r\n\r\n## Troubleshooting\r\n\r\n### CORS Errors\r\n\r\nIf you encounter CORS issues, ensure that the client request headers include the correct `Origin`.\r\n\r\n### SSL Verification Errors\r\n\r\nTo disable SSL verification, set `verify_ssl=False` when initializing the server.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An asynchronous HTTP proxy server library using aiohttp, designed to forward requests from clients to a target server",
"version": "0.2.4",
"project_urls": {
"Homepage": "https://github.com/benkimz/shadowserver"
},
"split_keywords": [
"proxy",
" server",
" proxy-server",
" http",
" https",
" aiohttp",
" asyncio"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0e452e835c3d7ae23b805285cdd14f25af82c219962e27ff78118590af624ad3",
"md5": "390743dfb3efcaecab8c09b2c2e6e873",
"sha256": "042c7aed1474269c7c926d17143b053764df27a2ec55161ee6649ae2c98068be"
},
"downloads": -1,
"filename": "shadowserver-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "390743dfb3efcaecab8c09b2c2e6e873",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7531,
"upload_time": "2024-11-08T09:53:46",
"upload_time_iso_8601": "2024-11-08T09:53:46.019392Z",
"url": "https://files.pythonhosted.org/packages/0e/45/2e835c3d7ae23b805285cdd14f25af82c219962e27ff78118590af624ad3/shadowserver-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "44c8d1b04a2793cf505a512d502991bccf1796102c36572ea6e1b64487809c11",
"md5": "3ac05e79bce9de36b1ea5b0891efaf6c",
"sha256": "6fabbb2ac6d35db6ddf34bdd788c60f7e0c67a933978079dac1aa4d04a1b1b6b"
},
"downloads": -1,
"filename": "shadowserver-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "3ac05e79bce9de36b1ea5b0891efaf6c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 7449,
"upload_time": "2024-11-08T09:53:48",
"upload_time_iso_8601": "2024-11-08T09:53:48.787541Z",
"url": "https://files.pythonhosted.org/packages/44/c8/d1b04a2793cf505a512d502991bccf1796102c36572ea6e1b64487809c11/shadowserver-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-08 09:53:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "benkimz",
"github_project": "shadowserver",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "shadowserver"
}
JSON
