| Name | groove JSON |
| Version |
1.0.4
JSON |
| download |
| home_page | |
| Summary | |
| upload_time | 2022-12-24 18:48:18 |
| maintainer | |
| docs_url | None |
| author | Pierce Freeman |
| requires_python | >=3.10,<4.0 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Groove
Python APIs for Groove, a proxy server built for web crawling and unit test mocking. Highlights of its primary features:
- HTTP and HTTPs support over HTTP/1 and HTTP/2.
- Local CA certificate generation and installation on Mac and Linux to support system curl and Chromium.
- Different tiers of caching support - from disabling completely to aggressively maintaining all body archives.
- Limit outbound requests of the same URL to 1 concurrent request to save on bandwidth if requests are already inflight.
- Record and replay requests made to outgoing servers. Recreate testing flows in unit tests while separating them from crawling business logic.
- 3rd party proxy support for commercial proxies.
- Custom TLS Hello Client support to maintain a Chromium-like TLS handshake while intercepting requests and re-forwarding on packets.
For more information, see the [Github](https://github.com/piercefreeman/grooveproxy) project.
## Usage
Add groove to your project and install the local certificates that allow for https certificate generation:
```
pip install groove
install-ca
```
Instantiating Groove with the default parameters is usually fine for most deployments. To ensure we clean up resources once you're completed with the proxy, wrap your code in the `launch` contextmanager.
```python
from groove.proxy import Groove
from requests import get
from pathlib import Path
proxy = Groove()
with proxy.launch():
response = get(
"https://www.example.com",
proxies={
"http": proxy.base_url_proxy,
"https": proxy.base_url_proxy,
},
verify=str(Path("~/.grooveproxy/ca.crt").expanduser()),
)
assert response.status_code == 200
```
Create a fully fake outbound for testing:
```python
from groove.proxy import Groove
from groove.tape import TapeRecord, TapeRequest, TapeResponse, TapeSession
from requests import get
from pathlib import Path
records = [
TapeRecord(
request=TapeRequest(
url="https://example.com:443/",
method="GET",
headers={},
body=b"",
),
response=TapeResponse(
status=200,
headers={},
body=b64encode("Test response".encode())
),
)
]
proxy = Groove()
with proxy.launch():
proxy.tape_load(
TapeSession(
records=records
)
)
response = get(
"https://www.example.com",
proxies={
"http": proxy.base_url_proxy,
"https": proxy.base_url_proxy,
},
verify=str(Path("~/.grooveproxy/ca.crt").expanduser())
)
assert response.content == b"Test response"
```
Raw data
{
"_id": null,
"home_page": "",
"name": "groove",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Pierce Freeman",
"author_email": "pierce@freeman.vc",
"download_url": "https://files.pythonhosted.org/packages/5b/a6/ced84bed5a639a893167ea9041535350dc0c108251ae0c171adca8a0f061/groove-1.0.4.tar.gz",
"platform": null,
"description": "# Groove\n\nPython APIs for Groove, a proxy server built for web crawling and unit test mocking. Highlights of its primary features:\n\n- HTTP and HTTPs support over HTTP/1 and HTTP/2.\n- Local CA certificate generation and installation on Mac and Linux to support system curl and Chromium.\n- Different tiers of caching support - from disabling completely to aggressively maintaining all body archives.\n- Limit outbound requests of the same URL to 1 concurrent request to save on bandwidth if requests are already inflight.\n- Record and replay requests made to outgoing servers. Recreate testing flows in unit tests while separating them from crawling business logic.\n- 3rd party proxy support for commercial proxies.\n- Custom TLS Hello Client support to maintain a Chromium-like TLS handshake while intercepting requests and re-forwarding on packets.\n\nFor more information, see the [Github](https://github.com/piercefreeman/grooveproxy) project.\n\n## Usage\n\nAdd groove to your project and install the local certificates that allow for https certificate generation:\n\n```\npip install groove\ninstall-ca\n```\n\nInstantiating Groove with the default parameters is usually fine for most deployments. To ensure we clean up resources once you're completed with the proxy, wrap your code in the `launch` contextmanager.\n\n```python\nfrom groove.proxy import Groove\nfrom requests import get\nfrom pathlib import Path\n\nproxy = Groove()\nwith proxy.launch():\n response = get(\n \"https://www.example.com\",\n proxies={\n \"http\": proxy.base_url_proxy,\n \"https\": proxy.base_url_proxy,\n },\n verify=str(Path(\"~/.grooveproxy/ca.crt\").expanduser()),\n )\n assert response.status_code == 200\n```\n\nCreate a fully fake outbound for testing:\n\n```python\nfrom groove.proxy import Groove\nfrom groove.tape import TapeRecord, TapeRequest, TapeResponse, TapeSession\nfrom requests import get\nfrom pathlib import Path\n\nrecords = [\n TapeRecord(\n request=TapeRequest(\n url=\"https://example.com:443/\",\n method=\"GET\",\n headers={},\n body=b\"\",\n ),\n response=TapeResponse(\n status=200,\n headers={},\n body=b64encode(\"Test response\".encode())\n ),\n )\n]\n\nproxy = Groove()\nwith proxy.launch():\n proxy.tape_load(\n TapeSession(\n records=records\n )\n )\n\n response = get(\n \"https://www.example.com\",\n proxies={\n \"http\": proxy.base_url_proxy,\n \"https\": proxy.base_url_proxy,\n },\n verify=str(Path(\"~/.grooveproxy/ca.crt\").expanduser())\n )\n assert response.content == b\"Test response\"\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "1.0.4",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "395e74e7dc6b054d65175d0625ba4e98",
"sha256": "e89696e8ccb6e683758b5f6dee99936ad94fb2c1c48b5aac33e13f2ded37f7ca"
},
"downloads": -1,
"filename": "groove-1.0.4-cp310-cp310-macosx_11_0_x86_64.whl",
"has_sig": false,
"md5_digest": "395e74e7dc6b054d65175d0625ba4e98",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10,<4.0",
"size": 14506,
"upload_time": "2022-12-24T18:48:16",
"upload_time_iso_8601": "2022-12-24T18:48:16.124640Z",
"url": "https://files.pythonhosted.org/packages/c0/01/25dbdb9f3b29f0eeb36384b2b212e8bb6075dd7d8c5a9a8301f5031f2ed5/groove-1.0.4-cp310-cp310-macosx_11_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "4dd4c7150a829926b00908190365d7da",
"sha256": "17cdd7fe8f0ab93a89f28e1e81316f8e28b2d83889b4d6d39d087e1e2f46f51c"
},
"downloads": -1,
"filename": "groove-1.0.4-cp310-cp310-manylinux_2_31_x86_64.whl",
"has_sig": false,
"md5_digest": "4dd4c7150a829926b00908190365d7da",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10,<4.0",
"size": 14510,
"upload_time": "2022-12-24T18:48:17",
"upload_time_iso_8601": "2022-12-24T18:48:17.328270Z",
"url": "https://files.pythonhosted.org/packages/31/c5/d4bf3ffa410935e6641780b1ac52c5e7a621569b54ca06fb1a71b2a7ac7d/groove-1.0.4-cp310-cp310-manylinux_2_31_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "59ceeb5054bc8725f2d21c2fa2e8b277",
"sha256": "7f722df5098d27473ada8473dc8166b6a45545f71e211d251aaba3357e994c3b"
},
"downloads": -1,
"filename": "groove-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "59ceeb5054bc8725f2d21c2fa2e8b277",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 41956,
"upload_time": "2022-12-24T18:48:18",
"upload_time_iso_8601": "2022-12-24T18:48:18.335685Z",
"url": "https://files.pythonhosted.org/packages/5b/a6/ced84bed5a639a893167ea9041535350dc0c108251ae0c171adca8a0f061/groove-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-24 18:48:18",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "groove"
}
