curlwright


Namecurlwright JSON
Version 2.0.0 PyPI version JSON
download
home_pagehttps://github.com/seifreed/Curlwright
SummaryCloudflare bypass tool using Playwright for curl requests
upload_time2025-09-06 11:49:27
maintainerNone
docs_urlNone
authorMarc Rivero
requires_python>=3.9
licenseMIT
keywords cloudflare bypass playwright curl web-scraping automation browser-automation turnstile
VCS
bugtrack_url
requirements playwright asyncio
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # CurlWright

[![Python Version](https://img.shields.io/pypi/pyversions/curlwright)](https://pypi.org/project/curlwright/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![PyPI version](https://img.shields.io/pypi/v/curlwright)](https://pypi.org/project/curlwright/)

CurlWright is a Cloudflare bypass tool that leverages Playwright to execute curl commands with full browser capabilities, allowing you to access protected websites seamlessly.

## Features

- ✅ **Automatic Cloudflare Bypass** - Handles Cloudflare challenges automatically
- ✅ **Full Curl Support** - Parse and execute complex curl commands
- ✅ **Turnstile Support** - Handles Cloudflare Turnstile challenges
- ✅ **Cookie Management** - Persistent cookie storage and session management
- ✅ **Modular Architecture** - Clean, maintainable, and extensible codebase
- ✅ **Retry Mechanism** - Automatic retries with configurable delays
- ✅ **Headless & Visual Mode** - Run in background or watch the browser
- ✅ **Server Mode** - Run on servers without X11/GUI requirements

## Installation

### From PyPI

```bash
pip install curlwright
```

### From Source

```bash
# Clone the repository
git clone https://github.com/seifreed/Curlwright.git
cd Curlwright

# Install in development mode
pip install -e .

# Install Playwright browsers
playwright install chromium
```

## Quick Start

### As a Command Line Tool

```bash
# Direct curl command
curlwright -c "curl https://example.com"

# From file
curlwright -f request.txt

# With custom options
curlwright -c "curl https://example.com" --headless --timeout 60 --retries 5
```

### As a Python Library

```python
import asyncio
from curlwright import RequestExecutor

async def main():
    executor = RequestExecutor(headless=True)
    
    curl_command = 'curl -H "User-Agent: Custom" https://example.com'
    result = await executor.execute(curl_command)
    
    print(f"Status: {result['status']}")
    print(f"Body: {result['body']}")
    
    await executor.close()

asyncio.run(main())
```

## Command Line Options

```
usage: curlwright [-h] (-c CURL | -f FILE) [--headless] [--no-gui] [--user-agent USER_AGENT]
                  [--timeout TIMEOUT] [--retries RETRIES] [--delay DELAY]
                  [-o OUTPUT] [-v]

Bypass Cloudflare protection using Playwright and execute curl commands

Required Arguments:
  -c CURL, --curl CURL        Curl command to execute
  -f FILE, --file FILE        File containing curl command

Browser Options:
  --headless                   Run browser in headless mode
  --no-gui                     Run in server mode without X11/display requirement (implies --headless)
  --user-agent USER_AGENT      Custom user agent string
  --timeout TIMEOUT            Request timeout in seconds (default: 30)

Retry Options:
  --retries RETRIES           Number of retries on failure (default: 3)
  --delay DELAY               Delay between retries in seconds (default: 5)

Output Options:
  -o OUTPUT, --output OUTPUT   Save response to file
  -v, --verbose               Verbose output
```

## Examples

### Simple GET Request

```bash
curlwright -c "curl https://httpbin.org/get"
```

### POST Request with JSON Data

```bash
curlwright -c 'curl -X POST -H "Content-Type: application/json" -d "{\"key\":\"value\"}" https://httpbin.org/post'
```

### Request with Headers and Authentication

```bash
curlwright -c 'curl -H "Authorization: Bearer TOKEN" -H "Accept: application/json" https://api.example.com/data'
```

### Using a Request File

Create a file `request.txt`:
```
curl -X GET \
  -H "User-Agent: MyApp/1.0" \
  -H "Accept: application/json" \
  -b "session=abc123" \
  https://protected.example.com/api/data
```

Then execute:
```bash
curlwright -f request.txt -o response.json
```

### Server Mode (No GUI/X11)

For running on servers without display support:

```bash
# Run on a VPS or container without X11
curlwright -c "curl https://api.example.com/data" --no-gui

# Process API requests on a headless server
curlwright -f api_request.txt --no-gui -o result.json

# Use in CI/CD pipelines
curlwright -c "curl https://protected-site.com" --no-gui --timeout 60
```

The `--no-gui` flag is optimized for server environments and includes:
- No X11/display requirement
- Reduced memory footprint
- Disabled GPU acceleration
- Optimized for containerized environments (Docker, Kubernetes)
- Suitable for VPS, cloud servers, and CI/CD pipelines

## Python API

### Basic Usage

```python
from curlwright import RequestExecutor

executor = RequestExecutor(headless=True, timeout=30)
result = await executor.execute('curl https://example.com')
```

### Advanced Usage with Cookie Management

```python
from curlwright import RequestExecutor
from curlwright.utils import CookieManager

# Initialize with cookie persistence
cookie_manager = CookieManager('cookies.pkl')
executor = RequestExecutor(headless=False)

# Execute request
result = await executor.execute('curl https://example.com')

# Save cookies for next session
await cookie_manager.save_cookies(executor.browser_manager.context)
```

### Parsing Curl Commands

```python
from curlwright.parsers import CurlParser

parser = CurlParser()
request = parser.parse('curl -X POST -H "Content-Type: application/json" https://api.example.com')

print(f"Method: {request.method}")
print(f"URL: {request.url}")
print(f"Headers: {request.headers}")
```

## Project Structure

```
curlwright/
├── curlwright.py          # CLI entry point
├── requirements.txt        # Project dependencies
├── setup.py               # Package configuration
├── LICENSE                # MIT License
├── README.md              # Documentation
├── pyproject.toml         # Modern Python packaging
└── src/
    ├── __init__.py        # Package initialization
    ├── cli.py             # Command line interface
    ├── core/
    │   ├── __init__.py
    │   ├── browser_manager.py    # Playwright browser management
    │   └── request_executor.py   # Request execution logic
    ├── parsers/
    │   ├── __init__.py
    │   └── curl_parser.py        # Curl command parser
    └── utils/
        ├── __init__.py
        ├── logger.py              # Logging configuration
        └── cookie_manager.py      # Cookie management
```

## Requirements

- Python 3.13+
- Playwright
- Modern browser (Chromium)

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Author

**Marc Rivero** | [mriverolopez@gmail.com](mailto:mriverolopez@gmail.com)

GitHub: [https://github.com/seifreed/Curlwright](https://github.com/seifreed/Curlwright)

## Disclaimer

This tool is for educational and testing purposes only. Always respect website terms of service and use responsibly. The authors are not responsible for any misuse or damage caused by this tool.

## Support

If you encounter any issues or have questions, please [open an issue](https://github.com/seifreed/Curlwright/issues) on GitHub.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/seifreed/Curlwright",
    "name": "curlwright",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": null,
    "keywords": "cloudflare, bypass, playwright, curl, web-scraping, automation, browser-automation, turnstile",
    "author": "Marc Rivero",
    "author_email": "Marc Rivero <mriverolopez@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/bc/d6/829387e59dab8fc881cd4e184ac3f6ca867ce89a3d3f59ab1348f8f030b4/curlwright-2.0.0.tar.gz",
    "platform": null,
    "description": "# CurlWright\n\n[![Python Version](https://img.shields.io/pypi/pyversions/curlwright)](https://pypi.org/project/curlwright/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![PyPI version](https://img.shields.io/pypi/v/curlwright)](https://pypi.org/project/curlwright/)\n\nCurlWright is a Cloudflare bypass tool that leverages Playwright to execute curl commands with full browser capabilities, allowing you to access protected websites seamlessly.\n\n## Features\n\n- \u2705 **Automatic Cloudflare Bypass** - Handles Cloudflare challenges automatically\n- \u2705 **Full Curl Support** - Parse and execute complex curl commands\n- \u2705 **Turnstile Support** - Handles Cloudflare Turnstile challenges\n- \u2705 **Cookie Management** - Persistent cookie storage and session management\n- \u2705 **Modular Architecture** - Clean, maintainable, and extensible codebase\n- \u2705 **Retry Mechanism** - Automatic retries with configurable delays\n- \u2705 **Headless & Visual Mode** - Run in background or watch the browser\n- \u2705 **Server Mode** - Run on servers without X11/GUI requirements\n\n## Installation\n\n### From PyPI\n\n```bash\npip install curlwright\n```\n\n### From Source\n\n```bash\n# Clone the repository\ngit clone https://github.com/seifreed/Curlwright.git\ncd Curlwright\n\n# Install in development mode\npip install -e .\n\n# Install Playwright browsers\nplaywright install chromium\n```\n\n## Quick Start\n\n### As a Command Line Tool\n\n```bash\n# Direct curl command\ncurlwright -c \"curl https://example.com\"\n\n# From file\ncurlwright -f request.txt\n\n# With custom options\ncurlwright -c \"curl https://example.com\" --headless --timeout 60 --retries 5\n```\n\n### As a Python Library\n\n```python\nimport asyncio\nfrom curlwright import RequestExecutor\n\nasync def main():\n    executor = RequestExecutor(headless=True)\n    \n    curl_command = 'curl -H \"User-Agent: Custom\" https://example.com'\n    result = await executor.execute(curl_command)\n    \n    print(f\"Status: {result['status']}\")\n    print(f\"Body: {result['body']}\")\n    \n    await executor.close()\n\nasyncio.run(main())\n```\n\n## Command Line Options\n\n```\nusage: curlwright [-h] (-c CURL | -f FILE) [--headless] [--no-gui] [--user-agent USER_AGENT]\n                  [--timeout TIMEOUT] [--retries RETRIES] [--delay DELAY]\n                  [-o OUTPUT] [-v]\n\nBypass Cloudflare protection using Playwright and execute curl commands\n\nRequired Arguments:\n  -c CURL, --curl CURL        Curl command to execute\n  -f FILE, --file FILE        File containing curl command\n\nBrowser Options:\n  --headless                   Run browser in headless mode\n  --no-gui                     Run in server mode without X11/display requirement (implies --headless)\n  --user-agent USER_AGENT      Custom user agent string\n  --timeout TIMEOUT            Request timeout in seconds (default: 30)\n\nRetry Options:\n  --retries RETRIES           Number of retries on failure (default: 3)\n  --delay DELAY               Delay between retries in seconds (default: 5)\n\nOutput Options:\n  -o OUTPUT, --output OUTPUT   Save response to file\n  -v, --verbose               Verbose output\n```\n\n## Examples\n\n### Simple GET Request\n\n```bash\ncurlwright -c \"curl https://httpbin.org/get\"\n```\n\n### POST Request with JSON Data\n\n```bash\ncurlwright -c 'curl -X POST -H \"Content-Type: application/json\" -d \"{\\\"key\\\":\\\"value\\\"}\" https://httpbin.org/post'\n```\n\n### Request with Headers and Authentication\n\n```bash\ncurlwright -c 'curl -H \"Authorization: Bearer TOKEN\" -H \"Accept: application/json\" https://api.example.com/data'\n```\n\n### Using a Request File\n\nCreate a file `request.txt`:\n```\ncurl -X GET \\\n  -H \"User-Agent: MyApp/1.0\" \\\n  -H \"Accept: application/json\" \\\n  -b \"session=abc123\" \\\n  https://protected.example.com/api/data\n```\n\nThen execute:\n```bash\ncurlwright -f request.txt -o response.json\n```\n\n### Server Mode (No GUI/X11)\n\nFor running on servers without display support:\n\n```bash\n# Run on a VPS or container without X11\ncurlwright -c \"curl https://api.example.com/data\" --no-gui\n\n# Process API requests on a headless server\ncurlwright -f api_request.txt --no-gui -o result.json\n\n# Use in CI/CD pipelines\ncurlwright -c \"curl https://protected-site.com\" --no-gui --timeout 60\n```\n\nThe `--no-gui` flag is optimized for server environments and includes:\n- No X11/display requirement\n- Reduced memory footprint\n- Disabled GPU acceleration\n- Optimized for containerized environments (Docker, Kubernetes)\n- Suitable for VPS, cloud servers, and CI/CD pipelines\n\n## Python API\n\n### Basic Usage\n\n```python\nfrom curlwright import RequestExecutor\n\nexecutor = RequestExecutor(headless=True, timeout=30)\nresult = await executor.execute('curl https://example.com')\n```\n\n### Advanced Usage with Cookie Management\n\n```python\nfrom curlwright import RequestExecutor\nfrom curlwright.utils import CookieManager\n\n# Initialize with cookie persistence\ncookie_manager = CookieManager('cookies.pkl')\nexecutor = RequestExecutor(headless=False)\n\n# Execute request\nresult = await executor.execute('curl https://example.com')\n\n# Save cookies for next session\nawait cookie_manager.save_cookies(executor.browser_manager.context)\n```\n\n### Parsing Curl Commands\n\n```python\nfrom curlwright.parsers import CurlParser\n\nparser = CurlParser()\nrequest = parser.parse('curl -X POST -H \"Content-Type: application/json\" https://api.example.com')\n\nprint(f\"Method: {request.method}\")\nprint(f\"URL: {request.url}\")\nprint(f\"Headers: {request.headers}\")\n```\n\n## Project Structure\n\n```\ncurlwright/\n\u251c\u2500\u2500 curlwright.py          # CLI entry point\n\u251c\u2500\u2500 requirements.txt        # Project dependencies\n\u251c\u2500\u2500 setup.py               # Package configuration\n\u251c\u2500\u2500 LICENSE                # MIT License\n\u251c\u2500\u2500 README.md              # Documentation\n\u251c\u2500\u2500 pyproject.toml         # Modern Python packaging\n\u2514\u2500\u2500 src/\n    \u251c\u2500\u2500 __init__.py        # Package initialization\n    \u251c\u2500\u2500 cli.py             # Command line interface\n    \u251c\u2500\u2500 core/\n    \u2502   \u251c\u2500\u2500 __init__.py\n    \u2502   \u251c\u2500\u2500 browser_manager.py    # Playwright browser management\n    \u2502   \u2514\u2500\u2500 request_executor.py   # Request execution logic\n    \u251c\u2500\u2500 parsers/\n    \u2502   \u251c\u2500\u2500 __init__.py\n    \u2502   \u2514\u2500\u2500 curl_parser.py        # Curl command parser\n    \u2514\u2500\u2500 utils/\n        \u251c\u2500\u2500 __init__.py\n        \u251c\u2500\u2500 logger.py              # Logging configuration\n        \u2514\u2500\u2500 cookie_manager.py      # Cookie management\n```\n\n## Requirements\n\n- Python 3.13+\n- Playwright\n- Modern browser (Chromium)\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Author\n\n**Marc Rivero** | [mriverolopez@gmail.com](mailto:mriverolopez@gmail.com)\n\nGitHub: [https://github.com/seifreed/Curlwright](https://github.com/seifreed/Curlwright)\n\n## Disclaimer\n\nThis tool is for educational and testing purposes only. Always respect website terms of service and use responsibly. The authors are not responsible for any misuse or damage caused by this tool.\n\n## Support\n\nIf you encounter any issues or have questions, please [open an issue](https://github.com/seifreed/Curlwright/issues) on GitHub.\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Cloudflare bypass tool using Playwright for curl requests",
    "version": "2.0.0",
    "project_urls": {
        "Documentation": "https://github.com/seifreed/Curlwright#readme",
        "Homepage": "https://github.com/seifreed/Curlwright",
        "Issues": "https://github.com/seifreed/Curlwright/issues",
        "Repository": "https://github.com/seifreed/Curlwright"
    },
    "split_keywords": [
        "cloudflare",
        " bypass",
        " playwright",
        " curl",
        " web-scraping",
        " automation",
        " browser-automation",
        " turnstile"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "2268318e6f15e6e0ca5000fd6199636f64cdfba70e607d9e11b56bb614cff9ad",
                "md5": "21030a696fdb65fde658ed6b59368ea8",
                "sha256": "9e35fda756124a1b25400cf7035a90174bd5e34112fa4c567f93b21ed50ab359"
            },
            "downloads": -1,
            "filename": "curlwright-2.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "21030a696fdb65fde658ed6b59368ea8",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 18748,
            "upload_time": "2025-09-06T11:49:26",
            "upload_time_iso_8601": "2025-09-06T11:49:26.245262Z",
            "url": "https://files.pythonhosted.org/packages/22/68/318e6f15e6e0ca5000fd6199636f64cdfba70e607d9e11b56bb614cff9ad/curlwright-2.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "bcd6829387e59dab8fc881cd4e184ac3f6ca867ce89a3d3f59ab1348f8f030b4",
                "md5": "4a73856be3649303bed42bec4e6fd8f0",
                "sha256": "e150e45fea90931597521140fde16edbf5e9e0ceb5c55a2fd3e52f74d37fc694"
            },
            "downloads": -1,
            "filename": "curlwright-2.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "4a73856be3649303bed42bec4e6fd8f0",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 17177,
            "upload_time": "2025-09-06T11:49:27",
            "upload_time_iso_8601": "2025-09-06T11:49:27.493159Z",
            "url": "https://files.pythonhosted.org/packages/bc/d6/829387e59dab8fc881cd4e184ac3f6ca867ce89a3d3f59ab1348f8f030b4/curlwright-2.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-09-06 11:49:27",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "seifreed",
    "github_project": "Curlwright",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [
        {
            "name": "playwright",
            "specs": [
                [
                    ">=",
                    "1.40.0"
                ]
            ]
        },
        {
            "name": "asyncio",
            "specs": [
                [
                    ">=",
                    "3.4.3"
                ]
            ]
        }
    ],
    "lcname": "curlwright"
}
        
Elapsed time: 1.46727s