| Name | httpchain JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | None |
| Summary | Declarative HTTP workflow engine for building complex API request chains with dependency management and data extraction |
| upload_time | 2025-11-03 15:17:53 |
| maintainer | None |
| docs_url | None |
| author | Jatin Banga |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
api
automation
http
osint
web-scraping
workflow
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# HttpChain
[](https://badge.fury.io/py/httpchain)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
**Declarative HTTP workflow engine** for building complex API request chains with dependency management and data extraction.
Perfect for **OSINT workflows**, **web scraping pipelines**, **API automation**, and **multi-step data collection** tasks.
## 🚀 Quick Start
```bash
pip install httpchain
```
```python
import asyncio
import json
from httpchain import HttpChainExecutor
# Define your workflow
workflow = {
"version": 1,
"name": "User Profile Checker",
"chain_variables": ["username"],
"steps": [
{
"name": "get_profile",
"request": {
"request_method": "GET",
"request_url": "https://api.github.com/users/{{username}}",
"extractors": [
{
"extractor_key": "user_id",
"extractor_type": "jsonpatharray",
"jsonpatharray_extractor": ["json_body", "id"]
},
{
"extractor_key": "followers",
"extractor_type": "jsonpatharray",
"jsonpatharray_extractor": ["json_body", "followers"]
}
]
}
}
]
}
async def main():
executor = HttpChainExecutor()
executor.load_json(workflow)
result = await executor.execute(username="octocat")
print(f"User ID: {result.variable_state.user_id}")
print(f"Followers: {result.variable_state.followers}")
asyncio.run(main())
```
## ✨ Key Features
- **🔗 Declarative Workflows**: Define complex HTTP request chains in JSON
- **📊 Automatic Dependency Resolution**: Steps execute when their dependencies are ready
- **🎯 Smart Data Extraction**: Extract data using JSONPath, Regex, or boolean checks
- **🔄 Variable Substitution**: Use extracted data in subsequent requests
- **⚡ Parallel Execution**: Independent steps run concurrently
- **🛡️ Built-in Retries**: Configurable retry logic and error handling
- **🎭 Header Randomization**: Randomize User-Agent and headers for stealth
- **📝 Conditional Logic**: Skip steps based on extracted data conditions
## 🎯 Perfect For
- **OSINT Investigations**: Multi-platform account lookups and data correlation
- **API Automation**: Complex workflows spanning multiple services
- **Web Scraping**: Extract and correlate data across multiple pages
- **Data Pipelines**: Transform and enrich data through API chains
- **Security Research**: Automated reconnaissance and data gathering
## 📖 Core Concepts
### Workflow Structure
```json
{
"version": 1,
"name": "My Workflow",
"chain_variables": ["input1", "input2"],
"steps": [...]
}
```
### Step Dependencies
Steps wait for required variables before executing:
```json
{
"name": "step2",
"depends_on_variables": ["token_from_step1"],
"request": {
"request_headers": {"Authorization": "Bearer {{token_from_step1}}"}
}
}
```
### Data Extraction
Extract data from responses using multiple methods:
```json
{
"extractors": [
{
"extractor_key": "user_id",
"extractor_type": "jsonpatharray",
"jsonpatharray_extractor": ["json_body", "data", "id"]
},
{
"extractor_key": "csrf_token",
"extractor_type": "regex",
"regex_extractor": {
"path": ["html_body"],
"pattern": "csrf_token=([a-f0-9]+)"
}
}
]
}
```
### Conditional Execution
Run steps only when conditions are met:
```json
{
"condition": {
"operator": "and",
"checks": [
{
"variable_name": "user_exists",
"operator": "equals",
"value": true
}
]
}
}
```
## 🛠️ Advanced Features
### Parallel Execution
Steps with no shared dependencies run in parallel automatically.
### Error Handling
Built-in retries, timeouts, and graceful failure handling.
### Header Randomization
```json
{
"randomize_headers": true
}
```
### Multiple Extraction Types
- **JSONPath**: Navigate JSON responses
- **Regex**: Extract from text/HTML with patterns
- **Declarative**: Boolean checks and conditions
## 📚 Examples
Check out the `/examples` directory for complete workflows:
- **X.com Account Checker**: Multi-step authentication and profile extraction
- **Multi-Platform OSINT**: Parallel account verification across platforms
- **Domain RDAP Lookup**: Fallback chains for domain information
- **Instagram Profile Data**: Complex data extraction workflows
## 🔧 Installation & Setup
```bash
# Basic installation
pip install httpchain
# With examples dependencies
pip install httpchain[examples]
# Development setup (from source)
git clone https://github.com/jatin-dot-py/httpchain.git
cd httpchain
pip install -e ".[examples]"
```
## 📋 Requirements
- Python 3.10+
- httpx
- beautifulsoup4
- jsonpath-ng
## 🤝 Contributing
Contributions welcome! Please read our contributing guidelines and submit pull requests.
## 📄 License
MIT License - see [LICENSE](LICENSE) file for details.
## 🔗 Links
- **Documentation**: [Full API Reference](https://github.com/jatin-dot-py/httpchain#readme)
- **Examples**: [Example Workflows](/examples)
- **Issues**: [Bug Reports & Feature Requests](https://github.com/jatin-dot-py/httpchain/issues)
---
**Built for developers who need to orchestrate complex HTTP workflows with precision and reliability.**
Raw data
{
"_id": null,
"home_page": null,
"name": "httpchain",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "api, automation, http, osint, web-scraping, workflow",
"author": "Jatin Banga",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4f/24/33ad4cbe1614cd444b92bc1da1c50ac60bbfb024491dc1d1a3c616b12dab/httpchain-1.0.1.tar.gz",
"platform": null,
"description": "# HttpChain\n\n[](https://badge.fury.io/py/httpchain)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n\n**Declarative HTTP workflow engine** for building complex API request chains with dependency management and data extraction.\n\nPerfect for **OSINT workflows**, **web scraping pipelines**, **API automation**, and **multi-step data collection** tasks.\n\n## \ud83d\ude80 Quick Start\n\n```bash\npip install httpchain\n```\n\n```python\nimport asyncio\nimport json\nfrom httpchain import HttpChainExecutor\n\n# Define your workflow\nworkflow = {\n \"version\": 1,\n \"name\": \"User Profile Checker\",\n \"chain_variables\": [\"username\"],\n \"steps\": [\n {\n \"name\": \"get_profile\",\n \"request\": {\n \"request_method\": \"GET\",\n \"request_url\": \"https://api.github.com/users/{{username}}\",\n \"extractors\": [\n {\n \"extractor_key\": \"user_id\",\n \"extractor_type\": \"jsonpatharray\",\n \"jsonpatharray_extractor\": [\"json_body\", \"id\"]\n },\n {\n \"extractor_key\": \"followers\",\n \"extractor_type\": \"jsonpatharray\", \n \"jsonpatharray_extractor\": [\"json_body\", \"followers\"]\n }\n ]\n }\n }\n ]\n}\n\nasync def main():\n executor = HttpChainExecutor()\n executor.load_json(workflow)\n result = await executor.execute(username=\"octocat\")\n \n print(f\"User ID: {result.variable_state.user_id}\")\n print(f\"Followers: {result.variable_state.followers}\")\n\nasyncio.run(main())\n```\n\n## \u2728 Key Features\n\n- **\ud83d\udd17 Declarative Workflows**: Define complex HTTP request chains in JSON\n- **\ud83d\udcca Automatic Dependency Resolution**: Steps execute when their dependencies are ready\n- **\ud83c\udfaf Smart Data Extraction**: Extract data using JSONPath, Regex, or boolean checks\n- **\ud83d\udd04 Variable Substitution**: Use extracted data in subsequent requests\n- **\u26a1 Parallel Execution**: Independent steps run concurrently\n- **\ud83d\udee1\ufe0f Built-in Retries**: Configurable retry logic and error handling\n- **\ud83c\udfad Header Randomization**: Randomize User-Agent and headers for stealth\n- **\ud83d\udcdd Conditional Logic**: Skip steps based on extracted data conditions\n\n## \ud83c\udfaf Perfect For\n\n- **OSINT Investigations**: Multi-platform account lookups and data correlation\n- **API Automation**: Complex workflows spanning multiple services \n- **Web Scraping**: Extract and correlate data across multiple pages\n- **Data Pipelines**: Transform and enrich data through API chains\n- **Security Research**: Automated reconnaissance and data gathering\n\n## \ud83d\udcd6 Core Concepts\n\n### Workflow Structure\n\n```json\n{\n \"version\": 1,\n \"name\": \"My Workflow\",\n \"chain_variables\": [\"input1\", \"input2\"],\n \"steps\": [...]\n}\n```\n\n### Step Dependencies\n\nSteps wait for required variables before executing:\n\n```json\n{\n \"name\": \"step2\",\n \"depends_on_variables\": [\"token_from_step1\"],\n \"request\": {\n \"request_headers\": {\"Authorization\": \"Bearer {{token_from_step1}}\"}\n }\n}\n```\n\n### Data Extraction\n\nExtract data from responses using multiple methods:\n\n```json\n{\n \"extractors\": [\n {\n \"extractor_key\": \"user_id\",\n \"extractor_type\": \"jsonpatharray\", \n \"jsonpatharray_extractor\": [\"json_body\", \"data\", \"id\"]\n },\n {\n \"extractor_key\": \"csrf_token\",\n \"extractor_type\": \"regex\",\n \"regex_extractor\": {\n \"path\": [\"html_body\"],\n \"pattern\": \"csrf_token=([a-f0-9]+)\"\n }\n }\n ]\n}\n```\n\n### Conditional Execution\n\nRun steps only when conditions are met:\n\n```json\n{\n \"condition\": {\n \"operator\": \"and\",\n \"checks\": [\n {\n \"variable_name\": \"user_exists\",\n \"operator\": \"equals\",\n \"value\": true\n }\n ]\n }\n}\n```\n\n## \ud83d\udee0\ufe0f Advanced Features\n\n### Parallel Execution\nSteps with no shared dependencies run in parallel automatically.\n\n### Error Handling\nBuilt-in retries, timeouts, and graceful failure handling.\n\n### Header Randomization\n```json\n{\n \"randomize_headers\": true\n}\n```\n\n### Multiple Extraction Types\n- **JSONPath**: Navigate JSON responses\n- **Regex**: Extract from text/HTML with patterns\n- **Declarative**: Boolean checks and conditions\n\n## \ud83d\udcda Examples\n\nCheck out the `/examples` directory for complete workflows:\n\n- **X.com Account Checker**: Multi-step authentication and profile extraction\n- **Multi-Platform OSINT**: Parallel account verification across platforms \n- **Domain RDAP Lookup**: Fallback chains for domain information\n- **Instagram Profile Data**: Complex data extraction workflows\n\n## \ud83d\udd27 Installation & Setup\n\n```bash\n# Basic installation\npip install httpchain\n\n# With examples dependencies\npip install httpchain[examples]\n\n# Development setup (from source)\ngit clone https://github.com/jatin-dot-py/httpchain.git\ncd httpchain\npip install -e \".[examples]\"\n```\n\n## \ud83d\udccb Requirements\n\n- Python 3.10+\n- httpx\n- beautifulsoup4\n- jsonpath-ng\n\n## \ud83e\udd1d Contributing\n\nContributions welcome! Please read our contributing guidelines and submit pull requests.\n\n## \ud83d\udcc4 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n## \ud83d\udd17 Links\n\n- **Documentation**: [Full API Reference](https://github.com/jatin-dot-py/httpchain#readme)\n- **Examples**: [Example Workflows](/examples)\n- **Issues**: [Bug Reports & Feature Requests](https://github.com/jatin-dot-py/httpchain/issues)\n\n---\n\n**Built for developers who need to orchestrate complex HTTP workflows with precision and reliability.**",
"bugtrack_url": null,
"license": "MIT",
"summary": "Declarative HTTP workflow engine for building complex API request chains with dependency management and data extraction",
"version": "1.0.1",
"project_urls": {
"Documentation": "https://github.com/jatin-dot-py/httpchain#readme",
"Homepage": "https://github.com/jatin-dot-py/httpchain",
"Issues": "https://github.com/jatin-dot-py/httpchain/issues",
"Repository": "https://github.com/jatin-dot-py/httpchain"
},
"split_keywords": [
"api",
" automation",
" http",
" osint",
" web-scraping",
" workflow"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "231579126a73136521a0b494ca9730f9b2bdb3e5f1ed643c46151555bae7bb3f",
"md5": "8beb215be46a898b870872ba45abe6df",
"sha256": "3f54430f86c006e89b9a8038fb6ca2c6199ed34484601bfae2b7087c24aac2e3"
},
"downloads": -1,
"filename": "httpchain-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8beb215be46a898b870872ba45abe6df",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 23523,
"upload_time": "2025-11-03T15:17:51",
"upload_time_iso_8601": "2025-11-03T15:17:51.351335Z",
"url": "https://files.pythonhosted.org/packages/23/15/79126a73136521a0b494ca9730f9b2bdb3e5f1ed643c46151555bae7bb3f/httpchain-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4f2433ad4cbe1614cd444b92bc1da1c50ac60bbfb024491dc1d1a3c616b12dab",
"md5": "648df9a927c132adf806b827002b13c9",
"sha256": "072988213474a6db06cbb506ea0ce55de26549d932b377665dbc3a3225785276"
},
"downloads": -1,
"filename": "httpchain-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "648df9a927c132adf806b827002b13c9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 46643,
"upload_time": "2025-11-03T15:17:53",
"upload_time_iso_8601": "2025-11-03T15:17:53.253441Z",
"url": "https://files.pythonhosted.org/packages/4f/24/33ad4cbe1614cd444b92bc1da1c50ac60bbfb024491dc1d1a3c616b12dab/httpchain-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-11-03 15:17:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jatin-dot-py",
"github_project": "httpchain#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "httpchain"
}
