# 🔀 Unparallel
<!-- --8<-- [start:index] -->
<div align="center" markdown="1">
Create async HTTP requests with Python in no time.
[](https://github.com/RafaelWO/unparallel/actions?query=workflow%3Atest)
<br>
[](https://pypi.org/project/unparallel/)
[](https://pypi.org/project/unparallel/)
<br>
[](https://github.com/psf/black)
[](https://github.com/PyCQA/bandit)
[](https://github.com/RafaelWO/unparallel/blob/main/.pre-commit-config.yaml)
[](https://github.com/RafaelWO/unparallel/blob/main/LICENSE)
<br>
[](https://rafaelwo.github.io/unparallel/)
[](https://squidfunk.github.io/mkdocs-material/)
</div>
With Unparallel you can easily create thousands of web requests in an efficient way leveraging Python's async capabilities.
Unparallel is built on top of [HTTPX](https://github.com/encode/httpx/) and aims to support its rich set of features.
## Installation
```bash
pip install unparallel
```
## Example
A simple example of doing several GET requests to an HTTP web service:
```python
import asyncio
from unparallel import up
async def main():
urls = [f"https://httpbin.org/get?i={i}" for i in range(5)]
results = await up(urls)
print([item["args"] for item in results])
asyncio.run(main())
```
This prints:
```
Making async requests: 100%|███████████| 5/5 [00:00<00:00, 9.98it/s]
[{'i': '0'}, {'i': '1'}, {'i': '2'}, {'i': '3'}, {'i': '4'}]
```
Similarly, we can do a bunch of POST requests. This time we will use a single path but multiple payloads:
```python
import asyncio
from unparallel import up
async def main():
url = "https://httpbin.org/post"
payloads = [{"obj_id": i} for i in range(5)]
results = await up(url, method="post", payloads=payloads)
print([item["data"] for item in results])
asyncio.run(main())
```
This prints:
```
Making async requests: 100%|███████████| 5/5 [00:00<00:00, 9.98it/s]
['{"obj_id": 0}', '{"obj_id": 1}', '{"obj_id": 2}', '{"obj_id": 3}', '{"obj_id": 4}']
```
For more details on the usage and examples, check out the [docs][docs-usage].
## Why unparallel? Why async?
Async is a really powerful feature - especially when you have to wait for I/O.
Here is an example of making 20 web requests synchronously vs. asynchronously via `unparallel`.
![Sync-vs-Async][sync-async-gif]
As you can see, the async version finishes in less than a second.
<details><summary>Code for sync</summary>
```python
import httpx
from tqdm import tqdm
def main():
url = "https://httpbin.org"
paths = [f"/get?i={i}" for i in range(20)]
results = [
httpx.get(f"{url}{path}") for path in tqdm(paths, desc="Making sync requests")
]
assert len(results) == 20
if __name__ == "__main__":
main()
```
</details>
<details><summary>Code for async</summary>
```python
import asyncio
from unparallel import up
async def main():
url = "https://httpbin.org"
paths = [f"/get?i={i}" for i in range(20)]
results = await up(paths, base_url=url)
assert len(results) == 20
if __name__ == "__main__":
asyncio.run(main())
```
</details>
## Contributing
As this project is still in early development, I'm happy for any feedback and contributions!
Please refer to the [contributing guidelines][contrib] for details.
## License
This project is licensed under the terms of the `MIT` license. See [LICENSE](https://github.com/RafaelWO/unparallel/blob/main/LICENSE) for more details.
## Credits
This project was heavily inspired by the blog post [Making 1 million requests with python-aiohttp](https://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html) by Paweł Miech.
I created this project with [python-package-template](https://github.com/TezRomacH/python-package-template).
<!-- --8<-- [end:index] -->
[docs-usage]: https://rafaelwo.github.io/unparallel/usage/
[sync-async-gif]: https://raw.githubusercontent.com/RafaelWO/unparallel/main/docs/assets/sync-vs-async.gif
[contrib]: https://github.com/RafaelWO/unparallel/blob/main/CONTRIBUTING.md
Raw data
{
"_id": null,
"home_page": "https://github.com/RafaelWO/unparallel",
"name": "unparallel",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "async,http,requests,network",
"author": "RafaelWO",
"author_email": "weingartner.rafael@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/d9/93/152375ea9c80cbaa3f4f33e75f071fbeb3076e9205e45170040172a4c708/unparallel-0.3.0.tar.gz",
"platform": null,
"description": "# \ud83d\udd00 Unparallel\n\n<!-- --8<-- [start:index] -->\n\n<div align=\"center\" markdown=\"1\">\n\nCreate async HTTP requests with Python in no time.\n\n[](https://github.com/RafaelWO/unparallel/actions?query=workflow%3Atest)\n\n<br>\n[](https://pypi.org/project/unparallel/)\n[](https://pypi.org/project/unparallel/)\n<br>\n[](https://github.com/psf/black)\n[](https://github.com/PyCQA/bandit)\n[](https://github.com/RafaelWO/unparallel/blob/main/.pre-commit-config.yaml)\n[](https://github.com/RafaelWO/unparallel/blob/main/LICENSE)\n<br>\n[](https://rafaelwo.github.io/unparallel/)\n[](https://squidfunk.github.io/mkdocs-material/)\n\n</div>\n\nWith Unparallel you can easily create thousands of web requests in an efficient way leveraging Python's async capabilities.\n\nUnparallel is built on top of [HTTPX](https://github.com/encode/httpx/) and aims to support its rich set of features.\n\n## Installation\n\n```bash\npip install unparallel\n```\n\n## Example\nA simple example of doing several GET requests to an HTTP web service:\n\n```python\nimport asyncio\nfrom unparallel import up\n\nasync def main():\n urls = [f\"https://httpbin.org/get?i={i}\" for i in range(5)]\n results = await up(urls)\n print([item[\"args\"] for item in results])\n\nasyncio.run(main())\n```\n\nThis prints:\n```\nMaking async requests: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5/5 [00:00<00:00, 9.98it/s]\n[{'i': '0'}, {'i': '1'}, {'i': '2'}, {'i': '3'}, {'i': '4'}]\n```\n\n\nSimilarly, we can do a bunch of POST requests. This time we will use a single path but multiple payloads:\n\n```python\nimport asyncio\nfrom unparallel import up\n\nasync def main():\n url = \"https://httpbin.org/post\"\n payloads = [{\"obj_id\": i} for i in range(5)]\n results = await up(url, method=\"post\", payloads=payloads)\n print([item[\"data\"] for item in results])\n\nasyncio.run(main())\n```\n\nThis prints:\n```\nMaking async requests: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5/5 [00:00<00:00, 9.98it/s]\n['{\"obj_id\": 0}', '{\"obj_id\": 1}', '{\"obj_id\": 2}', '{\"obj_id\": 3}', '{\"obj_id\": 4}']\n```\n\nFor more details on the usage and examples, check out the [docs][docs-usage].\n\n## Why unparallel? Why async?\nAsync is a really powerful feature - especially when you have to wait for I/O.\nHere is an example of making 20 web requests synchronously vs. asynchronously via `unparallel`.\n\n![Sync-vs-Async][sync-async-gif]\n\nAs you can see, the async version finishes in less than a second.\n\n<details><summary>Code for sync</summary>\n\n```python\nimport httpx\nfrom tqdm import tqdm\n\n\ndef main():\n url = \"https://httpbin.org\"\n paths = [f\"/get?i={i}\" for i in range(20)]\n results = [\n httpx.get(f\"{url}{path}\") for path in tqdm(paths, desc=\"Making sync requests\")\n ]\n assert len(results) == 20\n\n\nif __name__ == \"__main__\":\n main()\n```\n\n</details>\n\n<details><summary>Code for async</summary>\n\n```python\nimport asyncio\n\nfrom unparallel import up\n\n\nasync def main():\n url = \"https://httpbin.org\"\n paths = [f\"/get?i={i}\" for i in range(20)]\n\n results = await up(paths, base_url=url)\n\n assert len(results) == 20\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n</details>\n\n## Contributing\nAs this project is still in early development, I'm happy for any feedback and contributions! \nPlease refer to the [contributing guidelines][contrib] for details.\n\n## License\n\nThis project is licensed under the terms of the `MIT` license. See [LICENSE](https://github.com/RafaelWO/unparallel/blob/main/LICENSE) for more details.\n\n## Credits \nThis project was heavily inspired by the blog post [Making 1 million requests with python-aiohttp](https://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html) by Pawe\u0142 Miech.\n\nI created this project with [python-package-template](https://github.com/TezRomacH/python-package-template).\n\n<!-- --8<-- [end:index] -->\n\n[docs-usage]: https://rafaelwo.github.io/unparallel/usage/\n[sync-async-gif]: https://raw.githubusercontent.com/RafaelWO/unparallel/main/docs/assets/sync-vs-async.gif\n[contrib]: https://github.com/RafaelWO/unparallel/blob/main/CONTRIBUTING.md\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Create async web requests in no time",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/RafaelWO/unparallel",
"Repository": "https://github.com/RafaelWO/unparallel"
},
"split_keywords": [
"async",
"http",
"requests",
"network"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "aa237f93644df891489127e50a0e539f92dbb965001a2d63ec8174b1df40a6da",
"md5": "571dfc3c1eac627ec34ab930839b99d0",
"sha256": "18ced43c05e7267036670c3b3a00ccd5977f5284dcb3d9e1443950c1a0cdf5b3"
},
"downloads": -1,
"filename": "unparallel-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "571dfc3c1eac627ec34ab930839b99d0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 8269,
"upload_time": "2024-03-02T16:44:13",
"upload_time_iso_8601": "2024-03-02T16:44:13.229406Z",
"url": "https://files.pythonhosted.org/packages/aa/23/7f93644df891489127e50a0e539f92dbb965001a2d63ec8174b1df40a6da/unparallel-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d993152375ea9c80cbaa3f4f33e75f071fbeb3076e9205e45170040172a4c708",
"md5": "63db8f582aac682f6e990b5e00e7b2f0",
"sha256": "72cb94c69ac97f4e4251b38892dc227b009a2a3c980e318dc73aaa52c8327409"
},
"downloads": -1,
"filename": "unparallel-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "63db8f582aac682f6e990b5e00e7b2f0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 8524,
"upload_time": "2024-03-02T16:44:14",
"upload_time_iso_8601": "2024-03-02T16:44:14.897132Z",
"url": "https://files.pythonhosted.org/packages/d9/93/152375ea9c80cbaa3f4f33e75f071fbeb3076e9205e45170040172a4c708/unparallel-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-02 16:44:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "RafaelWO",
"github_project": "unparallel",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "unparallel"
}
JSON
