| Name | iteriotools JSON |
| Version |
0.0.4
JSON |
| download |
| home_page | |
| Summary | Python3.10+ utilities to translate iterators of bytes into file-like objects |
| upload_time | 2024-01-27 21:42:19 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
io
iterators
generators
itertools
utility
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
Ever wanted to stream large JSON objects with [`FastAPI`](https://fastapi.tiangolo.com/) asynchrounously?
Look no more, `iteriotools` is the solution you've been looking for!
It provides a `RawIterIO` class which can wrap an iterator of bytes into a read-only file-like object.
# Example
```bash
cd example
```
To run the example project's application:
```bash
pipenv run uvicorn --reload project:app
```
To demonstrate the impact of `RawIterIO` on streaming large JSON objects, first emulate a "liveness probe" by running
the following:
```bash
while time curl --silent http://127.0.0.1:8000/health > /dev/null; do :; done
```
Then, while still having the probe running, emulate a few clients `GET`-ing some large JSON object with:
```bash
(
trap -- 'kill 0' EXIT
for _ in {0..31}; do
while curl --silent http://127.0.0.1:8000/users/batched > /dev/null; do :; done &
done
wait
)
```
You should notice a significant increase in response time on the "liveness probe" (e.g. 0.003 seconds --> 25 seconds).
Now, with the version that utilizes `RawIterIO` to stream JSON chunk by chunk:
```bash
(
trap -- 'kill 0' EXIT
for _ in {0..31}; do
while curl --silent http://127.0.0.1:8000/users/streamed > /dev/null; do :; done &
done
wait
)
```
You should still notice a significant increase in response time, but way less extreme (e.g. 0.003 seconds -> 2
seconds), and that numbers scales better with the size of the JSON object being streamed instead of being batched.
Raw data
{
"_id": null,
"home_page": "",
"name": "iteriotools",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "io,iterators,generators,itertools,utility",
"author": "",
"author_email": "Quentin Bouget <ypsah@devyard.org>",
"download_url": "https://files.pythonhosted.org/packages/c3/31/2991f0166bf5a01e4a0a95ad433dfd00c4486ea01ab188dffa4f5660156a/iteriotools-0.0.4.tar.gz",
"platform": null,
"description": "Ever wanted to stream large JSON objects with [`FastAPI`](https://fastapi.tiangolo.com/) asynchrounously?\nLook no more, `iteriotools` is the solution you've been looking for!\n\nIt provides a `RawIterIO` class which can wrap an iterator of bytes into a read-only file-like object.\n\n# Example\n\n```bash\ncd example\n```\n\nTo run the example project's application:\n\n```bash\npipenv run uvicorn --reload project:app\n```\n\nTo demonstrate the impact of `RawIterIO` on streaming large JSON objects, first emulate a \"liveness probe\" by running\nthe following:\n\n```bash\nwhile time curl --silent http://127.0.0.1:8000/health > /dev/null; do :; done\n```\n\nThen, while still having the probe running, emulate a few clients `GET`-ing some large JSON object with:\n\n```bash\n(\n trap -- 'kill 0' EXIT\n for _ in {0..31}; do\n while curl --silent http://127.0.0.1:8000/users/batched > /dev/null; do :; done &\n done\n wait\n)\n```\n\nYou should notice a significant increase in response time on the \"liveness probe\" (e.g. 0.003 seconds --> 25 seconds).\n\nNow, with the version that utilizes `RawIterIO` to stream JSON chunk by chunk:\n\n```bash\n(\n trap -- 'kill 0' EXIT\n for _ in {0..31}; do\n while curl --silent http://127.0.0.1:8000/users/streamed > /dev/null; do :; done &\n done\n wait\n)\n```\n\nYou should still notice a significant increase in response time, but way less extreme (e.g. 0.003 seconds -> 2\nseconds), and that numbers scales better with the size of the JSON object being streamed instead of being batched.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python3.10+ utilities to translate iterators of bytes into file-like objects",
"version": "0.0.4",
"project_urls": {
"homepage": "https://gitlab.com/ypsah/iteriotools",
"repository": "https://gitlab.com/ypsah/iteriotools"
},
"split_keywords": [
"io",
"iterators",
"generators",
"itertools",
"utility"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6bdda69173dbf87aff127c84d7fd3c41b58e06a89745bfe80ad57131c6ba0e31",
"md5": "b874d0f5cfadf50859818138f8a25893",
"sha256": "766167e0e2ba8a20829fcbe159b381edc00bbfc0b857fe7e2b44f890a14b8b28"
},
"downloads": -1,
"filename": "iteriotools-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b874d0f5cfadf50859818138f8a25893",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 3564,
"upload_time": "2024-01-27T21:42:17",
"upload_time_iso_8601": "2024-01-27T21:42:17.839084Z",
"url": "https://files.pythonhosted.org/packages/6b/dd/a69173dbf87aff127c84d7fd3c41b58e06a89745bfe80ad57131c6ba0e31/iteriotools-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c3312991f0166bf5a01e4a0a95ad433dfd00c4486ea01ab188dffa4f5660156a",
"md5": "b0ded5c34180a1f8a8b4a8c77de1b1c4",
"sha256": "c9ade6cb5adffca82d8bc83c4da9f930b4c43395f657d5826f30aac1b7f6b852"
},
"downloads": -1,
"filename": "iteriotools-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "b0ded5c34180a1f8a8b4a8c77de1b1c4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 3628,
"upload_time": "2024-01-27T21:42:19",
"upload_time_iso_8601": "2024-01-27T21:42:19.425873Z",
"url": "https://files.pythonhosted.org/packages/c3/31/2991f0166bf5a01e4a0a95ad433dfd00c4486ea01ab188dffa4f5660156a/iteriotools-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-27 21:42:19",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "ypsah",
"gitlab_project": "iteriotools",
"lcname": "iteriotools"
}
