| Name | json-streamer JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | |
| Summary | A JSON parser capable of parsing incomplete JSON strings from streams/generators. |
| upload_time | 2023-09-21 19:07:40 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | |
| license | MIT |
| keywords |
json
stream
partial
generator
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# JSON Streamer
JSON Streamer is an opportunistic parser designed for processing incomplete JSON strings in real-time from streams.
This can be very useful for parsing JSON strings from network streams, such as websockets and don't want to wait for
the entire JSON string to be received before acting on it.
A good use-case is for parsing OpenAI's GPT response with streaming completion enabled or even with functions enabled.
(This is actually a real use-case! see how [openai-streaming](https://github.com/AlmogBaku/openai-streaming) use this
package just for that)
## Installation
Install the package using pip:
```bash
pip install json-streamer
```
## Features
* **Real-time Parsing**: Process incomplete JSON strings as they are streamed.
* **Flexibility**: Built on a generator-based architecture for ease of integration.
* **Partial Object Handling**: Yields partially parsed JSON objects to be acted upon immediately.
## Usage
### Basic Usage
```python
from json_streamer import loads
# Define a stream of JSON data
def stream_json():
json_string = '{"field1": "value1", "field2": 42}'
chunk_size = 4
for i in range(0, len(json_string), chunk_size):
yield json_string[i:i + chunk_size]
# Create a stream generator
mystream = stream_json()
# Use the 'loads()' function to parse the stream
ret = loads(mystream)
# Iterate through the generator to get parsed JSON objects
for s in ret:
print(s) # Output will be tuples of (ParseState, JSON object)
```
The `loads()` function returns a generator that yields tuples of `(ParseState, JSON object)`.
### Using with Generator receivers
Utilize the parser with a generator receiver for additional control:
```python
from json_streamer import loads
# Initialize the parser generator
parser = loads()
next(parser)
# Stream and send chunks to the parser
for chunk in stream_json():
recv = parser.send(chunk)
print(recv) # Output will be tuples of (ParseState, JSON object)
```
## How it Works
The parser internally maintains a stack of opening symbols like brackets, braces, and quotes. Upon each new data chunk,
it attempts to match these with closing symbols to parse available JSON objects.
## License
This project is licensed under the terms of the MIT license.
Raw data
{
"_id": null,
"home_page": "",
"name": "json-streamer",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "json,stream,partial,generator",
"author": "",
"author_email": "Almog Baku <almog.baku@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/91/19/a99d10a24cc1abcdffd1b87f2a1cd4d4a1faeb31752ba071765d60cd070f/json-streamer-0.1.0.tar.gz",
"platform": null,
"description": "# JSON Streamer\n\nJSON Streamer is an opportunistic parser designed for processing incomplete JSON strings in real-time from streams.\n\nThis can be very useful for parsing JSON strings from network streams, such as websockets and don't want to wait for\nthe entire JSON string to be received before acting on it.\n\nA good use-case is for parsing OpenAI's GPT response with streaming completion enabled or even with functions enabled.\n(This is actually a real use-case! see how [openai-streaming](https://github.com/AlmogBaku/openai-streaming) use this\npackage just for that)\n\n## Installation\n\nInstall the package using pip:\n\n```bash\npip install json-streamer\n```\n\n## Features\n\n* **Real-time Parsing**: Process incomplete JSON strings as they are streamed.\n* **Flexibility**: Built on a generator-based architecture for ease of integration.\n* **Partial Object Handling**: Yields partially parsed JSON objects to be acted upon immediately.\n\n## Usage\n\n### Basic Usage\n\n```python\nfrom json_streamer import loads\n\n\n# Define a stream of JSON data\ndef stream_json():\n json_string = '{\"field1\": \"value1\", \"field2\": 42}'\n chunk_size = 4\n for i in range(0, len(json_string), chunk_size):\n yield json_string[i:i + chunk_size]\n\n\n# Create a stream generator\nmystream = stream_json()\n\n# Use the 'loads()' function to parse the stream\nret = loads(mystream)\n\n# Iterate through the generator to get parsed JSON objects\nfor s in ret:\n print(s) # Output will be tuples of (ParseState, JSON object)\n```\n\nThe `loads()` function returns a generator that yields tuples of `(ParseState, JSON object)`.\n\n### Using with Generator receivers\n\nUtilize the parser with a generator receiver for additional control:\n\n```python\nfrom json_streamer import loads\n\n# Initialize the parser generator\nparser = loads()\nnext(parser)\n\n# Stream and send chunks to the parser\nfor chunk in stream_json():\n recv = parser.send(chunk)\n print(recv) # Output will be tuples of (ParseState, JSON object)\n```\n\n## How it Works\n\nThe parser internally maintains a stack of opening symbols like brackets, braces, and quotes. Upon each new data chunk,\nit attempts to match these with closing symbols to parse available JSON objects.\n\n## License\n\nThis project is licensed under the terms of the MIT license.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A JSON parser capable of parsing incomplete JSON strings from streams/generators.",
"version": "0.1.0",
"project_urls": {
"Bug Reports": "https://github.com/AlmogBaku/json-streamer/issues",
"Homepage": "https://github.com/AlmogBaku/json-streamer",
"Source": "https://github.com/AlmogBaku/json-streamer/"
},
"split_keywords": [
"json",
"stream",
"partial",
"generator"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d4e1ccf4b6a69d1f34ff79c4b9ea3b558bb8c958709da420217e423d724c9dc2",
"md5": "2a453ff0a1a8433ccffa01ce90e5d763",
"sha256": "db45bef31cd8420fddd61b4c15a9d789c60946b83766f30ce7a30a03ab7c5319"
},
"downloads": -1,
"filename": "json_streamer-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2a453ff0a1a8433ccffa01ce90e5d763",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 4934,
"upload_time": "2023-09-21T19:07:39",
"upload_time_iso_8601": "2023-09-21T19:07:39.063695Z",
"url": "https://files.pythonhosted.org/packages/d4/e1/ccf4b6a69d1f34ff79c4b9ea3b558bb8c958709da420217e423d724c9dc2/json_streamer-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9119a99d10a24cc1abcdffd1b87f2a1cd4d4a1faeb31752ba071765d60cd070f",
"md5": "b4655743f8df4c0cce5ac0e991f7f268",
"sha256": "928716960d2252e7fb80a48aafefdde3b4ba8a38abeca7feb7bf332211158c91"
},
"downloads": -1,
"filename": "json-streamer-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "b4655743f8df4c0cce5ac0e991f7f268",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4562,
"upload_time": "2023-09-21T19:07:40",
"upload_time_iso_8601": "2023-09-21T19:07:40.492585Z",
"url": "https://files.pythonhosted.org/packages/91/19/a99d10a24cc1abcdffd1b87f2a1cd4d4a1faeb31752ba071765d60cd070f/json-streamer-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-21 19:07:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AlmogBaku",
"github_project": "json-streamer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "json-streamer"
}
