# jsonl
[](https://github.com/rmoralespp/jsonl/actions?query=event%3Arelease+workflow%3ACI)
[](https://pypi.python.org/pypi/py-jsonl)
[](https://github.com/rmoralespp/jsonl)
[](https://app.codecov.io/gh/rmoralespp/jsonl)
[](https://github.com/rmoralespp/jsonl/blob/main/LICENSE)
[](https://github.com/charliermarsh/ruff)
[](https://pepy.tech/project/py-jsonl)
## About
**jsonl** is a Python library designed to simplify working with JSON Lines data, adhering to
the [JSON Lines format](https://jsonlines.org/).
### Features
- 🌎 Provides an API similar to Python's standard `json` module.
- 🚀 Supports custom serialization/deserialization callbacks, with the standard `json` module as the default.
- 🗜️ Supports compression and decompression using `gzip`, `bzip2`, and `xz` formats.
- 🔧 Can load files with broken lines, skipping any malformed entries.
- 📦 Includes an easy-to-use utility for writing to multiple JSON Lines files.
## Installation
To install **jsonl** using `pip`, run the following command:
```bash
pip install py-jsonl
```
## Getting Started
**Dumping data to a JSON Lines File**
Use `jsonl.dump` to incrementally write an iterable of dictionaries to a JSON Lines file:
```python
import jsonl
data = [
{"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]},
{"name": "May", "wins": []},
]
jsonl.dump(data, "file.jsonl")
```
**Loading data from a JSON Lines File**
Use `jsonl.load` to incrementally load a JSON Lines file into an iterable of objects:
```python
import jsonl
iterable = jsonl.load("file.jsonl")
print(tuple(iterable))
```
**Dumping data to Multiple JSON Lines Files**
Use `jsonl.dump_fork` to incrementally write an iterable to multiple JSON Lines files:
This example uses `jsonl.dump_fork` to incrementally write fake daily temperature data for multiple cities to separate JSON
Lines files, exporting records for the first days of specified years.
It efficiently manages data by creating individual files for each city, optimizing memory usage.
```python
import datetime
import itertools
import random
import jsonl
def fetch_temperature_by_city():
"""
Yielding filenames for each city with fake daily temperature data for the initial days of
the specified years.
"""
years = [2023, 2024]
first_days = 10
cities = ["New York", "Los Angeles", "Chicago"]
for year, city in itertools.product(years, cities):
start = datetime.datetime(year, 1, 1)
dates = (start + datetime.timedelta(days=day) for day in range(first_days))
daily_temperature = (
{"date": date.isoformat(), "city": city, "temperature": round(random.uniform(-10, 35), 2)}
for date in dates
)
yield (f"{city}.jsonl", daily_temperature)
# Write the generated data to files in JSON Lines format
jsonl.dump_fork(fetch_temperature_by_city())
```
## Documentation
For more detailed information and usage examples, refer to the
project [documentation](https://rmoralespp.github.io/jsonl/)
## Development
To contribute to the project, you can run the following commands for testing and documentation:
### Running Unit Tests
Install the development dependencies and run the tests:
```
(env)$ pip install -r requirements-dev.txt # Skip if already installed
(env)$ pytest tests/
(env)$ pytest --cov jsonl # Run tests with coverage
```
### Building the Documentation
To build the documentation locally, use the following commands:
```
(env)$ pip install -r requirements-docs.txt # Skip if already installed
(env)$ mkdocs serve # Start live-reloading docs server
(env)$ mkdocs build # Build the documentation site
```
## License
This project is licensed under the [MIT license](LICENSE).
Raw data
{
"_id": null,
"home_page": null,
"name": "py-jsonl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "jsonlines, ndjson, jsonl, json, gzip, bzip2, xz, utilities, serialization, deserialization",
"author": null,
"author_email": "rmoralespp <rmoralespp@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/cb/b2/232740ebb7428b282b63fb226984f8a3fe594c79ed82bf2f145d666320d2/py_jsonl-1.3.10.tar.gz",
"platform": null,
"description": "# jsonl\n\n[](https://github.com/rmoralespp/jsonl/actions?query=event%3Arelease+workflow%3ACI)\n[](https://pypi.python.org/pypi/py-jsonl)\n[](https://github.com/rmoralespp/jsonl)\n[](https://app.codecov.io/gh/rmoralespp/jsonl)\n[](https://github.com/rmoralespp/jsonl/blob/main/LICENSE)\n[](https://github.com/charliermarsh/ruff)\n[](https://pepy.tech/project/py-jsonl)\n\n## About\n\n**jsonl** is a Python library designed to simplify working with JSON Lines data, adhering to\nthe [JSON Lines format](https://jsonlines.org/).\n\n### Features\n\n- \ud83c\udf0e Provides an API similar to Python's standard `json` module.\n- \ud83d\ude80 Supports custom serialization/deserialization callbacks, with the standard `json` module as the default.\n- \ud83d\udddc\ufe0f Supports compression and decompression using `gzip`, `bzip2`, and `xz` formats.\n- \ud83d\udd27 Can load files with broken lines, skipping any malformed entries.\n- \ud83d\udce6 Includes an easy-to-use utility for writing to multiple JSON Lines files.\n\n## Installation\n\nTo install **jsonl** using `pip`, run the following command:\n\n```bash\npip install py-jsonl\n```\n\n## Getting Started\n\n**Dumping data to a JSON Lines File**\n\nUse `jsonl.dump` to incrementally write an iterable of dictionaries to a JSON Lines file:\n\n```python\nimport jsonl\n\ndata = [\n {\"name\": \"Gilbert\", \"wins\": [[\"straight\", \"7\u2663\"], [\"one pair\", \"10\u2665\"]]},\n {\"name\": \"May\", \"wins\": []},\n]\n\njsonl.dump(data, \"file.jsonl\")\n```\n\n**Loading data from a JSON Lines File**\n\nUse `jsonl.load` to incrementally load a JSON Lines file into an iterable of objects:\n\n```python\nimport jsonl\n\niterable = jsonl.load(\"file.jsonl\")\nprint(tuple(iterable))\n```\n\n**Dumping data to Multiple JSON Lines Files**\n\nUse `jsonl.dump_fork` to incrementally write an iterable to multiple JSON Lines files:\n\nThis example uses `jsonl.dump_fork` to incrementally write fake daily temperature data for multiple cities to separate JSON\nLines files, exporting records for the first days of specified years.\nIt efficiently manages data by creating individual files for each city, optimizing memory usage.\n\n```python\nimport datetime\nimport itertools\nimport random\n\nimport jsonl\n\n\ndef fetch_temperature_by_city():\n \"\"\"\n Yielding filenames for each city with fake daily temperature data for the initial days of\n the specified years.\n \"\"\"\n\n years = [2023, 2024]\n first_days = 10\n cities = [\"New York\", \"Los Angeles\", \"Chicago\"]\n\n for year, city in itertools.product(years, cities):\n start = datetime.datetime(year, 1, 1)\n dates = (start + datetime.timedelta(days=day) for day in range(first_days))\n daily_temperature = (\n {\"date\": date.isoformat(), \"city\": city, \"temperature\": round(random.uniform(-10, 35), 2)}\n for date in dates\n )\n yield (f\"{city}.jsonl\", daily_temperature)\n\n\n# Write the generated data to files in JSON Lines format\njsonl.dump_fork(fetch_temperature_by_city())\n```\n\n## Documentation\n\nFor more detailed information and usage examples, refer to the\nproject [documentation](https://rmoralespp.github.io/jsonl/)\n\n## Development\n\nTo contribute to the project, you can run the following commands for testing and documentation:\n\n### Running Unit Tests\n\nInstall the development dependencies and run the tests:\n\n```\n(env)$ pip install -r requirements-dev.txt # Skip if already installed\n(env)$ pytest tests/\n(env)$ pytest --cov jsonl # Run tests with coverage\n```\n\n### Building the Documentation\n\nTo build the documentation locally, use the following commands:\n\n```\n(env)$ pip install -r requirements-docs.txt # Skip if already installed\n(env)$ mkdocs serve # Start live-reloading docs server\n(env)$ mkdocs build # Build the documentation site\n```\n\n## License\n\nThis project is licensed under the [MIT license](LICENSE).\n",
"bugtrack_url": null,
"license": null,
"summary": "A simple Python library for handling jsonlines files.",
"version": "1.3.10",
"project_urls": {
"Changelog": "https://github.com/rmoralespp/jsonl/blob/main/CHANGELOG.md",
"Documentation": "https://rmoralespp.github.io/jsonl/",
"Homepage": "https://github.com/rmoralespp/jsonl",
"Issues": "https://github.com/rmoralespp/jsonl/issues",
"Source": "https://github.com/rmoralespp/jsonl"
},
"split_keywords": [
"jsonlines",
" ndjson",
" jsonl",
" json",
" gzip",
" bzip2",
" xz",
" utilities",
" serialization",
" deserialization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cf4fd5c302142ceb8760b8afe84d812f0943425cbb74e755fb1f3f743d85a74c",
"md5": "4daa7cb3bd5878ce8e06790969943f58",
"sha256": "938a0b59ef36057b6872ff115c67d058785b992519a08ff8d1bd73982691bb90"
},
"downloads": -1,
"filename": "py_jsonl-1.3.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4daa7cb3bd5878ce8e06790969943f58",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 6153,
"upload_time": "2025-01-09T12:05:21",
"upload_time_iso_8601": "2025-01-09T12:05:21.096874Z",
"url": "https://files.pythonhosted.org/packages/cf/4f/d5c302142ceb8760b8afe84d812f0943425cbb74e755fb1f3f743d85a74c/py_jsonl-1.3.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cbb2232740ebb7428b282b63fb226984f8a3fe594c79ed82bf2f145d666320d2",
"md5": "e677b2d17d12c3c30be77af513dadf7d",
"sha256": "fb5beefc160acecf1c67fcbbe4fbb9e35258787c4f7781427c3addceb7540807"
},
"downloads": -1,
"filename": "py_jsonl-1.3.10.tar.gz",
"has_sig": false,
"md5_digest": "e677b2d17d12c3c30be77af513dadf7d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8104,
"upload_time": "2025-01-09T12:05:22",
"upload_time_iso_8601": "2025-01-09T12:05:22.065181Z",
"url": "https://files.pythonhosted.org/packages/cb/b2/232740ebb7428b282b63fb226984f8a3fe594c79ed82bf2f145d666320d2/py_jsonl-1.3.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-09 12:05:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rmoralespp",
"github_project": "jsonl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "py-jsonl"
}
JSON
