<p align="center">
<!-- Python -->
<a href="https://www.python.org" alt="Python">
<img src="https://badges.aleen42.com/src/python.svg" />
</a>
<!-- Version -->
<a href="https://badge.fury.io/py/unified-io"><img src="https://badge.fury.io/py/unified-io.svg" alt="PyPI version" height="18"></a>
<!-- Black -->
<a href="https://github.com/psf/black" alt="Code style: black">
<img src="https://img.shields.io/badge/code%20style-black-000000.svg" />
</a>
<!-- License -->
<a href="https://lbesson.mit-license.org/"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT"></a>
</a>
</p>
## ⚡️ Introduction
[unified-io](https://github.com/AmenRa/unified-io) is a Python utility that attempts to unify several I/O operations (i.e., read/write data in different formats) under a similar interface while making them more concise and user-friendly.
The library provides a unified interface for reading/writing files, which is based on the following principles:
- Read/write interfaces consist of concise functions with similar signatures.
- Read/write interfaces allows passing keyword arguments to the underlying I/O functions to preserve flexibility.
- Read operations can be performed lazily using generators.
- Before reading/writing, the user can specify a callback function that will be applied to each element of the data stream.
- _read_ functions have _load_ aliases (e.g., `read_csv` has a `load_csv` alias) and _write_ functions have _save_ aliases (e.g., `write_csv` has a `save_csv` alias.
- Use very efficient stuff for each format (e.g., [`orjson`](https://github.com/ijl/orjson) for `json` files). Suggestions are welcome!
## ✨ Supported formats
- [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) and similar formats (e.g., TSV, PSV, etc.)
- [GZIP](https://en.wikipedia.org/wiki/Gzip)
- [JSON](https://en.wikipedia.org/wiki/JSON)
- [JSONl](https://jsonlines.org)
- [LZ4](https://en.wikipedia.org/wiki/LZ4_(compression_algorithm))
- [Numpy](https://numpy.org) (`.npy`)
- [Pickle](https://docs.python.org/3/library/pickle.html)
- [Text](https://en.wikipedia.org/wiki/Text_file)
## 🔌 Requirements
```
python>=3.7
```
## 💾 Installation
```bash
pip install unified-io
```
## 💡 Examples
The API is designed to be as simple as possible. For example, the following code snippet reads a CSV file, applies a callback function to each element of the data stream, and writes the result to a JSONl file:
```python
from unified_io import read_csv, write_jsonl
def callback(x):
return {"id": x["id"], "title": x["title"].lower()}
# Using a generator we avoid loading the entire file into memory
data = read_csv('input.csv', callback=callback, generator=True)
write_jsonl('output.jsonl', data)
```
## 📚 Documentation
Browse the [documentation](https://amenra.github.io/unified-io) for more details and examples.
## 🎁 Feature Requests
Would you like to see other features implemented? Please, open a [feature request](https://github.com/AmenRa/unified-io/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=%5BFeature+Request%5D+title).
## 🤘 Want to contribute?
Would you like to contribute? Please, drop me an [e-mail](mailto:elias.bssn@gmail.com?subject=[GitHub]%20unified-io).
## 📄 License
[unified-io](https://github.com/AmenRa/unified-io) is an open-sourced software licensed under the [MIT license](LICENSE).
Raw data
{
"_id": null,
"home_page": "https://github.com/AmenRa/unified-io",
"name": "unified-io",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "io,oneliner,oneliners,utils,utilities",
"author": "Elias Bassani",
"author_email": "elias.bssn@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a6/de/1c95c9be397e2dd5658a6bc638b8e5f5a2691f4030ffff8ca4206759f04b/unified-io-0.1.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <!-- Python -->\n <a href=\"https://www.python.org\" alt=\"Python\">\n <img src=\"https://badges.aleen42.com/src/python.svg\" />\n </a>\n <!-- Version -->\n <a href=\"https://badge.fury.io/py/unified-io\"><img src=\"https://badge.fury.io/py/unified-io.svg\" alt=\"PyPI version\" height=\"18\"></a>\n <!-- Black -->\n <a href=\"https://github.com/psf/black\" alt=\"Code style: black\">\n <img src=\"https://img.shields.io/badge/code%20style-black-000000.svg\" />\n </a>\n <!-- License -->\n <a href=\"https://lbesson.mit-license.org/\"><img src=\"https://img.shields.io/badge/License-MIT-blue.svg\" alt=\"License: MIT\"></a>\n </a>\n</p>\n\n## \u26a1\ufe0f Introduction\n\n[unified-io](https://github.com/AmenRa/unified-io) is a Python utility that attempts to unify several I/O operations (i.e., read/write data in different formats) under a similar interface while making them more concise and user-friendly.\n\nThe library provides a unified interface for reading/writing files, which is based on the following principles: \n\n- Read/write interfaces consist of concise functions with similar signatures. \n- Read/write interfaces allows passing keyword arguments to the underlying I/O functions to preserve flexibility. \n- Read operations can be performed lazily using generators. \n- Before reading/writing, the user can specify a callback function that will be applied to each element of the data stream. \n- _read_ functions have _load_ aliases (e.g., `read_csv` has a `load_csv` alias) and _write_ functions have _save_ aliases (e.g., `write_csv` has a `save_csv` alias. \n- Use very efficient stuff for each format (e.g., [`orjson`](https://github.com/ijl/orjson) for `json` files). Suggestions are welcome!\n\n## \u2728 Supported formats\n\n- [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) and similar formats (e.g., TSV, PSV, etc.)\n- [GZIP](https://en.wikipedia.org/wiki/Gzip)\n- [JSON](https://en.wikipedia.org/wiki/JSON)\n- [JSONl](https://jsonlines.org)\n- [LZ4](https://en.wikipedia.org/wiki/LZ4_(compression_algorithm))\n- [Numpy](https://numpy.org) (`.npy`)\n- [Pickle](https://docs.python.org/3/library/pickle.html)\n- [Text](https://en.wikipedia.org/wiki/Text_file)\n\n## \ud83d\udd0c Requirements\n```\npython>=3.7\n```\n\n## \ud83d\udcbe Installation\n```bash\npip install unified-io\n```\n\n## \ud83d\udca1 Examples\n\nThe API is designed to be as simple as possible. For example, the following code snippet reads a CSV file, applies a callback function to each element of the data stream, and writes the result to a JSONl file:\n```python\nfrom unified_io import read_csv, write_jsonl\n\ndef callback(x):\n return {\"id\": x[\"id\"], \"title\": x[\"title\"].lower()}\n\n# Using a generator we avoid loading the entire file into memory\ndata = read_csv('input.csv', callback=callback, generator=True)\n\nwrite_jsonl('output.jsonl', data)\n```\n\n## \ud83d\udcda Documentation\nBrowse the [documentation](https://amenra.github.io/unified-io) for more details and examples.\n\n## \ud83c\udf81 Feature Requests\nWould you like to see other features implemented? Please, open a [feature request](https://github.com/AmenRa/unified-io/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=%5BFeature+Request%5D+title).\n\n\n## \ud83e\udd18 Want to contribute?\nWould you like to contribute? Please, drop me an [e-mail](mailto:elias.bssn@gmail.com?subject=[GitHub]%20unified-io).\n\n\n## \ud83d\udcc4 License\n[unified-io](https://github.com/AmenRa/unified-io) is an open-sourced software licensed under the [MIT license](LICENSE).\n",
"bugtrack_url": null,
"license": "",
"summary": "Unified I/O interface for Python",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/AmenRa/unified-io"
},
"split_keywords": [
"io",
"oneliner",
"oneliners",
"utils",
"utilities"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "919a1022038f7ff6d3563a28ebe046c07dcce2712f02d4e3e849979a4e04a9b6",
"md5": "cf2b1e6b4c20b22b5f84374650a5b8c4",
"sha256": "18d82ff7bc1c2d7b005d463b50ad01294e32fb50d9b5aba8b692087a7376efc2"
},
"downloads": -1,
"filename": "unified_io-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cf2b1e6b4c20b22b5f84374650a5b8c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 12293,
"upload_time": "2023-05-19T12:24:40",
"upload_time_iso_8601": "2023-05-19T12:24:40.133948Z",
"url": "https://files.pythonhosted.org/packages/91/9a/1022038f7ff6d3563a28ebe046c07dcce2712f02d4e3e849979a4e04a9b6/unified_io-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a6de1c95c9be397e2dd5658a6bc638b8e5f5a2691f4030ffff8ca4206759f04b",
"md5": "38b79795b1a8db8326c213d6dad1c617",
"sha256": "87eee20cbdefde6143598ffeeb74cb8ff256f73bf69785d5eb76947dedb03a76"
},
"downloads": -1,
"filename": "unified-io-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "38b79795b1a8db8326c213d6dad1c617",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 8381,
"upload_time": "2023-05-19T12:24:42",
"upload_time_iso_8601": "2023-05-19T12:24:42.565853Z",
"url": "https://files.pythonhosted.org/packages/a6/de/1c95c9be397e2dd5658a6bc638b8e5f5a2691f4030ffff8ca4206759f04b/unified-io-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-19 12:24:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AmenRa",
"github_project": "unified-io",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "unified-io"
}
JSON
