[![PyPI Version][pypi-image]][pypi-url]
[![Python Version][versions-image]][versions-url]
[![Github Stars][stars-image]][stars-url]
[![codecov][codecov-image]][codecov-url]
[![Build Status][build-image]][build-url]
[![Documentation][doc-image]][doc-url]
[![License][license-image]][license-url]
# Protarrow
**Protarrow** is a python library for converting from protobuf to arrow and back.
It is used at [Tradewell Technologies](https://www.tradewelltech.co/),
to share date between transactional and analytical applications,
with little boilerplate code and zero data loss.
# Installation
```shell
pip install protarrow
```
# Usage
Taking a simple protobuf:
```protobuf
message MyProto {
string name = 1;
int32 id = 2;
repeated int32 values = 3;
}
```
It can be converted to a `pyarrow.Table`:
```python
import protarrow
my_protos = [
MyProto(name="foo", id=1, values=[1, 2, 4]),
MyProto(name="bar", id=2, values=[3, 4, 5]),
]
table = protarrow.messages_to_table(my_protos, MyProto)
```
| name | id | values |
|:-------|-----:|:---------|
| foo | 1 | [1 2 4] |
| bar | 2 | [3 4 5] |
And the table can be converted back to proto:
```python
protos_from_table = protarrow.table_to_messages(table, MyProto)
```
See the [documentation](https://protarrow.readthedocs.io/en/latest/)
<!-- Badges: -->
[pypi-image]: https://img.shields.io/pypi/v/protarrow
[pypi-url]: https://pypi.org/project/protarrow/
[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml/badge.svg
[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml
[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow
[stars-url]: https://github.com/tradewelltech/protarrow
[versions-image]: https://img.shields.io/pypi/pyversions/protarrow
[versions-url]: https://pypi.org/project/protarrow/
[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest
[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest
[license-image]: http://img.shields.io/:license-Apache%202-blue.svg
[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE
[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70
[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow
Raw data
{
"_id": null,
"home_page": "https://github.com/tradewelltech/protarrow",
"name": "protarrow",
"maintainer": "0x26res",
"docs_url": null,
"requires_python": ">=3.8,<3.11",
"maintainer_email": "0x26res@gmail.com",
"keywords": "apache-arrow,protobuf,data",
"author": "Tradewell Tech",
"author_email": "engineering@tradewelltech.co",
"download_url": "https://files.pythonhosted.org/packages/d5/30/ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc/protarrow-0.1.3.tar.gz",
"platform": null,
"description": "[![PyPI Version][pypi-image]][pypi-url]\n[![Python Version][versions-image]][versions-url]\n[![Github Stars][stars-image]][stars-url]\n[![codecov][codecov-image]][codecov-url]\n[![Build Status][build-image]][build-url]\n[![Documentation][doc-image]][doc-url]\n[![License][license-image]][license-url]\n\n# Protarrow\n\n**Protarrow** is a python library for converting from protobuf to arrow and back.\n\nIt is used at [Tradewell Technologies](https://www.tradewelltech.co/), \nto share date between transactional and analytical applications,\nwith little boilerplate code and zero data loss.\n\n# Installation\n\n```shell\npip install protarrow\n```\n\n# Usage\n\nTaking a simple protobuf:\n\n```protobuf\nmessage MyProto {\n string name = 1;\n int32 id = 2;\n repeated int32 values = 3;\n}\n```\n\nIt can be converted to a `pyarrow.Table`:\n\n```python\nimport protarrow\n\nmy_protos = [\n MyProto(name=\"foo\", id=1, values=[1, 2, 4]),\n MyProto(name=\"bar\", id=2, values=[3, 4, 5]),\n]\n\ntable = protarrow.messages_to_table(my_protos, MyProto)\n```\n\n\n| name | id | values |\n|:-------|-----:|:---------|\n| foo | 1 | [1 2 4] |\n| bar | 2 | [3 4 5] |\n\nAnd the table can be converted back to proto:\n\n```python\nprotos_from_table = protarrow.table_to_messages(table, MyProto)\n```\n\nSee the [documentation](https://protarrow.readthedocs.io/en/latest/)\n\n\n<!-- Badges: -->\n\n[pypi-image]: https://img.shields.io/pypi/v/protarrow\n[pypi-url]: https://pypi.org/project/protarrow/\n[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml/badge.svg\n[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml\n[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow\n[stars-url]: https://github.com/tradewelltech/protarrow\n[versions-image]: https://img.shields.io/pypi/pyversions/protarrow\n[versions-url]: https://pypi.org/project/protarrow/\n[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest\n[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest\n[license-image]: http://img.shields.io/:license-Apache%202-blue.svg\n[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE\n[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70\n[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Convert from protobuf to arrow and back",
"version": "0.1.3",
"split_keywords": [
"apache-arrow",
"protobuf",
"data"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "272a1b1553000cdf20f4e270d94d59914f8c8f0cf7e3d7fc91df5b0c4bc0234d",
"md5": "7b4454954132a61523a9ba128cd28c4b",
"sha256": "d930e7d17975be384cdc13e8d51ef5285c285e4d5c95f430c3a74ac3dcd29b8c"
},
"downloads": -1,
"filename": "protarrow-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7b4454954132a61523a9ba128cd28c4b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.11",
"size": 17708,
"upload_time": "2023-01-04T16:34:41",
"upload_time_iso_8601": "2023-01-04T16:34:41.463546Z",
"url": "https://files.pythonhosted.org/packages/27/2a/1b1553000cdf20f4e270d94d59914f8c8f0cf7e3d7fc91df5b0c4bc0234d/protarrow-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d530ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc",
"md5": "647e5431eb546900b0b99acab6e1939a",
"sha256": "d338c3b4dc6af574b2c1b0290e1d49d843145fef5c984c6ffff6c24b1183476f"
},
"downloads": -1,
"filename": "protarrow-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "647e5431eb546900b0b99acab6e1939a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.11",
"size": 16270,
"upload_time": "2023-01-04T16:34:42",
"upload_time_iso_8601": "2023-01-04T16:34:42.780255Z",
"url": "https://files.pythonhosted.org/packages/d5/30/ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc/protarrow-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-04 16:34:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "tradewelltech",
"github_project": "protarrow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "protarrow"
}
JSON
