[![PyPI Version][pypi-image]][pypi-url]
[![Python Version][versions-image]][versions-url]
[![Github Stars][stars-image]][stars-url]
[![codecov][codecov-image]][codecov-url]
[![Build Status][build-image]][build-url]
[![Documentation][doc-image]][doc-url]
[![License][license-image]][license-url]
[![Downloads][downloads-image]][downloads-url]
[![Downloads][downloads-month-image]][downloads-month-url]
[![Code style: black][codestyle-image]][codestyle-url]
[![snyk][snyk-image]][snyk-url]
# Protarrow
**Protarrow** is a python library for converting from protobuf to arrow and back.
It is used at [Tradewell Technologies](https://www.tradewelltech.co/),
to share data between transactional and analytical applications,
with little boilerplate code and zero data loss.
# Installation
```shell
pip install protarrow
```
# Usage
Taking a simple protobuf:
```protobuf
message MyProto {
string name = 1;
int32 id = 2;
repeated int32 values = 3;
}
```
It can be converted to a `pyarrow.Table`:
```python
import protarrow
my_protos = [
MyProto(name="foo", id=1, values=[1, 2, 4]),
MyProto(name="bar", id=2, values=[3, 4, 5]),
]
table = protarrow.messages_to_table(my_protos, MyProto)
```
| name | id | values |
|:-------|-----:|:---------|
| foo | 1 | [1 2 4] |
| bar | 2 | [3 4 5] |
And the table can be converted back to proto:
```python
protos_from_table = protarrow.table_to_messages(table, MyProto)
```
See the [documentation](https://protarrow.readthedocs.io/en/latest/)
<!-- Badges: -->
[pypi-image]: https://img.shields.io/pypi/v/protarrow
[pypi-url]: https://pypi.org/project/protarrow/
[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/ci.yaml/badge.svg
[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/ci.yaml
[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow
[stars-url]: https://github.com/tradewelltech/protarrow
[versions-image]: https://img.shields.io/pypi/pyversions/protarrow
[versions-url]: https://pypi.org/project/protarrow/
[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest
[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest
[license-image]: http://img.shields.io/:license-Apache%202-blue.svg
[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE
[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70
[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow
[downloads-image]: https://pepy.tech/badge/protarrow
[downloads-url]: https://static.pepy.tech/badge/protarrow
[downloads-month-image]: https://pepy.tech/badge/protarrow/month
[downloads-month-url]: https://static.pepy.tech/badge/protarrow/month
[codestyle-image]: https://img.shields.io/badge/code%20style-black-000000.svg
[codestyle-url]: https://github.com/ambv/black
[snyk-image]: https://snyk.io/advisor/python/protarrow/badge.svg
[snyk-url]: https://snyk.io/advisor/python/protarrow
Raw data
{
"_id": null,
"home_page": "https://github.com/tradewelltech/protarrow",
"name": "protarrow",
"maintainer": "0x26res",
"docs_url": null,
"requires_python": "<3.12,>=3.8",
"maintainer_email": "0x26res@gmail.com",
"keywords": "apache-arrow, protobuf, data",
"author": "Tradewell Tech",
"author_email": "engineering@tradewelltech.co",
"download_url": "https://files.pythonhosted.org/packages/f4/1d/749bfe28402e1ed1347e09c1a13747b0fbfa728775747c264f3c5ffd650a/protarrow-0.4.0.tar.gz",
"platform": null,
"description": "[![PyPI Version][pypi-image]][pypi-url]\n[![Python Version][versions-image]][versions-url]\n[![Github Stars][stars-image]][stars-url]\n[![codecov][codecov-image]][codecov-url]\n[![Build Status][build-image]][build-url]\n[![Documentation][doc-image]][doc-url]\n[![License][license-image]][license-url]\n[![Downloads][downloads-image]][downloads-url]\n[![Downloads][downloads-month-image]][downloads-month-url]\n[![Code style: black][codestyle-image]][codestyle-url]\n[![snyk][snyk-image]][snyk-url]\n\n\n# Protarrow\n\n**Protarrow** is a python library for converting from protobuf to arrow and back.\n\nIt is used at [Tradewell Technologies](https://www.tradewelltech.co/), \nto share data between transactional and analytical applications,\nwith little boilerplate code and zero data loss.\n\n# Installation\n\n```shell\npip install protarrow\n```\n\n# Usage\n\nTaking a simple protobuf:\n\n```protobuf\nmessage MyProto {\n string name = 1;\n int32 id = 2;\n repeated int32 values = 3;\n}\n```\n\nIt can be converted to a `pyarrow.Table`:\n\n```python\nimport protarrow\n\nmy_protos = [\n MyProto(name=\"foo\", id=1, values=[1, 2, 4]),\n MyProto(name=\"bar\", id=2, values=[3, 4, 5]),\n]\n\ntable = protarrow.messages_to_table(my_protos, MyProto)\n```\n\n\n| name | id | values |\n|:-------|-----:|:---------|\n| foo | 1 | [1 2 4] |\n| bar | 2 | [3 4 5] |\n\nAnd the table can be converted back to proto:\n\n```python\nprotos_from_table = protarrow.table_to_messages(table, MyProto)\n```\n\nSee the [documentation](https://protarrow.readthedocs.io/en/latest/)\n\n\n<!-- Badges: -->\n\n[pypi-image]: https://img.shields.io/pypi/v/protarrow\n[pypi-url]: https://pypi.org/project/protarrow/\n[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/ci.yaml/badge.svg\n[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/ci.yaml\n[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow\n[stars-url]: https://github.com/tradewelltech/protarrow\n[versions-image]: https://img.shields.io/pypi/pyversions/protarrow\n[versions-url]: https://pypi.org/project/protarrow/\n[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest\n[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest\n[license-image]: http://img.shields.io/:license-Apache%202-blue.svg\n[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE\n[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70\n[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow\n[downloads-image]: https://pepy.tech/badge/protarrow\n[downloads-url]: https://static.pepy.tech/badge/protarrow\n[downloads-month-image]: https://pepy.tech/badge/protarrow/month\n[downloads-month-url]: https://static.pepy.tech/badge/protarrow/month\n[codestyle-image]: https://img.shields.io/badge/code%20style-black-000000.svg\n[codestyle-url]: https://github.com/ambv/black\n[snyk-image]: https://snyk.io/advisor/python/protarrow/badge.svg\n[snyk-url]: https://snyk.io/advisor/python/protarrow\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Convert from protobuf to arrow and back",
"version": "0.4.0",
"project_urls": {
"Documentation": "https://protarrow.readthedocs.io/en/latest/",
"Homepage": "https://github.com/tradewelltech/protarrow",
"Repository": "https://github.com/tradewelltech/protarrow"
},
"split_keywords": [
"apache-arrow",
" protobuf",
" data"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7974c846dd29d444ea7f5376567c67535f3fbc7fa8e51c627c23c9a1722b08e7",
"md5": "6b7c134a44625c280bb7aa6a694c3288",
"sha256": "ce8594152f068e18e7c7551a58abbfbdb5ae88aef4026d79fbb44c88387b4199"
},
"downloads": -1,
"filename": "protarrow-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6b7c134a44625c280bb7aa6a694c3288",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.8",
"size": 18476,
"upload_time": "2024-04-11T12:53:41",
"upload_time_iso_8601": "2024-04-11T12:53:41.082800Z",
"url": "https://files.pythonhosted.org/packages/79/74/c846dd29d444ea7f5376567c67535f3fbc7fa8e51c627c23c9a1722b08e7/protarrow-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f41d749bfe28402e1ed1347e09c1a13747b0fbfa728775747c264f3c5ffd650a",
"md5": "1dbd60100912ffe0740e494a349c7771",
"sha256": "46412993078ae99a139ecaf3b4939388e2340110a223fde685f8d613fee673ca"
},
"downloads": -1,
"filename": "protarrow-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "1dbd60100912ffe0740e494a349c7771",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.8",
"size": 16754,
"upload_time": "2024-04-11T12:53:42",
"upload_time_iso_8601": "2024-04-11T12:53:42.808660Z",
"url": "https://files.pythonhosted.org/packages/f4/1d/749bfe28402e1ed1347e09c1a13747b0fbfa728775747c264f3c5ffd650a/protarrow-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-11 12:53:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tradewelltech",
"github_project": "protarrow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "protarrow"
}
JSON
