protarrow


Nameprotarrow JSON
Version 0.1.3 PyPI version JSON
download
home_pagehttps://github.com/tradewelltech/protarrow
SummaryConvert from protobuf to arrow and back
upload_time2023-01-04 16:34:42
maintainer0x26res
docs_urlNone
authorTradewell Tech
requires_python>=3.8,<3.11
licenseApache-2.0
keywords apache-arrow protobuf data
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            [![PyPI Version][pypi-image]][pypi-url]
[![Python Version][versions-image]][versions-url]
[![Github Stars][stars-image]][stars-url]
[![codecov][codecov-image]][codecov-url]
[![Build Status][build-image]][build-url]
[![Documentation][doc-image]][doc-url]
[![License][license-image]][license-url]

# Protarrow

**Protarrow** is a python library for converting from protobuf to arrow and back.

It is used at [Tradewell Technologies](https://www.tradewelltech.co/), 
to share date between transactional and analytical applications,
with little boilerplate code and zero data loss.

# Installation

```shell
pip install protarrow
```

# Usage

Taking a simple protobuf:

```protobuf
message MyProto {
  string name = 1;
  int32 id = 2;
  repeated int32 values = 3;
}
```

It can be converted to a `pyarrow.Table`:

```python
import protarrow

my_protos = [
    MyProto(name="foo", id=1, values=[1, 2, 4]),
    MyProto(name="bar", id=2, values=[3, 4, 5]),
]

table = protarrow.messages_to_table(my_protos, MyProto)
```


| name   |   id | values   |
|:-------|-----:|:---------|
| foo    |    1 | [1 2 4]  |
| bar    |    2 | [3 4 5]  |

And the table can be converted back to proto:

```python
protos_from_table = protarrow.table_to_messages(table, MyProto)
```

See the [documentation](https://protarrow.readthedocs.io/en/latest/)


<!-- Badges: -->

[pypi-image]: https://img.shields.io/pypi/v/protarrow
[pypi-url]: https://pypi.org/project/protarrow/
[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml/badge.svg
[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml
[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow
[stars-url]: https://github.com/tradewelltech/protarrow
[versions-image]: https://img.shields.io/pypi/pyversions/protarrow
[versions-url]: https://pypi.org/project/protarrow/
[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest
[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest
[license-image]: http://img.shields.io/:license-Apache%202-blue.svg
[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE
[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70
[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/tradewelltech/protarrow",
    "name": "protarrow",
    "maintainer": "0x26res",
    "docs_url": null,
    "requires_python": ">=3.8,<3.11",
    "maintainer_email": "0x26res@gmail.com",
    "keywords": "apache-arrow,protobuf,data",
    "author": "Tradewell Tech",
    "author_email": "engineering@tradewelltech.co",
    "download_url": "https://files.pythonhosted.org/packages/d5/30/ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc/protarrow-0.1.3.tar.gz",
    "platform": null,
    "description": "[![PyPI Version][pypi-image]][pypi-url]\n[![Python Version][versions-image]][versions-url]\n[![Github Stars][stars-image]][stars-url]\n[![codecov][codecov-image]][codecov-url]\n[![Build Status][build-image]][build-url]\n[![Documentation][doc-image]][doc-url]\n[![License][license-image]][license-url]\n\n# Protarrow\n\n**Protarrow** is a python library for converting from protobuf to arrow and back.\n\nIt is used at [Tradewell Technologies](https://www.tradewelltech.co/), \nto share date between transactional and analytical applications,\nwith little boilerplate code and zero data loss.\n\n# Installation\n\n```shell\npip install protarrow\n```\n\n# Usage\n\nTaking a simple protobuf:\n\n```protobuf\nmessage MyProto {\n  string name = 1;\n  int32 id = 2;\n  repeated int32 values = 3;\n}\n```\n\nIt can be converted to a `pyarrow.Table`:\n\n```python\nimport protarrow\n\nmy_protos = [\n    MyProto(name=\"foo\", id=1, values=[1, 2, 4]),\n    MyProto(name=\"bar\", id=2, values=[3, 4, 5]),\n]\n\ntable = protarrow.messages_to_table(my_protos, MyProto)\n```\n\n\n| name   |   id | values   |\n|:-------|-----:|:---------|\n| foo    |    1 | [1 2 4]  |\n| bar    |    2 | [3 4 5]  |\n\nAnd the table can be converted back to proto:\n\n```python\nprotos_from_table = protarrow.table_to_messages(table, MyProto)\n```\n\nSee the [documentation](https://protarrow.readthedocs.io/en/latest/)\n\n\n<!-- Badges: -->\n\n[pypi-image]: https://img.shields.io/pypi/v/protarrow\n[pypi-url]: https://pypi.org/project/protarrow/\n[build-image]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml/badge.svg\n[build-url]: https://github.com/tradewelltech/protarrow/actions/workflows/build.yaml\n[stars-image]: https://img.shields.io/github/stars/tradewelltech/protarrow\n[stars-url]: https://github.com/tradewelltech/protarrow\n[versions-image]: https://img.shields.io/pypi/pyversions/protarrow\n[versions-url]: https://pypi.org/project/protarrow/\n[doc-image]: https://readthedocs.org/projects/protarrow/badge/?version=latest\n[doc-url]: https://protarrow.readthedocs.io/en/latest/?badge=latest\n[license-image]: http://img.shields.io/:license-Apache%202-blue.svg\n[license-url]: https://github.com/tradewelltech/protarrow/blob/master/LICENSE\n[codecov-image]: https://codecov.io/gh/tradewelltech/protarrow/branch/master/graph/badge.svg?token=XMFH27IL70\n[codecov-url]: https://codecov.io/gh/tradewelltech/protarrow\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "Convert from protobuf to arrow and back",
    "version": "0.1.3",
    "split_keywords": [
        "apache-arrow",
        "protobuf",
        "data"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "272a1b1553000cdf20f4e270d94d59914f8c8f0cf7e3d7fc91df5b0c4bc0234d",
                "md5": "7b4454954132a61523a9ba128cd28c4b",
                "sha256": "d930e7d17975be384cdc13e8d51ef5285c285e4d5c95f430c3a74ac3dcd29b8c"
            },
            "downloads": -1,
            "filename": "protarrow-0.1.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "7b4454954132a61523a9ba128cd28c4b",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8,<3.11",
            "size": 17708,
            "upload_time": "2023-01-04T16:34:41",
            "upload_time_iso_8601": "2023-01-04T16:34:41.463546Z",
            "url": "https://files.pythonhosted.org/packages/27/2a/1b1553000cdf20f4e270d94d59914f8c8f0cf7e3d7fc91df5b0c4bc0234d/protarrow-0.1.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d530ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc",
                "md5": "647e5431eb546900b0b99acab6e1939a",
                "sha256": "d338c3b4dc6af574b2c1b0290e1d49d843145fef5c984c6ffff6c24b1183476f"
            },
            "downloads": -1,
            "filename": "protarrow-0.1.3.tar.gz",
            "has_sig": false,
            "md5_digest": "647e5431eb546900b0b99acab6e1939a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8,<3.11",
            "size": 16270,
            "upload_time": "2023-01-04T16:34:42",
            "upload_time_iso_8601": "2023-01-04T16:34:42.780255Z",
            "url": "https://files.pythonhosted.org/packages/d5/30/ab7deb93a9aabb59f04d41c7145b00cecd97c9778e050cd5f92a85ba06cc/protarrow-0.1.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-01-04 16:34:42",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "tradewelltech",
    "github_project": "protarrow",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "tox": true,
    "lcname": "protarrow"
}
        
Elapsed time: 0.06164s