==============
avro-to-python
==============
avro-to-python is a light tool for compiling avro schema files (.avsc) to python classes making using avro schemata easy.
* Free software: MIT license
* Documentation: https://avro-to-python.readthedocs.io.
Installation
^^^^^^^^^^^^
Pip install (recommended)
-------------------------
To install avro-to-python, run this command in your terminal:
.. code-block:: console
$ pip install avro-to-python
Install From Source ()
----------------------
The sources for avro-to-python can be downloaded source as well.
Clone the public repository:
.. code-block:: console
$ git clone git://github.com/srserves85/avro-to-python
Once you have a copy of the source, you can install it with:
.. code-block:: console
$ python setup.py install
or
.. code-block:: console
$ pip install -e .
Examples
^^^^^^^^
Majority of the use of avro-to-python is assumed to be used as a cli, but you can still import and use the python classes under the hood as well.
CLI (without --pip)
-------------------
To use the cli, here is the available cli commands:
.. code-block:: bash
avro-to-python [source] [target]
Options:
--pip TEXT make package pip installable using this name
--top_level_package override top level package name (optional)
--author TEXT author name of the pip installable package
--package_version TEXT version of the pip intallable package [default: 0.1.0]
--help Show this message and exit
The above will compile the avsc files and convert the to python classes found in [path_to_target_directory]
An example of doing this is the following:
.. code-block:: bash
avro-to-python [path_to_source_avsc_files] [path_to_target_directory]
If you run the above on a valid avro avsc file, you should then be able to import them as you would in the avro idl namespace Here is an example of a single avsc record from the namespace: *name.space* and name: *RecordClass*:
.. code-block:: python
from name.space import RecordClass
record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})
Tips: To generate classes in a subpackage of your existing application set the "--top_level_package" flags to your subpackage name:
.. code-block:: bash
avro-to-python [path_to_source_avsc_files] [path_to_my_subpackage_directory] --top_level_package my.subpackage
.. code-block:: python
from my.subpackage.name.space import RecordClass
record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})
CLI (with --pip)
----------------
You can also choose to make compiled avro packages ***pip installable*** by adding the "--pip" flags. An example of this is the following:
.. code-block:: bash
avro-to-python [path_to_source_avsc_files] [path_to_target_directory] --pip test_avro
By running this, you should be able to pip install the above package you created from the target directory you specified by running:
.. code-block:: bash
pip install -e path_to_target_directory
Now that you have the package installed, you can import it by it's package name and namespace. Here is the same example of the same avsc from above, only with a pip package of *test_avro*:
.. code-block:: python
from test_avro.name.space import RecordClass
record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})
You can customize the top level package name *test_avro*, modifying the "--top_level_package" flags:
.. code-block:: bash
avro-to-python [path_to_source_avsc_files] [path_to_target_directory] --pip test_avro --top_level_package event
.. code-block:: python
from event.name.space import RecordClass
record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})
avro-to-python in a Script
--------------------------
You can also use the reader and writer packages in avro-to-python as you would any other python package. Avro to python is split between a *reader* and *writer* classes. avro-to-python treates namespaces as acyclic trees and uses depth first search to ensure no duplication or namespace collisions on read and write. An example useage is below:
.. code-block:: python
from avro_to_python.reader import AvscReader
from avro_to_python.writer import AvroWriter
# initialize the reader object
reader = AvscReader(directory='tests/avsc/records/')
# generate the acyclic tree object
reader.read()
# initialize the writer object
writer = AvroWriter(reader.file_tree, pip='test_pip')
# compile python files using 'tests/test_records as the namespace root'
writer.write(root_dir='tests/test_records')
Roadmap
^^^^^^^
Reader
- [X] Create Namespace Trees on nested namespaces
- [X] Read Record and Enum File
- [X] Primitive types
- [X] Array Types
- [X] Union types
- [X] References to other files
- [X] Map Types
- [ ] Logical Types (Currently just converts to primitive types)
Writer
- [X] Base Schema Writer
- [X] Base Record Schema
- [X] Base Enum Schema
- [X] Primitive Types Schema
- [X] Array Types Schema
- [X] Union Types Schema
- [X] Map Types
- [ ] Logical Types Schema (Currently just converts to primitive types)
- [X] Add configs to pip install package
CLI
- [X] Wrap Writer and Reader into one cli commmit
- [X] Add pip install option (would include all files to pip install compiled package)
- [ ] Add better --help documentation
Documentation
- [ ] Document reader class
- [ ] Document writer class
- [ ] Document cli
=======
History
=======
0.3.3 (2022-01-26)
^^^^^^^^^^^^^^^^^^
* Fix array reference handling.
0.3.2 (2021-11-16)
^^^^^^^^^^^^^^^^^^
* Add map type to unions.
0.3.1 (2020-06-10)
^^^^^^^^^^^^^^^^^^
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/LeComptoirDesPharmacies/avro-to-python",
"name": "lcdp-avro-to-python",
"maintainer": null,
"docs_url": null,
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>3.5",
"maintainer_email": null,
"keywords": "avro-to-python",
"author": "Scott Rothbarth",
"author_email": "srserves85@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f2/28/9d07c4509eea134c3fa57746c8a9b8b4815a4166e3db7b8984634d7390b2/lcdp_avro_to_python-0.3.3.post8.tar.gz",
"platform": null,
"description": "==============\navro-to-python\n==============\n\navro-to-python is a light tool for compiling avro schema files (.avsc) to python classes making using avro schemata easy.\n\n\n* Free software: MIT license\n* Documentation: https://avro-to-python.readthedocs.io.\n\nInstallation\n^^^^^^^^^^^^\n\nPip install (recommended)\n-------------------------\nTo install avro-to-python, run this command in your terminal:\n\n.. code-block:: console\n\n $ pip install avro-to-python\n\nInstall From Source ()\n----------------------\n\nThe sources for avro-to-python can be downloaded source as well.\n\nClone the public repository:\n\n.. code-block:: console\n\n $ git clone git://github.com/srserves85/avro-to-python\n\n\nOnce you have a copy of the source, you can install it with:\n\n\n.. code-block:: console\n\n $ python setup.py install\n\nor\n\n.. code-block:: console\n\n $ pip install -e .\n\n\nExamples\n^^^^^^^^\n\nMajority of the use of avro-to-python is assumed to be used as a cli, but you can still import and use the python classes under the hood as well.\n\nCLI (without --pip)\n-------------------\nTo use the cli, here is the available cli commands:\n\n.. code-block:: bash\n\n avro-to-python [source] [target]\n Options:\n --pip TEXT make package pip installable using this name\n --top_level_package override top level package name (optional)\n --author TEXT author name of the pip installable package\n --package_version TEXT version of the pip intallable package [default: 0.1.0]\n --help Show this message and exit\n\n\nThe above will compile the avsc files and convert the to python classes found in [path_to_target_directory]\n\nAn example of doing this is the following:\n\n.. code-block:: bash\n\n avro-to-python [path_to_source_avsc_files] [path_to_target_directory]\n\n\nIf you run the above on a valid avro avsc file, you should then be able to import them as you would in the avro idl namespace Here is an example of a single avsc record from the namespace: *name.space* and name: *RecordClass*:\n\n.. code-block:: python\n\n from name.space import RecordClass\n\n record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})\n\nTips: To generate classes in a subpackage of your existing application set the \"--top_level_package\" flags to your subpackage name:\n\n.. code-block:: bash\n\n avro-to-python [path_to_source_avsc_files] [path_to_my_subpackage_directory] --top_level_package my.subpackage\n\n.. code-block:: python\n\n from my.subpackage.name.space import RecordClass\n\n record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})\n\nCLI (with --pip)\n----------------\nYou can also choose to make compiled avro packages ***pip installable*** by adding the \"--pip\" flags. An example of this is the following:\n.. code-block:: bash\n\n avro-to-python [path_to_source_avsc_files] [path_to_target_directory] --pip test_avro\n\nBy running this, you should be able to pip install the above package you created from the target directory you specified by running:\n\n.. code-block:: bash\n\n pip install -e path_to_target_directory\n\nNow that you have the package installed, you can import it by it's package name and namespace. Here is the same example of the same avsc from above, only with a pip package of *test_avro*:\n\n.. code-block:: python\n\n from test_avro.name.space import RecordClass\n\n record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})\n\nYou can customize the top level package name *test_avro*, modifying the \"--top_level_package\" flags:\n\n.. code-block:: bash\n\n avro-to-python [path_to_source_avsc_files] [path_to_target_directory] --pip test_avro --top_level_package event\n\n.. code-block:: python\n\n from event.name.space import RecordClass\n\n record = RecordClass({'foo': True, 'bar': 'true', 'baz': 10, 'food': 'CHOCOLATE'})\n\n\navro-to-python in a Script\n--------------------------\nYou can also use the reader and writer packages in avro-to-python as you would any other python package. Avro to python is split between a *reader* and *writer* classes. avro-to-python treates namespaces as acyclic trees and uses depth first search to ensure no duplication or namespace collisions on read and write. An example useage is below:\n\n.. code-block:: python\n\n from avro_to_python.reader import AvscReader\n from avro_to_python.writer import AvroWriter\n\n # initialize the reader object\n reader = AvscReader(directory='tests/avsc/records/')\n\n # generate the acyclic tree object\n reader.read()\n\n # initialize the writer object\n writer = AvroWriter(reader.file_tree, pip='test_pip')\n\n # compile python files using 'tests/test_records as the namespace root'\n writer.write(root_dir='tests/test_records')\n\n\n\nRoadmap\n^^^^^^^\n\nReader\n\n- [X] Create Namespace Trees on nested namespaces\n- [X] Read Record and Enum File\n- [X] Primitive types\n- [X] Array Types\n- [X] Union types\n- [X] References to other files\n- [X] Map Types\n- [ ] Logical Types (Currently just converts to primitive types)\n\nWriter\n\n- [X] Base Schema Writer\n- [X] Base Record Schema\n- [X] Base Enum Schema\n- [X] Primitive Types Schema\n- [X] Array Types Schema\n- [X] Union Types Schema\n- [X] Map Types\n- [ ] Logical Types Schema (Currently just converts to primitive types)\n- [X] Add configs to pip install package\n\nCLI\n\n- [X] Wrap Writer and Reader into one cli commmit\n- [X] Add pip install option (would include all files to pip install compiled package)\n- [ ] Add better --help documentation\n\nDocumentation\n\n- [ ] Document reader class\n- [ ] Document writer class\n- [ ] Document cli\n\n\n=======\nHistory\n=======\n\n0.3.3 (2022-01-26)\n^^^^^^^^^^^^^^^^^^\n\n* Fix array reference handling.\n\n\n0.3.2 (2021-11-16)\n^^^^^^^^^^^^^^^^^^\n\n* Add map type to unions.\n\n\n0.3.1 (2020-06-10)\n^^^^^^^^^^^^^^^^^^\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "avro-to-python is a light tool for compiling avro schema files (.avsc) to python classes making using avro schemata easy.",
"version": "0.3.3.post8",
"project_urls": {
"Homepage": "https://github.com/LeComptoirDesPharmacies/avro-to-python"
},
"split_keywords": [
"avro-to-python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "63ffadd948b752047a98ad7dddfbc2cb861e788e68ab3e518be72536dc2370f1",
"md5": "0087272a4a84b84fe59d4914fdcade6e",
"sha256": "c4fe39dd05ade695b7f3c16d43befeac0fa17bee40e04928d42da70df968d0c3"
},
"downloads": -1,
"filename": "lcdp_avro_to_python-0.3.3.post8-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "0087272a4a84b84fe59d4914fdcade6e",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>3.5",
"size": 33497,
"upload_time": "2024-11-22T15:02:17",
"upload_time_iso_8601": "2024-11-22T15:02:17.087485Z",
"url": "https://files.pythonhosted.org/packages/63/ff/add948b752047a98ad7dddfbc2cb861e788e68ab3e518be72536dc2370f1/lcdp_avro_to_python-0.3.3.post8-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f2289d07c4509eea134c3fa57746c8a9b8b4815a4166e3db7b8984634d7390b2",
"md5": "08285ad4c967151e52fcc580a78fcbff",
"sha256": "132a36a76a6a6f23989d731482ae2446dae740568a31bcc1c71cd6fb148e9029"
},
"downloads": -1,
"filename": "lcdp_avro_to_python-0.3.3.post8.tar.gz",
"has_sig": false,
"md5_digest": "08285ad4c967151e52fcc580a78fcbff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>3.5",
"size": 37804,
"upload_time": "2024-11-22T15:02:18",
"upload_time_iso_8601": "2024-11-22T15:02:18.459238Z",
"url": "https://files.pythonhosted.org/packages/f2/28/9d07c4509eea134c3fa57746c8a9b8b4815a4166e3db7b8984634d7390b2/lcdp_avro_to_python-0.3.3.post8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 15:02:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "LeComptoirDesPharmacies",
"github_project": "avro-to-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "Jinja2",
"specs": [
[
"==",
"3.1.2"
]
]
},
{
"name": "MarkupSafe",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "Click",
"specs": [
[
"==",
"8.0.4"
]
]
}
],
"tox": true,
"lcname": "lcdp-avro-to-python"
}
JSON
