## This is a fork of [betterproto](https://github.com/danielgtaylor/python-betterproto)
- Fixed input types imports missing
- Methods return non-string type hints
- Server methods now receive metadata
- Server-side exceptions get passed to client with original error message
- Removed Pydantic
- Support only `python3.10+`
- Seamless await for coroutines (see below)
# Seamless await for coroutines
Usually you write code like this:
```python
from package import Stub
stub = Stub(channel=channel)
await stub.cool_method()
async for res in stub.cool_stream():
pass
```
However, due to my personal needs, I added ability to seamlessly de-async coroutines and async generators.
```python
from bananaproto.event_loop import EventLoop, SingletonEventLoop
from package import Stub
# Creates a separate thread with loop running it in
loop = EventLoop().get_loop()
# Creates a single separate thread for the whole program execution with loop running it in
loop = SingletonEventLoop().get_loop()
# Any additional SingletonEventLoop().get_loop() would just return that same event loop
stub = Stub(channel=channel, synchronization_loop=loop)
# They now act as regular methods
stub.cool_method()
for res in stub.cool_stream():
pass
```
# BANANA Protobuf / gRPC Support for Python
[](https://www.python.org/downloads/)
[](https://github.com/BananaLoaf/python-bananaproto/blob/master/LICENSE)
[](https://pypi.org/project/bananaproto/)
[](https://github.com/BananaLoaf/python-bananaproto/actions/workflows/tests.yaml)
> :octocat: If you're reading this on github, please be aware that it might mention unreleased features! See the latest released README on [pypi](https://pypi.org/project/bananaproto/).
This project aims to provide an improved experience when using Protobuf / gRPC in a modern Python environment by making use of modern language features and generating readable, understandable, idiomatic Python code. It will not support legacy features or environments (e.g. Protobuf 2). The following are supported:
- Protobuf 3 & gRPC code generation
- Both binary & JSON serialization is built-in
- Python 3.7+ making use of:
- Enums
- Dataclasses
- `async`/`await`
- Timezone-aware `datetime` and `timedelta` objects
- Relative imports
- Mypy type checking
This project is heavily inspired by, and borrows functionality from:
- https://github.com/protocolbuffers/protobuf/tree/master/python
- https://github.com/eigenein/protobuf/
- https://github.com/vmagamedov/grpclib
## Motivation
This project exists because I am unhappy with the state of the official Google protoc plugin for Python.
- No `async` support (requires additional `grpclib` plugin)
- No typing support or code completion/intelligence (requires additional `mypy` plugin)
- No `__init__.py` module files get generated
- Output is not importable
- Import paths break in Python 3 unless you mess with `sys.path`
- Bugs when names clash (e.g. `codecs` package)
- Generated code is not idiomatic
- Completely unreadable runtime code-generation
- Much code looks like C++ or Java ported 1:1 to Python
- Capitalized function names like `HasField()` and `SerializeToString()`
- Uses `SerializeToString()` rather than the built-in `__bytes__()`
- Special wrapped types don't use Python's `None`
- Timestamp/duration types don't use Python's built-in `datetime` module
This project is a reimplementation from the ground up focused on idiomatic modern Python to help fix some of the above. While it may not be a 1:1 drop-in replacement due to changed method names and call patterns, the wire format is identical.
## Installation
First, install the package. Note that the `[compiler]` feature flag tells it to install extra dependencies only needed by the `protoc` plugin:
```sh
# Install both the library and compiler
pip install "bananaproto[compiler]"
# Install just the library (to use the generated code output)
pip install bananaproto
```
*Bananaproto* is under active development. To install the latest beta version, use `pip install --pre bananaproto`.
## Getting Started
### Compiling proto files
Given you installed the compiler and have a proto file, e.g `example.proto`:
```protobuf
syntax = "proto3";
package hello;
// Greeting represents a message you can tell a user.
message Greeting {
string message = 1;
}
```
You can run the following to invoke protoc directly:
```sh
mkdir lib
protoc -I . --python_bananaproto_out=lib example.proto
```
or run the following to invoke protoc via grpcio-tools:
```sh
pip install grpcio-tools
python -m grpc_tools.protoc -I . --python_bananaproto_out=lib example.proto
```
This will generate `lib/hello/__init__.py` which looks like:
```python
# Generated by the protocol buffer compiler. DO NOT EDIT!
# sources: example.proto
# plugin: python-bananaproto
from dataclasses import dataclass
import bananaproto
@dataclass
class Greeting(bananaproto.Message):
"""Greeting represents a message you can tell a user."""
message: str = bananaproto.string_field(1)
```
Now you can use it!
```python
>>> from lib.hello import Greeting
>>> test = Greeting()
>>> test
Greeting(message='')
>>> test.message = "Hey!"
>>> test
Greeting(message="Hey!")
>>> serialized = bytes(test)
>>> serialized
b'\n\x04Hey!'
>>> another = Greeting().parse(serialized)
>>> another
Greeting(message="Hey!")
>>> another.to_dict()
{"message": "Hey!"}
>>> another.to_json(indent=2)
'{\n "message": "Hey!"\n}'
```
### Async gRPC Support
The generated Protobuf `Message` classes are compatible with [grpclib](https://github.com/vmagamedov/grpclib) so you are free to use it if you like. That said, this project also includes support for async gRPC stub generation with better static type checking and code completion support. It is enabled by default.
Given an example service definition:
```protobuf
syntax = "proto3";
package echo;
message EchoRequest {
string value = 1;
// Number of extra times to echo
uint32 extra_times = 2;
}
message EchoResponse {
repeated string values = 1;
}
message EchoStreamResponse {
string value = 1;
}
service Echo {
rpc Echo(EchoRequest) returns (EchoResponse);
rpc EchoStream(EchoRequest) returns (stream EchoStreamResponse);
}
```
Generate echo proto file:
```
python -m grpc_tools.protoc -I . --python_bananaproto_out=. echo.proto
```
A client can be implemented as follows:
```python
import asyncio
import echo
from grpclib.client import Channel
async def main():
channel = Channel(host="127.0.0.1", port=50051)
service = echo.EchoStub(channel)
response = await service.echo(echo.EchoRequest(value="hello", extra_times=1))
print(response)
async for response in service.echo_stream(echo.EchoRequest(value="hello", extra_times=1)):
print(response)
# don't forget to close the channel when done!
channel.close()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
```
which would output
```python
EchoResponse(values=['hello', 'hello'])
EchoStreamResponse(value='hello')
EchoStreamResponse(value='hello')
```
This project also produces server-facing stubs that can be used to implement a Python
gRPC server.
To use them, simply subclass the base class in the generated files and override the
service methods:
```python
import asyncio
from echo import EchoBase, EchoRequest, EchoResponse, EchoStreamResponse
from grpclib.server import Server
from typing import AsyncIterator
class EchoService(EchoBase):
async def echo(self, echo_request: "EchoRequest") -> "EchoResponse":
return EchoResponse([echo_request.value for _ in range(echo_request.extra_times)])
async def echo_stream(self, echo_request: "EchoRequest") -> AsyncIterator["EchoStreamResponse"]:
for _ in range(echo_request.extra_times):
yield EchoStreamResponse(echo_request.value)
async def main():
server = Server([EchoService()])
await server.start("127.0.0.1", 50051)
await server.wait_closed()
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
```
### JSON
Both serializing and parsing are supported to/from JSON and Python dictionaries using the following methods:
- Dicts: `Message().to_dict()`, `Message().from_dict(...)`
- JSON: `Message().to_json()`, `Message().from_json(...)`
For compatibility the default is to convert field names to `camelCase`. You can control this behavior by passing a casing value, e.g:
```python
MyMessage().to_dict(casing=bananaproto.Casing.SNAKE)
```
### Determining if a message was sent
Sometimes it is useful to be able to determine whether a message has been sent on the wire. This is how the Google wrapper types work to let you know whether a value is unset, set as the default (zero value), or set as something else, for example.
Use `bananaproto.serialized_on_wire(message)` to determine if it was sent. This is a little bit different from the official Google generated Python code, and it lives outside the generated `Message` class to prevent name clashes. Note that it **only** supports Proto 3 and thus can **only** be used to check if `Message` fields are set. You cannot check if a scalar was sent on the wire.
```py
# Old way (official Google Protobuf package)
>>> mymessage.HasField('myfield')
# New way (this project)
>>> bananaproto.serialized_on_wire(mymessage.myfield)
```
### One-of Support
Protobuf supports grouping fields in a `oneof` clause. Only one of the fields in the group may be set at a given time. For example, given the proto:
```protobuf
syntax = "proto3";
message Test {
oneof foo {
bool on = 1;
int32 count = 2;
string name = 3;
}
}
```
You can use `bananaproto.which_one_of(message, group_name)` to determine which of the fields was set. It returns a tuple of the field name and value, or a blank string and `None` if unset.
```py
>>> test = Test()
>>> bananaproto.which_one_of(test, "foo")
["", None]
>>> test.on = True
>>> bananaproto.which_one_of(test, "foo")
["on", True]
# Setting one member of the group resets the others.
>>> test.count = 57
>>> bananaproto.which_one_of(test, "foo")
["count", 57]
>>> test.on
False
# Default (zero) values also work.
>>> test.name = ""
>>> bananaproto.which_one_of(test, "foo")
["name", ""]
>>> test.count
0
>>> test.on
False
```
Again this is a little different than the official Google code generator:
```py
# Old way (official Google protobuf package)
>>> message.WhichOneof("group")
"foo"
# New way (this project)
>>> bananaproto.which_one_of(message, "group")
["foo", "foo's value"]
```
### Well-Known Google Types
Google provides several well-known message types like a timestamp, duration, and several wrappers used to provide optional zero value support. Each of these has a special JSON representation and is handled a little differently from normal messages. The Python mapping for these is as follows:
| Google Message | Python Type | Default |
| --------------------------- | ---------------------------------------- | ---------------------- |
| `google.protobuf.duration` | [`datetime.timedelta`][td] | `0` |
| `google.protobuf.timestamp` | Timezone-aware [`datetime.datetime`][dt] | `1970-01-01T00:00:00Z` |
| `google.protobuf.*Value` | `Optional[...]` | `None` |
| `google.protobuf.*` | `bananaproto.lib.google.protobuf.*` | `None` |
[td]: https://docs.python.org/3/library/datetime.html#timedelta-objects
[dt]: https://docs.python.org/3/library/datetime.html#datetime.datetime
For the wrapper types, the Python type corresponds to the wrapped type, e.g. `google.protobuf.BoolValue` becomes `Optional[bool]` while `google.protobuf.Int32Value` becomes `Optional[int]`. All of the optional values default to `None`, so don't forget to check for that possible state. Given:
```protobuf
syntax = "proto3";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
message Test {
google.protobuf.BoolValue maybe = 1;
google.protobuf.Timestamp ts = 2;
google.protobuf.Duration duration = 3;
}
```
You can do stuff like:
```py
>>> t = Test().from_dict({"maybe": True, "ts": "2019-01-01T12:00:00Z", "duration": "1.200s"})
>>> t
Test(maybe=True, ts=datetime.datetime(2019, 1, 1, 12, 0, tzinfo=datetime.timezone.utc), duration=datetime.timedelta(seconds=1, microseconds=200000))
>>> t.ts - t.duration
datetime.datetime(2019, 1, 1, 11, 59, 58, 800000, tzinfo=datetime.timezone.utc)
>>> t.ts.isoformat()
'2019-01-01T12:00:00+00:00'
>>> t.maybe = None
>>> t.to_dict()
{'ts': '2019-01-01T12:00:00Z', 'duration': '1.200s'}
```
## Development
- _See how you can help → [Contributing](.github/CONTRIBUTING.md)_
### Requirements
- Python (3.7 or higher)
- [poetry](https://python-poetry.org/docs/#installation)
*Needed to install dependencies in a virtual environment*
- [poethepoet](https://github.com/nat-n/poethepoet) for running development tasks as defined in pyproject.toml
- Can be installed to your host environment via `pip install poethepoet` then executed as simple `poe`
- or run from the poetry venv as `poetry run poe`
### Setup
```sh
# Get set up with the virtual env & dependencies
poetry install -E compiler
# Activate the poetry environment
poetry shell
```
### Code style
This project enforces [black](https://github.com/psf/black) python code formatting.
Before committing changes run:
```sh
poe format
```
To avoid merge conflicts later, non-black formatted python code will fail in CI.
### Tests
There are two types of tests:
1. Standard tests
2. Custom tests
#### Standard tests
Adding a standard test case is easy.
- Create a new directory `bananaproto/tests/inputs/<name>`
- add `<name>.proto` with a message called `Test`
- add `<name>.json` with some test data (optional)
It will be picked up automatically when you run the tests.
- See also: [Standard Tests Development Guide](tests/README.md)
#### Custom tests
Custom tests are found in `tests/test_*.py` and are run with pytest.
#### Running
Here's how to run the tests.
```sh
# Generate assets from sample .proto files required by the tests
poe generate
# Run the tests
poe test
```
To run tests as they are run in CI (with tox) run:
```sh
poe full-test
```
### (Re)compiling Google Well-known Types
Bananaproto includes compiled versions for Google's well-known types at [src/bananaproto/lib/google](src/bananaproto/lib/google).
Be sure to regenerate these files when modifying the plugin output format, and validate by running the tests.
Normally, the plugin does not compile any references to `google.protobuf`, since they are pre-compiled. To force compilation of `google.protobuf`, use the option `--custom_opt=INCLUDE_GOOGLE`.
Assuming your `google.protobuf` source files (included with all releases of `protoc`) are located in `/usr/local/include`, you can regenerate them as follows:
```sh
protoc \
--plugin=protoc-gen-custom=src/bananaproto/plugin/main.py \
--custom_opt=INCLUDE_GOOGLE \
--custom_out=src/bananaproto/lib \
-I /usr/local/include/ \
/usr/local/include/google/protobuf/*.proto
```
### TODO
- [x] Fixed length fields
- [x] Packed fixed-length
- [x] Zig-zag signed fields (sint32, sint64)
- [x] Don't encode zero values for nested types
- [x] Enums
- [x] Repeated message fields
- [x] Maps
- [x] Maps of message fields
- [x] Support passthrough of unknown fields
- [x] Refs to nested types
- [x] Imports in proto files
- [x] Well-known Google types
- [ ] Support as request input
- [ ] Support as response output
- [ ] Automatically wrap/unwrap responses
- [x] OneOf support
- [x] Basic support on the wire
- [x] Check which was set from the group
- [x] Setting one unsets the others
- [ ] JSON that isn't completely naive.
- [x] 64-bit ints as strings
- [x] Maps
- [x] Lists
- [x] Bytes as base64
- [ ] Any support
- [x] Enum strings
- [x] Well known types support (timestamp, duration, wrappers)
- [x] Support different casing (orig vs. camel vs. others?)
- [x] Async service stubs
- [x] Unary-unary
- [x] Server streaming response
- [x] Client streaming request
- [x] Renaming messages and fields to conform to Python name standards
- [x] Renaming clashes with language keywords
- [x] Python package
- [x] Automate running tests
- [ ] Cleanup!
### Banana TODO
- [ ] Add *Reflections* chapter to README
- [ ] Return original folder structure
- [ ] Automatically import *_pb2 for reflections
- [ ] Fix comments in generated code
- [ ] Omit Empty argument from Stub and Service
## License
Copyright © 2019 Daniel G. Taylor
http://dgt.mit-license.org/
Raw data
{
"_id": null,
"home_page": "https://github.com/BananaLoaf/python-bananaproto",
"name": "bananaproto",
"maintainer": "BananaLoaf",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "bananaloaf@protonmail.com",
"keywords": "protobuf,gRPC",
"author": "Daniel G. Taylor",
"author_email": "danielgtaylor@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/eb/a4/4993e15e0ff02ff072d5b89e52ef7e3d22fad32b3d75af652f661a49ca94/bananaproto-1.3.0.tar.gz",
"platform": null,
"description": "## This is a fork of [betterproto](https://github.com/danielgtaylor/python-betterproto)\n\n- Fixed input types imports missing\n- Methods return non-string type hints\n- Server methods now receive metadata\n- Server-side exceptions get passed to client with original error message\n- Removed Pydantic\n- Support only `python3.10+`\n- Seamless await for coroutines (see below)\n\n# Seamless await for coroutines\n\nUsually you write code like this:\n\n```python\nfrom package import Stub\n\nstub = Stub(channel=channel)\n\nawait stub.cool_method()\n\nasync for res in stub.cool_stream():\n pass\n```\n\nHowever, due to my personal needs, I added ability to seamlessly de-async coroutines and async generators.\n\n```python\nfrom bananaproto.event_loop import EventLoop, SingletonEventLoop\nfrom package import Stub\n\n# Creates a separate thread with loop running it in\nloop = EventLoop().get_loop()\n\n# Creates a single separate thread for the whole program execution with loop running it in\nloop = SingletonEventLoop().get_loop()\n# Any additional SingletonEventLoop().get_loop() would just return that same event loop\n\nstub = Stub(channel=channel, synchronization_loop=loop)\n\n# They now act as regular methods\nstub.cool_method()\nfor res in stub.cool_stream():\n pass\n```\n\n# BANANA Protobuf / gRPC Support for Python\n\n[](https://www.python.org/downloads/)\n[](https://github.com/BananaLoaf/python-bananaproto/blob/master/LICENSE)\n[](https://pypi.org/project/bananaproto/)\n[](https://github.com/BananaLoaf/python-bananaproto/actions/workflows/tests.yaml)\n> :octocat: If you're reading this on github, please be aware that it might mention unreleased features! See the latest released README on [pypi](https://pypi.org/project/bananaproto/).\n\nThis project aims to provide an improved experience when using Protobuf / gRPC in a modern Python environment by making use of modern language features and generating readable, understandable, idiomatic Python code. It will not support legacy features or environments (e.g. Protobuf 2). The following are supported:\n\n- Protobuf 3 & gRPC code generation\n - Both binary & JSON serialization is built-in\n- Python 3.7+ making use of:\n - Enums\n - Dataclasses\n - `async`/`await`\n - Timezone-aware `datetime` and `timedelta` objects\n - Relative imports\n - Mypy type checking\n\nThis project is heavily inspired by, and borrows functionality from:\n\n- https://github.com/protocolbuffers/protobuf/tree/master/python\n- https://github.com/eigenein/protobuf/\n- https://github.com/vmagamedov/grpclib\n\n## Motivation\n\nThis project exists because I am unhappy with the state of the official Google protoc plugin for Python.\n\n- No `async` support (requires additional `grpclib` plugin)\n- No typing support or code completion/intelligence (requires additional `mypy` plugin)\n- No `__init__.py` module files get generated\n- Output is not importable\n - Import paths break in Python 3 unless you mess with `sys.path`\n- Bugs when names clash (e.g. `codecs` package)\n- Generated code is not idiomatic\n - Completely unreadable runtime code-generation\n - Much code looks like C++ or Java ported 1:1 to Python\n - Capitalized function names like `HasField()` and `SerializeToString()`\n - Uses `SerializeToString()` rather than the built-in `__bytes__()`\n - Special wrapped types don't use Python's `None`\n - Timestamp/duration types don't use Python's built-in `datetime` module\n\n\nThis project is a reimplementation from the ground up focused on idiomatic modern Python to help fix some of the above. While it may not be a 1:1 drop-in replacement due to changed method names and call patterns, the wire format is identical.\n\n## Installation\n\nFirst, install the package. Note that the `[compiler]` feature flag tells it to install extra dependencies only needed by the `protoc` plugin:\n\n```sh\n# Install both the library and compiler\npip install \"bananaproto[compiler]\"\n\n# Install just the library (to use the generated code output)\npip install bananaproto\n```\n\n*Bananaproto* is under active development. To install the latest beta version, use `pip install --pre bananaproto`.\n\n## Getting Started\n\n### Compiling proto files\n\nGiven you installed the compiler and have a proto file, e.g `example.proto`:\n\n```protobuf\nsyntax = \"proto3\";\n\npackage hello;\n\n// Greeting represents a message you can tell a user.\nmessage Greeting {\n string message = 1;\n}\n```\n\nYou can run the following to invoke protoc directly:\n\n```sh\nmkdir lib\nprotoc -I . --python_bananaproto_out=lib example.proto\n```\n\nor run the following to invoke protoc via grpcio-tools:\n\n```sh\npip install grpcio-tools\npython -m grpc_tools.protoc -I . --python_bananaproto_out=lib example.proto\n```\n\nThis will generate `lib/hello/__init__.py` which looks like:\n\n```python\n# Generated by the protocol buffer compiler. DO NOT EDIT!\n# sources: example.proto\n# plugin: python-bananaproto\nfrom dataclasses import dataclass\n\nimport bananaproto\n\n\n@dataclass\nclass Greeting(bananaproto.Message):\n \"\"\"Greeting represents a message you can tell a user.\"\"\"\n\n message: str = bananaproto.string_field(1)\n```\n\nNow you can use it!\n\n```python\n>>> from lib.hello import Greeting\n>>> test = Greeting()\n>>> test\nGreeting(message='')\n\n>>> test.message = \"Hey!\"\n>>> test\nGreeting(message=\"Hey!\")\n\n>>> serialized = bytes(test)\n>>> serialized\nb'\\n\\x04Hey!'\n\n>>> another = Greeting().parse(serialized)\n>>> another\nGreeting(message=\"Hey!\")\n\n>>> another.to_dict()\n{\"message\": \"Hey!\"}\n>>> another.to_json(indent=2)\n'{\\n \"message\": \"Hey!\"\\n}'\n```\n\n### Async gRPC Support\n\nThe generated Protobuf `Message` classes are compatible with [grpclib](https://github.com/vmagamedov/grpclib) so you are free to use it if you like. That said, this project also includes support for async gRPC stub generation with better static type checking and code completion support. It is enabled by default.\n\nGiven an example service definition:\n\n```protobuf\nsyntax = \"proto3\";\n\npackage echo;\n\nmessage EchoRequest {\n string value = 1;\n // Number of extra times to echo\n uint32 extra_times = 2;\n}\n\nmessage EchoResponse {\n repeated string values = 1;\n}\n\nmessage EchoStreamResponse {\n string value = 1;\n}\n\nservice Echo {\n rpc Echo(EchoRequest) returns (EchoResponse);\n rpc EchoStream(EchoRequest) returns (stream EchoStreamResponse);\n}\n```\n\nGenerate echo proto file:\n\n```\npython -m grpc_tools.protoc -I . --python_bananaproto_out=. echo.proto\n```\n\nA client can be implemented as follows:\n```python\nimport asyncio\nimport echo\n\nfrom grpclib.client import Channel\n\n\nasync def main():\n channel = Channel(host=\"127.0.0.1\", port=50051)\n service = echo.EchoStub(channel)\n response = await service.echo(echo.EchoRequest(value=\"hello\", extra_times=1))\n print(response)\n\n async for response in service.echo_stream(echo.EchoRequest(value=\"hello\", extra_times=1)):\n print(response)\n\n # don't forget to close the channel when done!\n channel.close()\n\n\nif __name__ == \"__main__\":\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n\n```\n\nwhich would output\n```python\nEchoResponse(values=['hello', 'hello'])\nEchoStreamResponse(value='hello')\nEchoStreamResponse(value='hello')\n```\n\nThis project also produces server-facing stubs that can be used to implement a Python\ngRPC server.\nTo use them, simply subclass the base class in the generated files and override the\nservice methods:\n\n```python\nimport asyncio\nfrom echo import EchoBase, EchoRequest, EchoResponse, EchoStreamResponse\nfrom grpclib.server import Server\nfrom typing import AsyncIterator\n\n\nclass EchoService(EchoBase):\n async def echo(self, echo_request: \"EchoRequest\") -> \"EchoResponse\":\n return EchoResponse([echo_request.value for _ in range(echo_request.extra_times)])\n\n async def echo_stream(self, echo_request: \"EchoRequest\") -> AsyncIterator[\"EchoStreamResponse\"]:\n for _ in range(echo_request.extra_times):\n yield EchoStreamResponse(echo_request.value)\n\n\nasync def main():\n server = Server([EchoService()])\n await server.start(\"127.0.0.1\", 50051)\n await server.wait_closed()\n\nif __name__ == '__main__':\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n```\n\n### JSON\n\nBoth serializing and parsing are supported to/from JSON and Python dictionaries using the following methods:\n\n- Dicts: `Message().to_dict()`, `Message().from_dict(...)`\n- JSON: `Message().to_json()`, `Message().from_json(...)`\n\nFor compatibility the default is to convert field names to `camelCase`. You can control this behavior by passing a casing value, e.g:\n\n```python\nMyMessage().to_dict(casing=bananaproto.Casing.SNAKE)\n```\n\n### Determining if a message was sent\n\nSometimes it is useful to be able to determine whether a message has been sent on the wire. This is how the Google wrapper types work to let you know whether a value is unset, set as the default (zero value), or set as something else, for example.\n\nUse `bananaproto.serialized_on_wire(message)` to determine if it was sent. This is a little bit different from the official Google generated Python code, and it lives outside the generated `Message` class to prevent name clashes. Note that it **only** supports Proto 3 and thus can **only** be used to check if `Message` fields are set. You cannot check if a scalar was sent on the wire.\n\n```py\n# Old way (official Google Protobuf package)\n>>> mymessage.HasField('myfield')\n\n# New way (this project)\n>>> bananaproto.serialized_on_wire(mymessage.myfield)\n```\n\n### One-of Support\n\nProtobuf supports grouping fields in a `oneof` clause. Only one of the fields in the group may be set at a given time. For example, given the proto:\n\n```protobuf\nsyntax = \"proto3\";\n\nmessage Test {\n oneof foo {\n bool on = 1;\n int32 count = 2;\n string name = 3;\n }\n}\n```\n\nYou can use `bananaproto.which_one_of(message, group_name)` to determine which of the fields was set. It returns a tuple of the field name and value, or a blank string and `None` if unset.\n\n```py\n>>> test = Test()\n>>> bananaproto.which_one_of(test, \"foo\")\n[\"\", None]\n\n>>> test.on = True\n>>> bananaproto.which_one_of(test, \"foo\")\n[\"on\", True]\n\n# Setting one member of the group resets the others.\n>>> test.count = 57\n>>> bananaproto.which_one_of(test, \"foo\")\n[\"count\", 57]\n>>> test.on\nFalse\n\n# Default (zero) values also work.\n>>> test.name = \"\"\n>>> bananaproto.which_one_of(test, \"foo\")\n[\"name\", \"\"]\n>>> test.count\n0\n>>> test.on\nFalse\n```\n\nAgain this is a little different than the official Google code generator:\n\n```py\n# Old way (official Google protobuf package)\n>>> message.WhichOneof(\"group\")\n\"foo\"\n\n# New way (this project)\n>>> bananaproto.which_one_of(message, \"group\")\n[\"foo\", \"foo's value\"]\n```\n\n### Well-Known Google Types\n\nGoogle provides several well-known message types like a timestamp, duration, and several wrappers used to provide optional zero value support. Each of these has a special JSON representation and is handled a little differently from normal messages. The Python mapping for these is as follows:\n\n| Google Message | Python Type | Default |\n| --------------------------- | ---------------------------------------- | ---------------------- |\n| `google.protobuf.duration` | [`datetime.timedelta`][td] | `0` |\n| `google.protobuf.timestamp` | Timezone-aware [`datetime.datetime`][dt] | `1970-01-01T00:00:00Z` |\n| `google.protobuf.*Value` | `Optional[...]` | `None` |\n| `google.protobuf.*` | `bananaproto.lib.google.protobuf.*` | `None` |\n\n[td]: https://docs.python.org/3/library/datetime.html#timedelta-objects\n[dt]: https://docs.python.org/3/library/datetime.html#datetime.datetime\n\nFor the wrapper types, the Python type corresponds to the wrapped type, e.g. `google.protobuf.BoolValue` becomes `Optional[bool]` while `google.protobuf.Int32Value` becomes `Optional[int]`. All of the optional values default to `None`, so don't forget to check for that possible state. Given:\n\n```protobuf\nsyntax = \"proto3\";\n\nimport \"google/protobuf/duration.proto\";\nimport \"google/protobuf/timestamp.proto\";\nimport \"google/protobuf/wrappers.proto\";\n\nmessage Test {\n google.protobuf.BoolValue maybe = 1;\n google.protobuf.Timestamp ts = 2;\n google.protobuf.Duration duration = 3;\n}\n```\n\nYou can do stuff like:\n\n```py\n>>> t = Test().from_dict({\"maybe\": True, \"ts\": \"2019-01-01T12:00:00Z\", \"duration\": \"1.200s\"})\n>>> t\nTest(maybe=True, ts=datetime.datetime(2019, 1, 1, 12, 0, tzinfo=datetime.timezone.utc), duration=datetime.timedelta(seconds=1, microseconds=200000))\n\n>>> t.ts - t.duration\ndatetime.datetime(2019, 1, 1, 11, 59, 58, 800000, tzinfo=datetime.timezone.utc)\n\n>>> t.ts.isoformat()\n'2019-01-01T12:00:00+00:00'\n\n>>> t.maybe = None\n>>> t.to_dict()\n{'ts': '2019-01-01T12:00:00Z', 'duration': '1.200s'}\n```\n\n\n## Development\n\n- _See how you can help → [Contributing](.github/CONTRIBUTING.md)_\n\n### Requirements\n\n- Python (3.7 or higher)\n\n- [poetry](https://python-poetry.org/docs/#installation)\n *Needed to install dependencies in a virtual environment*\n\n- [poethepoet](https://github.com/nat-n/poethepoet) for running development tasks as defined in pyproject.toml\n - Can be installed to your host environment via `pip install poethepoet` then executed as simple `poe`\n - or run from the poetry venv as `poetry run poe`\n\n### Setup\n\n```sh\n# Get set up with the virtual env & dependencies\npoetry install -E compiler\n\n# Activate the poetry environment\npoetry shell\n```\n\n### Code style\n\nThis project enforces [black](https://github.com/psf/black) python code formatting.\n\nBefore committing changes run:\n\n```sh\npoe format\n```\n\nTo avoid merge conflicts later, non-black formatted python code will fail in CI.\n\n### Tests\n\nThere are two types of tests:\n\n1. Standard tests\n2. Custom tests\n\n#### Standard tests\n\nAdding a standard test case is easy.\n\n- Create a new directory `bananaproto/tests/inputs/<name>`\n - add `<name>.proto` with a message called `Test`\n - add `<name>.json` with some test data (optional)\n\nIt will be picked up automatically when you run the tests.\n\n- See also: [Standard Tests Development Guide](tests/README.md)\n\n#### Custom tests\n\nCustom tests are found in `tests/test_*.py` and are run with pytest.\n\n#### Running\n\nHere's how to run the tests.\n\n```sh\n# Generate assets from sample .proto files required by the tests\npoe generate\n# Run the tests\npoe test\n```\n\nTo run tests as they are run in CI (with tox) run:\n\n```sh\npoe full-test\n```\n\n### (Re)compiling Google Well-known Types\n\nBananaproto includes compiled versions for Google's well-known types at [src/bananaproto/lib/google](src/bananaproto/lib/google).\nBe sure to regenerate these files when modifying the plugin output format, and validate by running the tests.\n\nNormally, the plugin does not compile any references to `google.protobuf`, since they are pre-compiled. To force compilation of `google.protobuf`, use the option `--custom_opt=INCLUDE_GOOGLE`.\n\nAssuming your `google.protobuf` source files (included with all releases of `protoc`) are located in `/usr/local/include`, you can regenerate them as follows:\n\n```sh\nprotoc \\\n --plugin=protoc-gen-custom=src/bananaproto/plugin/main.py \\\n --custom_opt=INCLUDE_GOOGLE \\\n --custom_out=src/bananaproto/lib \\\n -I /usr/local/include/ \\\n /usr/local/include/google/protobuf/*.proto\n```\n\n### TODO\n\n- [x] Fixed length fields\n - [x] Packed fixed-length\n- [x] Zig-zag signed fields (sint32, sint64)\n- [x] Don't encode zero values for nested types\n- [x] Enums\n- [x] Repeated message fields\n- [x] Maps\n - [x] Maps of message fields\n- [x] Support passthrough of unknown fields\n- [x] Refs to nested types\n- [x] Imports in proto files\n- [x] Well-known Google types\n - [ ] Support as request input\n - [ ] Support as response output\n - [ ] Automatically wrap/unwrap responses\n- [x] OneOf support\n - [x] Basic support on the wire\n - [x] Check which was set from the group\n - [x] Setting one unsets the others\n- [ ] JSON that isn't completely naive.\n - [x] 64-bit ints as strings\n - [x] Maps\n - [x] Lists\n - [x] Bytes as base64\n - [ ] Any support\n - [x] Enum strings\n - [x] Well known types support (timestamp, duration, wrappers)\n - [x] Support different casing (orig vs. camel vs. others?)\n- [x] Async service stubs\n - [x] Unary-unary\n - [x] Server streaming response\n - [x] Client streaming request\n- [x] Renaming messages and fields to conform to Python name standards\n- [x] Renaming clashes with language keywords\n- [x] Python package\n- [x] Automate running tests\n- [ ] Cleanup!\n\n### Banana TODO\n\n- [ ] Add *Reflections* chapter to README\n- [ ] Return original folder structure\n- [ ] Automatically import *_pb2 for reflections\n- [ ] Fix comments in generated code\n- [ ] Omit Empty argument from Stub and Service\n\n## License\n\nCopyright \u00a9 2019 Daniel G. Taylor\n\nhttp://dgt.mit-license.org/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A BANANA Protobuf / gRPC generator & library",
"version": "1.3.0",
"project_urls": {
"Homepage": "https://github.com/BananaLoaf/python-bananaproto",
"Repository": "https://github.com/BananaLoaf/python-bananaproto"
},
"split_keywords": [
"protobuf",
"grpc"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c5da0808ae74e2c9dc47024c04177aafc076f55c5a4eee2130baf97f185a58c1",
"md5": "cc2ec6ebee8025736ba5f3b5bfa149ea",
"sha256": "7dfc17c05e4acf045c5b5477ece36b91481dbd80974df0527f8a6c99b5ffcf9f"
},
"downloads": -1,
"filename": "bananaproto-1.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc2ec6ebee8025736ba5f3b5bfa149ea",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 65029,
"upload_time": "2024-03-15T15:37:55",
"upload_time_iso_8601": "2024-03-15T15:37:55.101362Z",
"url": "https://files.pythonhosted.org/packages/c5/da/0808ae74e2c9dc47024c04177aafc076f55c5a4eee2130baf97f185a58c1/bananaproto-1.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "eba44993e15e0ff02ff072d5b89e52ef7e3d22fad32b3d75af652f661a49ca94",
"md5": "7f1af71d39586f820ad5f95432f9a07c",
"sha256": "9701c7a59558af57fd6b0639d5c64a6f54a5591ceb8de25be09f8ef76b0e2ec7"
},
"downloads": -1,
"filename": "bananaproto-1.3.0.tar.gz",
"has_sig": false,
"md5_digest": "7f1af71d39586f820ad5f95432f9a07c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 66994,
"upload_time": "2024-03-15T15:37:56",
"upload_time_iso_8601": "2024-03-15T15:37:56.423316Z",
"url": "https://files.pythonhosted.org/packages/eb/a4/4993e15e0ff02ff072d5b89e52ef7e3d22fad32b3d75af652f661a49ca94/bananaproto-1.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-15 15:37:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "BananaLoaf",
"github_project": "python-bananaproto",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bananaproto"
}
JSON
