# Myprotosql
A set of mysql stored functions/procedures to read protobuf binary data
[](https://github.com/janickr/myprotosql/actions/workflows/tests-mysql5_7.yml)
[](https://github.com/janickr/myprotosql/actions/workflows/tests-mysql8.yml)
[](https://pypi.org/project/myprotosql/)
## Getting started (with *.proto files)
See [decode using .proto files](https://github.com/janickr/myprotosql#decode-using-proto-files) for an example.
- [Download and install](https://github.com/protocolbuffers/protobuf?tab=readme-ov-file#protobuf-compiler-installation) protoc
- Install myprotosql (requires python):
```bash
pip install myprotosql
```
- Run protoc with the myprotosql plugin (your `*.proto` files located in `./proto`, output in `./build`):
```bash
protoc --proto_path=proto --myprotosql_out=build ./proto/*
```
- Run the generated `install_myproto.sql` and `myproto_descriptors.sql` scripts in MySQL
If you used [this proto file](https://github.com/janickr/myprotosql#decode-using-proto-files), you can now decode your first protobuf message
```mysql
select myproto_decode_to_textformat(
0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());
```
## Getting started (without *.proto files)
This is similar to `protoc --decode_raw`. See [decode raw](https://github.com/janickr/myprotosql#decode-raw) for an example.
- Install myprotosql (requires python):
```bash
pip install myprotosql
```
- Generate the install script
```bash
myprotosql-install-script > install_myproto.sql
```
- Run the generated `install_myproto.sql` script in MySQL
Decode your first protobuf message:
```mysql
select myproto_decode_to_textformat(0x1a03089601, null, null);
```
#### Alternative
Instead of using pip and python to install myprotosql, you can also just download the `install_myproto.sql` from the github repository and run that in MySQL.
## Decoding
Running `install_myproto.sql` installs two functions that can be used to decode protobuf binary messages:
- myproto_decode_to_textformat(binary_message, message_type, type_descriptors)
- myproto_decode_to_jsonformat(binary_message, message_type, type_descriptors)
### Decode raw
Decoding without the `*.proto` files
#### Textformat
decode_raw the `0x1a03089601` binary data to textformat:
```mysql
select myproto_decode_to_textformat(0x1a03089601, null, null);
```
Returns:
```prototext
3: {
1: 150
}
```
#### JSON
```mysql
select myproto_decode_to_jsonformat(0x1a03089601, null, null);
```
Returns:
```json
{"3": {"1": 150}}
```
### Decode using .proto files
Let's say we have a `.proto` file like this:
```protobuf
package foo.bar;
message SubMessage {
optional int32 a = 1;
}
message ParentMessage {
optional SubMessage c = 3;
}
```
Check out [Getting started (with *.proto files)](https://github.com/janickr/myprotosql#getting-started-with-proto-files) to compile these `*.proto` files in something MySQL can understand.
#### Textformat
For example to decode the `0x1a03089601` binary data to textformat:
```mysql
select myproto_decode_to_textformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());
```
Returns:
```prototext
c: {
a: 150
}
```
#### JSON
```mysql
select myproto_decode_to_jsonformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());
```
Returns:
```json
{"c": {"a": 150}}
```
## Troubleshooting
### on Windows if you used Virtualenv
you need to specify the full path to the myprotosql plugin, like this:
```bash
protoc.exe --proto_path=proto --myprotosql_out=build --plugin=protoc-gen-myprotosql=.\venv\Scripts\protoc-gen-myprotosql.exe .\proto\*
```
This assumed your proto files are located in `.\proto` and your virtual env path is `.\venv`. In general the command is of the form:
```bash
protoc --proto_path=<the-path-to-your-proto-files> --myprotosql_out=<the-output-path> --plugin=protoc-gen-myprotosql=<the-path-to-the-myprotosql-plugin> <the-path-to-your-proto-files>\*
```
### Limitations of decode_raw
Decode raw has limitations because protobuf binary data does not contain all info to properly decode the data.
- output will not contain field names, only field numbers
- packed repeated scalar values will be decoded as one binary string
- numbers will be decoded as unsigned integers
If you need proper decoding, then read on and learn how to use information in your `*.proto` files
## Uninstalling
- Generate the uninstall script
```bash
myprotosql-uninstall-script > uninstall_myproto.sql
```
- Run the generated `uninstall_myproto.sql` script in MySQL
#### Alternative
Download the `uninstall_myproto.sql` from the github repository and run that in MySQL.
## Todo
- todos in code
Raw data
{
"_id": null,
"home_page": "https://github.com/janickr/myprotosql",
"name": "myprotosql",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "protoc, protobuf, mysql",
"author": "Janick Reynders",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/8e/ec/045cb186fbb1775beb0fbadfdb7f5c4e21d66e585c836c107210e3942c28/myprotosql-0.0.8.tar.gz",
"platform": null,
"description": "# Myprotosql\n\nA set of mysql stored functions/procedures to read protobuf binary data \n\n[](https://github.com/janickr/myprotosql/actions/workflows/tests-mysql5_7.yml)\n[](https://github.com/janickr/myprotosql/actions/workflows/tests-mysql8.yml)\n[](https://pypi.org/project/myprotosql/)\n\n## Getting started (with *.proto files)\nSee [decode using .proto files](https://github.com/janickr/myprotosql#decode-using-proto-files) for an example. \n\n- [Download and install](https://github.com/protocolbuffers/protobuf?tab=readme-ov-file#protobuf-compiler-installation) protoc \n \n- Install myprotosql (requires python): \n\n ```bash\n pip install myprotosql\n ``` \n \n- Run protoc with the myprotosql plugin (your `*.proto` files located in `./proto`, output in `./build`): \n \n ```bash\n protoc --proto_path=proto --myprotosql_out=build ./proto/*\n ``` \n \n- Run the generated `install_myproto.sql` and `myproto_descriptors.sql` scripts in MySQL \n If you used [this proto file](https://github.com/janickr/myprotosql#decode-using-proto-files), you can now decode your first protobuf message \n \n ```mysql\n select myproto_decode_to_textformat(\n 0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());\n ```\n\n## Getting started (without *.proto files)\nThis is similar to `protoc --decode_raw`. See [decode raw](https://github.com/janickr/myprotosql#decode-raw) for an example.\n\n- Install myprotosql (requires python): \n \n ```bash\n pip install myprotosql\n ```\n- Generate the install script \n \n ```bash\n myprotosql-install-script > install_myproto.sql\n ``` \n- Run the generated `install_myproto.sql` script in MySQL \n Decode your first protobuf message:\n \n ```mysql\n select myproto_decode_to_textformat(0x1a03089601, null, null);\n ```\n#### Alternative\nInstead of using pip and python to install myprotosql, you can also just download the `install_myproto.sql` from the github repository and run that in MySQL.\n\n## Decoding\nRunning `install_myproto.sql` installs two functions that can be used to decode protobuf binary messages:\n- myproto_decode_to_textformat(binary_message, message_type, type_descriptors)\n- myproto_decode_to_jsonformat(binary_message, message_type, type_descriptors)\n\n### Decode raw\nDecoding without the `*.proto` files\n\n#### Textformat\ndecode_raw the `0x1a03089601` binary data to textformat:\n```mysql\nselect myproto_decode_to_textformat(0x1a03089601, null, null);\n```\nReturns:\n```prototext\n3: {\n 1: 150\n}\n```\n#### JSON\n```mysql\nselect myproto_decode_to_jsonformat(0x1a03089601, null, null);\n```\nReturns:\n```json\n{\"3\": {\"1\": 150}}\n```\n\n### Decode using .proto files\n\nLet's say we have a `.proto` file like this:\n```protobuf\npackage foo.bar;\n\nmessage SubMessage {\n optional int32 a = 1;\n}\n\nmessage ParentMessage {\n optional SubMessage c = 3;\n}\n```\nCheck out [Getting started (with *.proto files)](https://github.com/janickr/myprotosql#getting-started-with-proto-files) to compile these `*.proto` files in something MySQL can understand. \n\n#### Textformat\n\nFor example to decode the `0x1a03089601` binary data to textformat:\n```mysql\nselect myproto_decode_to_textformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());\n```\nReturns:\n```prototext\nc: {\n a: 150\n}\n```\n#### JSON\n```mysql\nselect myproto_decode_to_jsonformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());\n```\nReturns:\n```json\n{\"c\": {\"a\": 150}}\n```\n\n## Troubleshooting\n### on Windows if you used Virtualenv\nyou need to specify the full path to the myprotosql plugin, like this:\n```bash\nprotoc.exe --proto_path=proto --myprotosql_out=build --plugin=protoc-gen-myprotosql=.\\venv\\Scripts\\protoc-gen-myprotosql.exe .\\proto\\* \n```\nThis assumed your proto files are located in `.\\proto` and your virtual env path is `.\\venv`. In general the command is of the form:\n```bash\nprotoc --proto_path=<the-path-to-your-proto-files> --myprotosql_out=<the-output-path> --plugin=protoc-gen-myprotosql=<the-path-to-the-myprotosql-plugin> <the-path-to-your-proto-files>\\*\n```\n\n### Limitations of decode_raw\nDecode raw has limitations because protobuf binary data does not contain all info to properly decode the data.\n- output will not contain field names, only field numbers\n- packed repeated scalar values will be decoded as one binary string\n- numbers will be decoded as unsigned integers\n\nIf you need proper decoding, then read on and learn how to use information in your `*.proto` files\n\n## Uninstalling\n- Generate the uninstall script \n \n ```bash\n myprotosql-uninstall-script > uninstall_myproto.sql\n ``` \n- Run the generated `uninstall_myproto.sql` script in MySQL \n \n#### Alternative\nDownload the `uninstall_myproto.sql` from the github repository and run that in MySQL.\n\n\n## Todo\n- todos in code\n",
"bugtrack_url": null,
"license": "GNU Lesser General Public License v3 or later (LGPLv3+)",
"summary": "Read protobuf binary data using vanilla mysql stored functions",
"version": "0.0.8",
"project_urls": {
"Homepage": "https://github.com/janickr/myprotosql"
},
"split_keywords": [
"protoc",
" protobuf",
" mysql"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e9e090af2fbc00ac13eb889df1c1c77dcf123c8db5fd128130e25493ca5c2f2f",
"md5": "f82e40f4aa5b532e82dccb720e8559ec",
"sha256": "685e35972b9f377a191e67b69ba8fbe39e24f9b3f3980c5a2f46d61c10461405"
},
"downloads": -1,
"filename": "myprotosql-0.0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f82e40f4aa5b532e82dccb720e8559ec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 30353,
"upload_time": "2024-09-02T21:33:25",
"upload_time_iso_8601": "2024-09-02T21:33:25.937056Z",
"url": "https://files.pythonhosted.org/packages/e9/e0/90af2fbc00ac13eb889df1c1c77dcf123c8db5fd128130e25493ca5c2f2f/myprotosql-0.0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8eec045cb186fbb1775beb0fbadfdb7f5c4e21d66e585c836c107210e3942c28",
"md5": "abc1c372ed34edf27c71ce20cbf9b2cd",
"sha256": "7c27375572335eb0a0202c9fbcdacee386f156a794402ff6859370412763c36c"
},
"downloads": -1,
"filename": "myprotosql-0.0.8.tar.gz",
"has_sig": false,
"md5_digest": "abc1c372ed34edf27c71ce20cbf9b2cd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 39010,
"upload_time": "2024-09-02T21:33:26",
"upload_time_iso_8601": "2024-09-02T21:33:26.975187Z",
"url": "https://files.pythonhosted.org/packages/8e/ec/045cb186fbb1775beb0fbadfdb7f5c4e21d66e585c836c107210e3942c28/myprotosql-0.0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-02 21:33:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "janickr",
"github_project": "myprotosql",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "myprotosql"
}
JSON
