# Struct2Tensor
[](https://github.com/google/struct2tensor)
[](https://badge.fury.io/py/struct2tensor)
<!--*
freshness: { owner: 'iindyk' reviewed: '2024-04-22' }
*-->
## Introduction
struct2tensor is a library for parsing structured data inside of tensorflow.
In particular, it makes it easy to manipulate structured data, e.g., slicing,
flattening, copying substructures, and so on, as part of a TensorFlow model
graph. The notebook in 'examples/prensor_playground.ipynb' provides a few
examples of struct2tensor in action and an introduction to the main
concepts. You can
[run the notebook in your browser](https://colab.research.google.com/github/google/struct2tensor/blob/master/examples/prensor_playground.ipynb)
through Google's colab environment, or [download the
file](examples/prensor_playground.ipynb) to run it in your own Jupyter
environment.
There are two main use cases of this repo:
1. To create a PIP package. The PIP package contains plug-ins (OpKernels) to an
existing tensorflow installation.
2. To staticlly link with tensorflow-serving.
As these processes are independent, one can follow either set of directions
below.
## Use a pre-built Linux PIP package.
From a virtual environment, run:
```bash
pip install struct2tensor
```
### Nightly Packages
Struct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org
on Google Cloud. To install the latest nightly package, please use the following
command:
```bash
pip install --extra-index-url https://pypi-nightly.tensorflow.org/simple struct2tensor
```
This will install the nightly packages for the major dependencies of
struct2tensor such as TensorFlow Metadata (TFMD).
## Creating a PIP package.
The struct2tensor PIP package is useful for creating models.
It works with tensorflow 2.x.
In order to unify the process, we recommend compiling struct2tensor inside
a docker container.
### Downloading the Code
Go to your home directory.
Download the source code.
```bash
git clone https://github.com/google/struct2tensor.git
cd ~/struct2tensor
```
### Use docker-compose
Install [docker-compose](https://docs.docker.com/compose/).
Use it to build a pip wheel for Python 3.8 with tensorflow version 2:
```bash
docker-compose build --build-arg PYTHON_VERSION=3.8 manylinux2014
docker-compose run -e TF_VERSION=RELEASED_TF_2 manylinux2014
```
This will create a manylinux package in the ~/struct2tensor/dist directory.
## Creating a static library
In order to construct a static library for tensorflow-serving, we run:
```bash
bazel build -c opt struct2tensor:struct2tensor_kernels_and_ops
```
This can also be linked into another library.
## [TensorFlow Serving](https://github.com/tensorflow/serving) docker image
struct2tensor needs a couple of custom TensorFlow ops to function. If you train
a model with struct2tensor and wants to serve it with TensorFlow Serving, the
TensorFlow Serving binary needs to link with those custom ops. We have a
pre-built docker image that contains such a binary. The `Dockerfile` is
available at `tools/tf_serving_docker/Dockerfile`. The image is available at
`gcr.io/tfx-oss-public/s2t_tf_serving`.
Please see the `Dockerfile` for details. But in brief, the image exposes port
8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set
two environment variables `MODEL_BASE_PATH` and `MODEL_NAME` to point it to
your model (either mount it to the container, or put your model on GCS).
It will look for a saved model at
`${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}`, where `VERSION_NUMBER`
is an integer.
## Compatibility
struct2tensor | tensorflow
---------------------------------------------------------------------- | ----------
[0.47.0](https://github.com/google/struct2tensor/releases/tag/v0.47.0) | 2.16.2
[0.46.0](https://github.com/google/struct2tensor/releases/tag/v0.46.0) | 2.15.0
[0.45.0](https://github.com/google/struct2tensor/releases/tag/v0.45.0) | 2.13.0
[0.44.0](https://github.com/google/struct2tensor/releases/tag/v0.44.0) | 2.12.0
[0.43.0](https://github.com/google/struct2tensor/releases/tag/v0.43.0) | 2.11.0
[0.42.0](https://github.com/google/struct2tensor/releases/tag/v0.42.0) | 2.10.0
[0.41.0](https://github.com/google/struct2tensor/releases/tag/v0.41.0) | 2.9.0
[0.40.0](https://github.com/google/struct2tensor/releases/tag/v0.40.0) | 2.9.0
[0.39.0](https://github.com/google/struct2tensor/releases/tag/v0.39.0) | 2.8.0
[0.38.0](https://github.com/google/struct2tensor/releases/tag/v0.38.0) | 2.8.0
[0.37.0](https://github.com/google/struct2tensor/releases/tag/v0.37.0) | 2.7.0
[0.36.0](https://github.com/google/struct2tensor/releases/tag/v0.36.0) | 2.7.0
[0.35.0](https://github.com/google/struct2tensor/releases/tag/v0.35.0) | 2.6.0
[0.34.0](https://github.com/google/struct2tensor/releases/tag/v0.34.0) | 2.6.0
[0.33.0](https://github.com/google/struct2tensor/releases/tag/v0.33.0) | 2.5.0
[0.32.0](https://github.com/google/struct2tensor/releases/tag/v0.32.0) | 2.5.0
[0.31.0](https://github.com/google/struct2tensor/releases/tag/v0.31.0) | 2.5.0
[0.30.0](https://github.com/google/struct2tensor/releases/tag/v0.30.0) | 2.4.0
[0.29.0](https://github.com/google/struct2tensor/releases/tag/v0.29.0) | 2.4.0
[0.28.0](https://github.com/google/struct2tensor/releases/tag/v0.28.0) | 2.4.0
[0.27.0](https://github.com/google/struct2tensor/releases/tag/v0.27.0) | 2.4.0
[0.26.0](https://github.com/google/struct2tensor/releases/tag/v0.26.0) | 2.3.0
[0.25.0](https://github.com/google/struct2tensor/releases/tag/v0.25.0) | 2.3.0
[0.24.0](https://github.com/google/struct2tensor/releases/tag/v0.24.0) | 2.3.0
[0.23.0](https://github.com/google/struct2tensor/releases/tag/v0.23.0) | 2.3.0
[0.22.0](https://github.com/google/struct2tensor/releases/tag/v0.22.0) | 2.2.0
[0.21.1](https://github.com/google/struct2tensor/releases/tag/v0.21.1) | 2.1.0
[0.21.0](https://github.com/google/struct2tensor/releases/tag/v0.21.0) | 2.1.0
0.0.1.dev* | 1.15
Raw data
{
"_id": null,
"home_page": null,
"name": "struct2tensor",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "tensorflow protobuf machine learning",
"author": "Google LLC",
"author_email": "tensorflow-extended-dev@googlegroups.com",
"download_url": null,
"platform": null,
"description": "# Struct2Tensor\n\n[](https://github.com/google/struct2tensor)\n[](https://badge.fury.io/py/struct2tensor)\n\n<!--*\nfreshness: { owner: 'iindyk' reviewed: '2024-04-22' }\n*-->\n\n## Introduction\nstruct2tensor is a library for parsing structured data inside of tensorflow.\nIn particular, it makes it easy to manipulate structured data, e.g., slicing,\nflattening, copying substructures, and so on, as part of a TensorFlow model\ngraph. The notebook in 'examples/prensor_playground.ipynb' provides a few\nexamples of struct2tensor in action and an introduction to the main\nconcepts. You can\n[run the notebook in your browser](https://colab.research.google.com/github/google/struct2tensor/blob/master/examples/prensor_playground.ipynb)\nthrough Google's colab environment, or [download the\nfile](examples/prensor_playground.ipynb) to run it in your own Jupyter\nenvironment.\n\n\nThere are two main use cases of this repo:\n\n1. To create a PIP package. The PIP package contains plug-ins (OpKernels) to an\n existing tensorflow installation.\n2. To staticlly link with tensorflow-serving.\n\nAs these processes are independent, one can follow either set of directions\nbelow.\n\n## Use a pre-built Linux PIP package.\n\n\nFrom a virtual environment, run:\n\n```bash\npip install struct2tensor\n```\n### Nightly Packages\n\nStruct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org\non Google Cloud. To install the latest nightly package, please use the following\ncommand:\n\n```bash\npip install --extra-index-url https://pypi-nightly.tensorflow.org/simple struct2tensor\n```\n\nThis will install the nightly packages for the major dependencies of\nstruct2tensor such as TensorFlow Metadata (TFMD).\n\n## Creating a PIP package.\n\nThe struct2tensor PIP package is useful for creating models.\nIt works with tensorflow 2.x.\n\nIn order to unify the process, we recommend compiling struct2tensor inside\na docker container.\n\n\n### Downloading the Code\n\nGo to your home directory.\n\nDownload the source code.\n\n```bash\ngit clone https://github.com/google/struct2tensor.git\ncd ~/struct2tensor\n```\n\n### Use docker-compose\nInstall [docker-compose](https://docs.docker.com/compose/).\n\nUse it to build a pip wheel for Python 3.8 with tensorflow version 2:\n\n```bash\ndocker-compose build --build-arg PYTHON_VERSION=3.8 manylinux2014\ndocker-compose run -e TF_VERSION=RELEASED_TF_2 manylinux2014\n```\n\nThis will create a manylinux package in the ~/struct2tensor/dist directory.\n\n\n## Creating a static library\n\nIn order to construct a static library for tensorflow-serving, we run:\n\n```bash\nbazel build -c opt struct2tensor:struct2tensor_kernels_and_ops\n```\n\nThis can also be linked into another library.\n\n## [TensorFlow Serving](https://github.com/tensorflow/serving) docker image\n\nstruct2tensor needs a couple of custom TensorFlow ops to function. If you train\na model with struct2tensor and wants to serve it with TensorFlow Serving, the\nTensorFlow Serving binary needs to link with those custom ops. We have a\npre-built docker image that contains such a binary. The `Dockerfile` is\navailable at `tools/tf_serving_docker/Dockerfile`. The image is available at\n`gcr.io/tfx-oss-public/s2t_tf_serving`.\n\nPlease see the `Dockerfile` for details. But in brief, the image exposes port\n8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set\ntwo environment variables `MODEL_BASE_PATH` and `MODEL_NAME` to point it to\nyour model (either mount it to the container, or put your model on GCS).\nIt will look for a saved model at\n`${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}`, where `VERSION_NUMBER`\nis an integer.\n\n\n## Compatibility\n\nstruct2tensor | tensorflow\n---------------------------------------------------------------------- | ----------\n[0.47.0](https://github.com/google/struct2tensor/releases/tag/v0.47.0) | 2.16.2\n[0.46.0](https://github.com/google/struct2tensor/releases/tag/v0.46.0) | 2.15.0\n[0.45.0](https://github.com/google/struct2tensor/releases/tag/v0.45.0) | 2.13.0\n[0.44.0](https://github.com/google/struct2tensor/releases/tag/v0.44.0) | 2.12.0\n[0.43.0](https://github.com/google/struct2tensor/releases/tag/v0.43.0) | 2.11.0\n[0.42.0](https://github.com/google/struct2tensor/releases/tag/v0.42.0) | 2.10.0\n[0.41.0](https://github.com/google/struct2tensor/releases/tag/v0.41.0) | 2.9.0\n[0.40.0](https://github.com/google/struct2tensor/releases/tag/v0.40.0) | 2.9.0\n[0.39.0](https://github.com/google/struct2tensor/releases/tag/v0.39.0) | 2.8.0\n[0.38.0](https://github.com/google/struct2tensor/releases/tag/v0.38.0) | 2.8.0\n[0.37.0](https://github.com/google/struct2tensor/releases/tag/v0.37.0) | 2.7.0\n[0.36.0](https://github.com/google/struct2tensor/releases/tag/v0.36.0) | 2.7.0\n[0.35.0](https://github.com/google/struct2tensor/releases/tag/v0.35.0) | 2.6.0\n[0.34.0](https://github.com/google/struct2tensor/releases/tag/v0.34.0) | 2.6.0\n[0.33.0](https://github.com/google/struct2tensor/releases/tag/v0.33.0) | 2.5.0\n[0.32.0](https://github.com/google/struct2tensor/releases/tag/v0.32.0) | 2.5.0\n[0.31.0](https://github.com/google/struct2tensor/releases/tag/v0.31.0) | 2.5.0\n[0.30.0](https://github.com/google/struct2tensor/releases/tag/v0.30.0) | 2.4.0\n[0.29.0](https://github.com/google/struct2tensor/releases/tag/v0.29.0) | 2.4.0\n[0.28.0](https://github.com/google/struct2tensor/releases/tag/v0.28.0) | 2.4.0\n[0.27.0](https://github.com/google/struct2tensor/releases/tag/v0.27.0) | 2.4.0\n[0.26.0](https://github.com/google/struct2tensor/releases/tag/v0.26.0) | 2.3.0\n[0.25.0](https://github.com/google/struct2tensor/releases/tag/v0.25.0) | 2.3.0\n[0.24.0](https://github.com/google/struct2tensor/releases/tag/v0.24.0) | 2.3.0\n[0.23.0](https://github.com/google/struct2tensor/releases/tag/v0.23.0) | 2.3.0\n[0.22.0](https://github.com/google/struct2tensor/releases/tag/v0.22.0) | 2.2.0\n[0.21.1](https://github.com/google/struct2tensor/releases/tag/v0.21.1) | 2.1.0\n[0.21.0](https://github.com/google/struct2tensor/releases/tag/v0.21.0) | 2.1.0\n0.0.1.dev* | 1.15\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Struct2Tensor is a package for parsing and manipulating structured data for TensorFlow",
"version": "0.47.0",
"project_urls": null,
"split_keywords": [
"tensorflow",
"protobuf",
"machine",
"learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "061f7db2cd6dab6709e80f19451993e7989286406be62d867e4187f992777fa3",
"md5": "72000a49585af156dd7d77fbe8622e60",
"sha256": "040b6a188e83f614eabcce5593dec9f4931cd9245e8cf03d7e06d936d0ac33f4"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp310-cp310-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "72000a49585af156dd7d77fbe8622e60",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 1876405,
"upload_time": "2024-10-10T21:12:51",
"upload_time_iso_8601": "2024-10-10T21:12:51.327191Z",
"url": "https://files.pythonhosted.org/packages/06/1f/7db2cd6dab6709e80f19451993e7989286406be62d867e4187f992777fa3/struct2tensor-0.47.0-cp310-cp310-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "de913ec0533fddd7dbb7b6e0de63813e6156e1697ba48ec1dd7387f6a1f670af",
"md5": "ebeb129922eee3468c5898062bdb1b5a",
"sha256": "57c2274e1bb8efbbadf711df1f775b41a1554b83ceb201f37c6952edcc66a979"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"has_sig": false,
"md5_digest": "ebeb129922eee3468c5898062bdb1b5a",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 3029976,
"upload_time": "2024-10-10T21:02:18",
"upload_time_iso_8601": "2024-10-10T21:02:18.448796Z",
"url": "https://files.pythonhosted.org/packages/de/91/3ec0533fddd7dbb7b6e0de63813e6156e1697ba48ec1dd7387f6a1f670af/struct2tensor-0.47.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "869104ec5f32705fba23e4a6af0c80d6ed565b68039da0e193de89fffde3d8a7",
"md5": "9176828148bd9fcbf02c6504ae1b37ae",
"sha256": "c3fc12b27e551ac51ca929594159f93e0bc56251af9fbc4a48a1a98ac048f9cb"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp311-cp311-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "9176828148bd9fcbf02c6504ae1b37ae",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": null,
"size": 1876406,
"upload_time": "2024-10-10T21:08:17",
"upload_time_iso_8601": "2024-10-10T21:08:17.619557Z",
"url": "https://files.pythonhosted.org/packages/86/91/04ec5f32705fba23e4a6af0c80d6ed565b68039da0e193de89fffde3d8a7/struct2tensor-0.47.0-cp311-cp311-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "28cfdb9f55700404af98fecb2d05f19610f627358702d6541602d4a1e173abee",
"md5": "892e4b75393cc7339f6f93085a9a46d4",
"sha256": "20034358c53f8f32bd45c2338168460fa9845325925cb0389895be7e2c641991"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"has_sig": false,
"md5_digest": "892e4b75393cc7339f6f93085a9a46d4",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": null,
"size": 3029974,
"upload_time": "2024-10-10T21:04:03",
"upload_time_iso_8601": "2024-10-10T21:04:03.009121Z",
"url": "https://files.pythonhosted.org/packages/28/cf/db9f55700404af98fecb2d05f19610f627358702d6541602d4a1e173abee/struct2tensor-0.47.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a6c0f53ce22778c18dff73aa161625c8f00a4200b0661f1ec65953a54d22c3eb",
"md5": "0dabf4bb08aed1767f798f033f8a5071",
"sha256": "9bfdd6bb365162022f416bf582685061b6d4c75085e97cb5d9426312221b07b5"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp39-cp39-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "0dabf4bb08aed1767f798f033f8a5071",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 1876405,
"upload_time": "2024-10-10T21:11:15",
"upload_time_iso_8601": "2024-10-10T21:11:15.998387Z",
"url": "https://files.pythonhosted.org/packages/a6/c0/f53ce22778c18dff73aa161625c8f00a4200b0661f1ec65953a54d22c3eb/struct2tensor-0.47.0-cp39-cp39-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d35b458fa29115b5192a416f845aaad933e803fa255ccdf40e5544bbcc61d1ba",
"md5": "003c8cafa78c2f6a95f5e334371293f0",
"sha256": "1eb9205becda68cc6d9d53c197cb6788de4ad3ebbab3e574e2957ba60ed6eaa0"
},
"downloads": -1,
"filename": "struct2tensor-0.47.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"has_sig": false,
"md5_digest": "003c8cafa78c2f6a95f5e334371293f0",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 3029974,
"upload_time": "2024-10-10T21:02:15",
"upload_time_iso_8601": "2024-10-10T21:02:15.348921Z",
"url": "https://files.pythonhosted.org/packages/d3/5b/458fa29115b5192a416f845aaad933e803fa255ccdf40e5544bbcc61d1ba/struct2tensor-0.47.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-10 21:12:51",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "struct2tensor"
}
JSON