# Python Streaming Data Types
Utilities for working with the FlatBuffers schemas used at the European
Spallation Source ERIC for data transport.
https://github.com/ess-dmsc/streaming-data-types
## FlatBuffer Schemas
| name | description |
|------|------------------------------------------------------------------------------|
| hs00 | Histogram schema (deprecated in favour of hs01) |
| hs01 | Histogram schema |
| ns10 | NICOS cache entry schema |
| pl72 | Run start |
| 6s4t | Run stop |
| f142 | Log data (deprecated in favour of f144) |
| f144 | Log data |
| ev42 | Event data (deprecated in favour of ev44) |
| ev43 | Event data from multiple pulses |
| ev44 | Event data with signed data types |
| x5f2 | Status messages |
| tdct | Timestamps |
| ep00 | EPICS connection info (deprecated in favour of ep01) |
| ep01 | EPICS connection info |
| rf5k | Forwarder configuration update (deprecated in favour of fc00) |
| fc00 | Forwarder configuration update |
| answ | File-writer command response |
| wrdn | File-writer finished writing |
| NDAr | **Deprecated** |
| ADAr | EPICS areaDetector data |
| al00 | Alarm/status messages used by the Forwarder and NICOS |
| senv | **Deprecated** |
| json | Generic JSON data |
| se00 | Arrays with optional timestamps, for example waveform data. Replaces _senv_. |
| da00 | Scipp-like data arrays, for histograms, etc. |
### hs00 and hs01
Schema for histogram data. It is one of the more complicated to use schemas.
It takes a Python dictionary as its input; this dictionary needs to have correctly
named fields.
The input histogram data for serialisation and the output deserialisation data
have the same dictionary "layout".
Example for a 2-D histogram:
```json
hist = {
"source": "some_source",
"timestamp": 123456,
"current_shape": [2, 5],
"dim_metadata": [
{
"length": 2,
"unit": "a",
"label": "x",
"bin_boundaries": np.array([10, 11, 12]),
},
{
"length": 5,
"unit": "b",
"label": "y",
"bin_boundaries": np.array([0, 1, 2, 3, 4, 5]),
},
],
"last_metadata_timestamp": 123456,
"data": np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]),
"errors": np.array([[5, 4, 3, 2, 1], [10, 9, 8, 7, 6]]),
"info": "info_string",
}
```
The arrays passed in for `data`, `errors` and `bin_boundaries` can be NumPy arrays
or regular lists, but on deserialisation they will be NumPy arrays.
## Developer documentation
See [README_DEV.md](README_DEV.md)
Raw data
{
"_id": null,
"home_page": "https://github.com/ess-dmsc/python-streaming-data-types",
"name": "ess-streaming-data-types",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6.0",
"maintainer_email": null,
"keywords": null,
"author": "ScreamingUdder",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4f/62/e55d3945a9e6792aa15b0bbc1328f89ab9639f6e8c00e88c9bb4365088fe/ess_streaming_data_types-0.26.1.tar.gz",
"platform": null,
"description": "\n# Python Streaming Data Types\nUtilities for working with the FlatBuffers schemas used at the European\nSpallation Source ERIC for data transport.\n\nhttps://github.com/ess-dmsc/streaming-data-types\n\n## FlatBuffer Schemas\n\n| name | description |\n|------|------------------------------------------------------------------------------|\n| hs00 | Histogram schema (deprecated in favour of hs01) |\n| hs01 | Histogram schema |\n| ns10 | NICOS cache entry schema |\n| pl72 | Run start |\n| 6s4t | Run stop |\n| f142 | Log data (deprecated in favour of f144) |\n| f144 | Log data |\n| ev42 | Event data (deprecated in favour of ev44) |\n| ev43 | Event data from multiple pulses |\n| ev44 | Event data with signed data types |\n| x5f2 | Status messages |\n| tdct | Timestamps |\n| ep00 | EPICS connection info (deprecated in favour of ep01) |\n| ep01 | EPICS connection info |\n| rf5k | Forwarder configuration update (deprecated in favour of fc00) |\n| fc00 | Forwarder configuration update |\n| answ | File-writer command response |\n| wrdn | File-writer finished writing |\n| NDAr | **Deprecated** |\n| ADAr | EPICS areaDetector data |\n| al00 | Alarm/status messages used by the Forwarder and NICOS |\n| senv | **Deprecated** |\n| json | Generic JSON data |\n| se00 | Arrays with optional timestamps, for example waveform data. Replaces _senv_. |\n| da00 | Scipp-like data arrays, for histograms, etc. |\n\n### hs00 and hs01\nSchema for histogram data. It is one of the more complicated to use schemas.\nIt takes a Python dictionary as its input; this dictionary needs to have correctly\nnamed fields.\n\nThe input histogram data for serialisation and the output deserialisation data\nhave the same dictionary \"layout\".\nExample for a 2-D histogram:\n```json\nhist = {\n \"source\": \"some_source\",\n \"timestamp\": 123456,\n \"current_shape\": [2, 5],\n \"dim_metadata\": [\n {\n \"length\": 2,\n \"unit\": \"a\",\n \"label\": \"x\",\n \"bin_boundaries\": np.array([10, 11, 12]),\n },\n {\n \"length\": 5,\n \"unit\": \"b\",\n \"label\": \"y\",\n \"bin_boundaries\": np.array([0, 1, 2, 3, 4, 5]),\n },\n ],\n \"last_metadata_timestamp\": 123456,\n \"data\": np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]),\n \"errors\": np.array([[5, 4, 3, 2, 1], [10, 9, 8, 7, 6]]),\n \"info\": \"info_string\",\n}\n```\nThe arrays passed in for `data`, `errors` and `bin_boundaries` can be NumPy arrays\nor regular lists, but on deserialisation they will be NumPy arrays.\n\n\n## Developer documentation\n\nSee [README_DEV.md](README_DEV.md)\n",
"bugtrack_url": null,
"license": "BSD 2-Clause License",
"summary": "Python utilities for handling ESS streamed data",
"version": "0.26.1",
"project_urls": {
"Homepage": "https://github.com/ess-dmsc/python-streaming-data-types"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1600fc5ab795aa4c81e45b0090e6a4e9ddc2c09db9e1a12ec410e6963dea406a",
"md5": "e95fe45465da6c18181eed064872d482",
"sha256": "47ef984451b51aed71c45a930a379ca7dbfb6cce7837a81aaaed7348b0bacbd7"
},
"downloads": -1,
"filename": "ess_streaming_data_types-0.26.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e95fe45465da6c18181eed064872d482",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 148632,
"upload_time": "2024-05-28T10:53:12",
"upload_time_iso_8601": "2024-05-28T10:53:12.112416Z",
"url": "https://files.pythonhosted.org/packages/16/00/fc5ab795aa4c81e45b0090e6a4e9ddc2c09db9e1a12ec410e6963dea406a/ess_streaming_data_types-0.26.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4f62e55d3945a9e6792aa15b0bbc1328f89ab9639f6e8c00e88c9bb4365088fe",
"md5": "bd59e74923d0d0602c85c3c08e5fa377",
"sha256": "82deeb65073dd701732f18baabdd47c8afd6bda21ee5f2f2a27f0ce19e24fa0e"
},
"downloads": -1,
"filename": "ess_streaming_data_types-0.26.1.tar.gz",
"has_sig": false,
"md5_digest": "bd59e74923d0d0602c85c3c08e5fa377",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 59745,
"upload_time": "2024-05-28T10:53:14",
"upload_time_iso_8601": "2024-05-28T10:53:14.379461Z",
"url": "https://files.pythonhosted.org/packages/4f/62/e55d3945a9e6792aa15b0bbc1328f89ab9639f6e8c00e88c9bb4365088fe/ess_streaming_data_types-0.26.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-28 10:53:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ess-dmsc",
"github_project": "python-streaming-data-types",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "flatbuffers",
"specs": [
[
"==",
"22.11.23"
]
]
},
{
"name": "numpy",
"specs": []
}
],
"tox": true,
"lcname": "ess-streaming-data-types"
}
JSON
