# tspace
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
# Overview
tspace is an data pipleline framework for deep reinforcement learning
with IO interface, processing and configuration
- Working in training and inferrence mode
- logging and monitoring with cutelog or TUI interface
- cascaded threading pool for well-structured Scheduling of
[ETL](https://en.wikipedia.org/wiki/Extract,_transform,_load) and ML
pipelines
- Customized Exception handling
- Graceful shutdown
- online and offline training
- local and distributed training
- Support for multiple models
- reinforcement learning models with DDPG
- time sequence models with LSTM and Transformer
- Data pipeline compatible to both ETL and ML dataflow
- Support for multiple data sources (local CAN or remote cloud object
storage)
- Support both NoSQL database and local or cloud data storage through
Dask with Parquet and Avro interface
- Full Pandas DataFrame support with raw json codecs
- Configuration system for vehicles, drivers, data sites, neural
network hyperparameters, database, HMI types, etc
- Timezone aware time sequence data processing
- Data object meta-info processing and storage linked to configuration
system
- Stateful time sequence processing with sequential model
- Type hint for data processing and configuration
- Pydantic integration
<img src="res/tspace_overview.svg" alt="Overview of tspace architecture" width="80%">
The diagram shows the basic architure of tspace. The main components
are:
- **Avatar**: orchestrates the whole ETL and ML workflow.
- **Data Object**: The data object is a container for the data. It
contains the raw data, the metainfo and the configuration. The data
object is passed through the data pipeline and can be modified by the
processing components.
- **Data Pipeline**: The data pipeline is a sequence of processing
components. Each processing component takes a data object as input and
returns a data object as output. The data pipeline is responsible for
transforming the data object into a format that can be used by the
machine learning model.
- **Machine Learning Model**: The machine learning model is responsible
for training and inferencing. It takes the data object as input and
returns a prediction as output. The machine learning model can be a
reinforcement learning model, a time sequence model or any other type
of model.
- **Configuration System**: The configuration system is responsible for
storing and managing the configuration of the data pipeline, the
machine learning model and other components. The configuration system
is used to configure the data pipeline, the machine learning model and
other components.
- **Logging and Monitoring**: The logging and monitoring component is
responsible for logging and monitoring the data pipeline, the machine
learning model and other components. It provides real-time feedback on
the performance of the data pipeline and the machine learning model.
- **Exception Handling**: The exception handling component is
responsible for handling exceptions that occur during the execution of
the data pipeline, the machine learning model and other components. It
provides a mechanism for gracefully handling errors and recovering
from failures.
## TODO
1. Add time sequence embedding database support with LanceDB for
TimeGPT
2. Batch mode for large scale inference and training with Unit of Work
pattern
# How to use
## Install
``` sh
pip install tspace
```
Fill me in please! Don’t forget code examples:
``` python
1+1
```
2
``` python
# say_hi("world")
foo()
```
'foo'
Raw data
{
"_id": null,
"home_page": "https://github.com/Binjian/tspace",
"name": "tspace",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "nbdev jupyter notebook python",
"author": "binjian xin",
"author_email": "binjian.xin@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/4f/20/203ed05d740b0545cb95cc6a84eeb41c9c607e55f45bdde13f54a3a4bdd9/tspace-0.0.1.tar.gz",
"platform": null,
"description": "# tspace\n\n\n<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->\n\n# Overview\n\ntspace is an data pipleline framework for deep reinforcement learning\nwith IO interface, processing and configuration\n\n- Working in training and inferrence mode\n - logging and monitoring with cutelog or TUI interface\n - cascaded threading pool for well-structured Scheduling of\n [ETL](https://en.wikipedia.org/wiki/Extract,_transform,_load) and ML\n pipelines\n - Customized Exception handling\n - Graceful shutdown\n - online and offline training\n - local and distributed training\n- Support for multiple models\n - reinforcement learning models with DDPG\n - time sequence models with LSTM and Transformer\n- Data pipeline compatible to both ETL and ML dataflow\n - Support for multiple data sources (local CAN or remote cloud object\n storage)\n - Support both NoSQL database and local or cloud data storage through\n Dask with Parquet and Avro interface\n - Full Pandas DataFrame support with raw json codecs\n - Configuration system for vehicles, drivers, data sites, neural\n network hyperparameters, database, HMI types, etc\n - Timezone aware time sequence data processing\n - Data object meta-info processing and storage linked to configuration\n system\n - Stateful time sequence processing with sequential model\n - Type hint for data processing and configuration\n - Pydantic integration\n\n<img src=\"res/tspace_overview.svg\" alt=\"Overview of tspace architecture\" width=\"80%\">\n\nThe diagram shows the basic architure of tspace. The main components\nare:\n\n- **Avatar**: orchestrates the whole ETL and ML workflow.\n- **Data Object**: The data object is a container for the data. It\n contains the raw data, the metainfo and the configuration. The data\n object is passed through the data pipeline and can be modified by the\n processing components.\n- **Data Pipeline**: The data pipeline is a sequence of processing\n components. Each processing component takes a data object as input and\n returns a data object as output. The data pipeline is responsible for\n transforming the data object into a format that can be used by the\n machine learning model.\n- **Machine Learning Model**: The machine learning model is responsible\n for training and inferencing. It takes the data object as input and\n returns a prediction as output. The machine learning model can be a\n reinforcement learning model, a time sequence model or any other type\n of model.\n- **Configuration System**: The configuration system is responsible for\n storing and managing the configuration of the data pipeline, the\n machine learning model and other components. The configuration system\n is used to configure the data pipeline, the machine learning model and\n other components.\n- **Logging and Monitoring**: The logging and monitoring component is\n responsible for logging and monitoring the data pipeline, the machine\n learning model and other components. It provides real-time feedback on\n the performance of the data pipeline and the machine learning model.\n- **Exception Handling**: The exception handling component is\n responsible for handling exceptions that occur during the execution of\n the data pipeline, the machine learning model and other components. It\n provides a mechanism for gracefully handling errors and recovering\n from failures.\n\n## TODO\n\n1. Add time sequence embedding database support with LanceDB for\n TimeGPT\n2. Batch mode for large scale inference and training with Unit of Work\n pattern\n\n# How to use\n\n## Install\n\n``` sh\npip install tspace\n```\n\nFill me in please! Don\u2019t forget code examples:\n\n``` python\n1+1\n```\n\n 2\n\n``` python\n# say_hi(\"world\")\nfoo()\n```\n\n 'foo'\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "io interface",
"version": "0.0.1",
"project_urls": {
"Homepage": "https://github.com/Binjian/tspace"
},
"split_keywords": [
"nbdev",
"jupyter",
"notebook",
"python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "912452f00ff863fb622533dd23366e01e9bc70226bcbb5c59fd7447386a69d05",
"md5": "ff3904f781caaf8331f52ae49aad9e72",
"sha256": "f1fd443eccb56b8db2837372c2c1885ff1bd19c97e8b7a8b3b792046e4fda36b"
},
"downloads": -1,
"filename": "tspace-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ff3904f781caaf8331f52ae49aad9e72",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 140704,
"upload_time": "2024-05-24T13:44:41",
"upload_time_iso_8601": "2024-05-24T13:44:41.849293Z",
"url": "https://files.pythonhosted.org/packages/91/24/52f00ff863fb622533dd23366e01e9bc70226bcbb5c59fd7447386a69d05/tspace-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4f20203ed05d740b0545cb95cc6a84eeb41c9c607e55f45bdde13f54a3a4bdd9",
"md5": "fee7c02087d79add436f04131bb158a6",
"sha256": "6a9b894afe4e73279994618eb412f8aa921d6d413a02692680ea86445e7dabb1"
},
"downloads": -1,
"filename": "tspace-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "fee7c02087d79add436f04131bb158a6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 113319,
"upload_time": "2024-05-24T13:44:44",
"upload_time_iso_8601": "2024-05-24T13:44:44.448240Z",
"url": "https://files.pythonhosted.org/packages/4f/20/203ed05d740b0545cb95cc6a84eeb41c9c607e55f45bdde13f54a3a4bdd9/tspace-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-24 13:44:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Binjian",
"github_project": "tspace",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tspace"
}
JSON
