| Name | Zeitgleich JSON |
| Version |
0.0.5
JSON |
| download |
| home_page | None |
| Summary | Zeitgleich is a time series library for handling data and timestamps with origins. It provides a way to manage time series data, validate formats, and perform operations on time series data while maintaining origin information. |
| upload_time | 2025-01-09 09:58:42 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.6 |
| license | None |
| keywords |
conversion
origin
time series
timestamp
timezone
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Zeitgleich
**Zeitgleich** is a serious time series library designed for handling timestamped data with origins, specifically developed for use within an MQTT ecosystem in combination with the [SwampClient](https://git.ift.tuwien.ac.at/lab/ift/sis/researchlin-x/swampclient). It allows you to manage different timestamp formats, validate data schemas, and perform pandas operations on time series data while maintaining origin information.
## Features
- **Pandas Operations:** Perform standard Pandas operations (arithmetic, indexing, aggregation) directly on time series data while maintaining origin information.
- **Uniform Timestamp Handling:** Convert between ISO, Unix (seconds/nanoseconds), and RFC3339 formats.
- **Flexible Timestamp Conversion:** Specify desired output timestamp formats, defaulting to RFC3339, with support for various input formats.
- **Timestamp Normalization:** Standardize timestamps to a consistent format, localized to a specified timezone.
- **Origin Extraction:** Extract the device and topic from the `origin`, typically representing the MQTT topic the data originated from.
- **Data Validation:** Validate that input data (dictionaries or DataFrames) contain the required timestamp and value fields, with configurable naming presets.
- **Customization:** Adjust timestamp formats, column names, and origin patterns using method parameters.
- **Multi-Origin Time Series:** Manage MQTT data from multiple topics simultaneously using the `MultiOriginTimeSeries` class.
## Installation
```bash
pip install Zeitgleich
```
## Quick Start
Zeitgleich provides several examples to help you get started. These examples demonstrate how to manage single and multiple origins, perform timestamp conversions, and validate data formats. Refer to the `examples` directory in the project repository for detailed implementations.
### Managing Single Origin with Timestamp Conversion
Initialize a `TimeSeriesData` object with a single origin, convert timestamps from ISO format to Unix nanoseconds, and extract origin information.
### Managing Multiple Origins with `MultiOriginTimeSeries`
Handle MQTT data from multiple topics using the `MultiOriginTimeSeries` class. Add data from different origins with varying input timestamp formats and perform aggregate operations.
### Utilizing `TimestampFormat` for Various Conversions
Use the `TimestampFormat` class to convert timestamps for single values, lists, dictionaries, and DataFrames, including handling timezones.
### Data Validation and Error Handling
Zeitgleich enforces data validation rules by handling incorrect origin formats and missing data columns.
For complete code implementations, refer to the `examples` directory in the project repository.
## Data Formats
### Timestamp Normalization
Zeitgleich normalizes timestamps by:
- **Parsed into `datetime` Objects:** Timestamps are converted into `datetime` objects for consistency and ease of manipulation.
- **Timezone-Aware:** Timestamps are localized to a specified timezone (default is UTC) and can be converted to other timezones if needed.
- **Consistent Format:** After conversion, all timestamps have a consistent format as specified by the `output_timestamp_format` parameter, facilitating time-based operations.
### Flexible Timestamp Conversion
Zeitgleich allows for flexible conversion between various timestamp formats by specifying both input and output formats. Here's how you can customize them:
- **Input Timestamp Format (`input_timestamp_format`):** Specify the format of your input timestamps using the `input_timestamp_format` parameter. Supported formats include:
- `TimestampFormat.ISO` (e.g., "2021-06-05T12:00:00.123456")
- `TimestampFormat.RFC3339` (e.g., "2021-06-05T12:00:00.123456Z")
- `TimestampFormat.UNIX` (e.g., `1622894400`)
- `TimestampFormat.EPOCH_S` (e.g., `1622894400.123456`)
- `TimestampFormat.EPOCH_NS` (e.g., `1622894400123456000`)
- **Output Timestamp Format (`output_timestamp_format`):** Specify the desired format for your output timestamps using the `output_timestamp_format` parameter. Default is `TimestampFormat.RFC3339`, but you can choose any supported format.
- **Timestamp Column Name (`timestamp_col`):** Use the `timestamp_col` parameter to specify the name of your timestamp column if it differs from the default `'timestamp'`.
- **Timezone (`timezone_obj`):** Use the `timezone_obj` parameter to set the timezone for timestamp localization. This parameter accepts a `datetime.timezone` object.
### Origin Format
By default, the origin is assumed to be in the format `<device>/<topic>`. This typically represents the MQTT topic the data originated from. You can adjust the regex patterns in the `get_device()` and `get_topic()` methods to match your specific origin format.
### DataFrame Format
The DataFrame should contain a `timestamp` column and at least one data column (e.g., `value`). The timestamp column will be converted to the desired format and, by default, set as the index of the DataFrame.
## Format Validation
Zeitgleich includes data validation to ensure that your time series data adheres to the expected formats for origins and data columns. This is achieved using the `validate_format` decorator from `FormatChecks.py`.
### Origin and Column Checks
When initializing `TimeSeriesData` or adding data to `MultiOriginTimeSeries`, the following validations are performed:
1. **Origin Format Check:**
- Ensures that the `origin` string matches the expected pattern (default: `<device>/<topic>`).
- Raises a `ValueError` if the origin does not conform to the specified regex pattern.
2. **Data Columns Check:**
- Confirms the presence of required data columns (`timestamp` and `value` by default).
- Raises a `ValueError` if any required column is missing.
For detailed examples, refer to the `examples` directory.
## Unit Tests
This project includes unit tests to ensure functionality. Tests cover:
- **Timestamp Format Detection:** Verifying that different timestamp formats are correctly identified.
- **Timestamp Conversion:** Ensuring accurate conversion between timestamp formats.
- **DataFrame Validation:** Checking that data conforms to expected formats.
- **Origin Extraction:** Testing extraction of device and topic from origin strings.
- **Pandas Operations on Time Series Data:** Validating that standard Pandas operations work as expected on the time series data.
To run the tests, use the following command:
```bash
make tests
```
*Note: A Makefile is provided to simplify running all tests.*
## Future Developments
In the future, a complementary library will be developed to handle database interactions, allowing pushing of data received from different MQTT topics to a PostgreSQL time series database. This feature is currently under development.
## License
Licensed under the MIT License.
Raw data
{
"_id": null,
"home_page": null,
"name": "Zeitgleich",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "conversion, origin, time series, timestamp, timezone",
"author": null,
"author_email": "Jakob Friedl <jakob.friedl@tuwien.ac.at>",
"download_url": "https://files.pythonhosted.org/packages/e4/f6/c9022d94a81d63a296c9ba6d8587d98ba6a18a03ebd757530eec8e57affb/zeitgleich-0.0.5.tar.gz",
"platform": null,
"description": "# Zeitgleich\n\n**Zeitgleich** is a serious time series library designed for handling timestamped data with origins, specifically developed for use within an MQTT ecosystem in combination with the [SwampClient](https://git.ift.tuwien.ac.at/lab/ift/sis/researchlin-x/swampclient). It allows you to manage different timestamp formats, validate data schemas, and perform pandas operations on time series data while maintaining origin information.\n\n## Features\n\n- **Pandas Operations:** Perform standard Pandas operations (arithmetic, indexing, aggregation) directly on time series data while maintaining origin information.\n- **Uniform Timestamp Handling:** Convert between ISO, Unix (seconds/nanoseconds), and RFC3339 formats.\n- **Flexible Timestamp Conversion:** Specify desired output timestamp formats, defaulting to RFC3339, with support for various input formats.\n- **Timestamp Normalization:** Standardize timestamps to a consistent format, localized to a specified timezone.\n- **Origin Extraction:** Extract the device and topic from the `origin`, typically representing the MQTT topic the data originated from.\n- **Data Validation:** Validate that input data (dictionaries or DataFrames) contain the required timestamp and value fields, with configurable naming presets.\n- **Customization:** Adjust timestamp formats, column names, and origin patterns using method parameters.\n- **Multi-Origin Time Series:** Manage MQTT data from multiple topics simultaneously using the `MultiOriginTimeSeries` class.\n\n## Installation\n\n```bash\npip install Zeitgleich\n```\n\n## Quick Start\n\nZeitgleich provides several examples to help you get started. These examples demonstrate how to manage single and multiple origins, perform timestamp conversions, and validate data formats. Refer to the `examples` directory in the project repository for detailed implementations.\n\n### Managing Single Origin with Timestamp Conversion\n\nInitialize a `TimeSeriesData` object with a single origin, convert timestamps from ISO format to Unix nanoseconds, and extract origin information.\n\n### Managing Multiple Origins with `MultiOriginTimeSeries`\n\nHandle MQTT data from multiple topics using the `MultiOriginTimeSeries` class. Add data from different origins with varying input timestamp formats and perform aggregate operations.\n\n### Utilizing `TimestampFormat` for Various Conversions\n\nUse the `TimestampFormat` class to convert timestamps for single values, lists, dictionaries, and DataFrames, including handling timezones.\n\n### Data Validation and Error Handling\n\nZeitgleich enforces data validation rules by handling incorrect origin formats and missing data columns.\n\nFor complete code implementations, refer to the `examples` directory in the project repository.\n\n## Data Formats\n\n### Timestamp Normalization\n\nZeitgleich normalizes timestamps by:\n\n- **Parsed into `datetime` Objects:** Timestamps are converted into `datetime` objects for consistency and ease of manipulation.\n- **Timezone-Aware:** Timestamps are localized to a specified timezone (default is UTC) and can be converted to other timezones if needed.\n- **Consistent Format:** After conversion, all timestamps have a consistent format as specified by the `output_timestamp_format` parameter, facilitating time-based operations.\n\n### Flexible Timestamp Conversion\n\nZeitgleich allows for flexible conversion between various timestamp formats by specifying both input and output formats. Here's how you can customize them:\n\n- **Input Timestamp Format (`input_timestamp_format`):** Specify the format of your input timestamps using the `input_timestamp_format` parameter. Supported formats include:\n - `TimestampFormat.ISO` (e.g., \"2021-06-05T12:00:00.123456\")\n - `TimestampFormat.RFC3339` (e.g., \"2021-06-05T12:00:00.123456Z\")\n - `TimestampFormat.UNIX` (e.g., `1622894400`)\n - `TimestampFormat.EPOCH_S` (e.g., `1622894400.123456`)\n - `TimestampFormat.EPOCH_NS` (e.g., `1622894400123456000`)\n\n- **Output Timestamp Format (`output_timestamp_format`):** Specify the desired format for your output timestamps using the `output_timestamp_format` parameter. Default is `TimestampFormat.RFC3339`, but you can choose any supported format.\n\n- **Timestamp Column Name (`timestamp_col`):** Use the `timestamp_col` parameter to specify the name of your timestamp column if it differs from the default `'timestamp'`.\n\n- **Timezone (`timezone_obj`):** Use the `timezone_obj` parameter to set the timezone for timestamp localization. This parameter accepts a `datetime.timezone` object.\n\n### Origin Format\n\nBy default, the origin is assumed to be in the format `<device>/<topic>`. This typically represents the MQTT topic the data originated from. You can adjust the regex patterns in the `get_device()` and `get_topic()` methods to match your specific origin format.\n\n### DataFrame Format\n\nThe DataFrame should contain a `timestamp` column and at least one data column (e.g., `value`). The timestamp column will be converted to the desired format and, by default, set as the index of the DataFrame.\n\n## Format Validation\n\nZeitgleich includes data validation to ensure that your time series data adheres to the expected formats for origins and data columns. This is achieved using the `validate_format` decorator from `FormatChecks.py`.\n\n### Origin and Column Checks\n\nWhen initializing `TimeSeriesData` or adding data to `MultiOriginTimeSeries`, the following validations are performed:\n\n1. **Origin Format Check:**\n - Ensures that the `origin` string matches the expected pattern (default: `<device>/<topic>`).\n - Raises a `ValueError` if the origin does not conform to the specified regex pattern.\n\n2. **Data Columns Check:**\n - Confirms the presence of required data columns (`timestamp` and `value` by default).\n - Raises a `ValueError` if any required column is missing.\n\nFor detailed examples, refer to the `examples` directory.\n\n## Unit Tests\n\nThis project includes unit tests to ensure functionality. Tests cover:\n\n- **Timestamp Format Detection:** Verifying that different timestamp formats are correctly identified.\n- **Timestamp Conversion:** Ensuring accurate conversion between timestamp formats.\n- **DataFrame Validation:** Checking that data conforms to expected formats.\n- **Origin Extraction:** Testing extraction of device and topic from origin strings.\n- **Pandas Operations on Time Series Data:** Validating that standard Pandas operations work as expected on the time series data.\n\nTo run the tests, use the following command:\n\n```bash\nmake tests\n```\n\n*Note: A Makefile is provided to simplify running all tests.*\n\n## Future Developments\n\nIn the future, a complementary library will be developed to handle database interactions, allowing pushing of data received from different MQTT topics to a PostgreSQL time series database. This feature is currently under development.\n\n## License\n\nLicensed under the MIT License.",
"bugtrack_url": null,
"license": null,
"summary": "Zeitgleich is a time series library for handling data and timestamps with origins. It provides a way to manage time series data, validate formats, and perform operations on time series data while maintaining origin information.",
"version": "0.0.5",
"project_urls": null,
"split_keywords": [
"conversion",
" origin",
" time series",
" timestamp",
" timezone"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5b6d9d6723d73484c2aac18b34760d2bd97d091a2d95b3e2e9e1eecd8eb98848",
"md5": "496b8931798ea9558eea1b42c8e7af5f",
"sha256": "5b6977b7d5929f19faff6c80587431bdf6ecb636fd22f6d00009274345bd9a71"
},
"downloads": -1,
"filename": "zeitgleich-0.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "496b8931798ea9558eea1b42c8e7af5f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 13268,
"upload_time": "2025-01-09T09:58:41",
"upload_time_iso_8601": "2025-01-09T09:58:41.307180Z",
"url": "https://files.pythonhosted.org/packages/5b/6d/9d6723d73484c2aac18b34760d2bd97d091a2d95b3e2e9e1eecd8eb98848/zeitgleich-0.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e4f6c9022d94a81d63a296c9ba6d8587d98ba6a18a03ebd757530eec8e57affb",
"md5": "3dbb3472265b27b082d9c5f363d8841b",
"sha256": "887f212e6c30191207cc10f1099130582679c8f24bb2f17f4bd00b202b69d9e6"
},
"downloads": -1,
"filename": "zeitgleich-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "3dbb3472265b27b082d9c5f363d8841b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 48193,
"upload_time": "2025-01-09T09:58:42",
"upload_time_iso_8601": "2025-01-09T09:58:42.600735Z",
"url": "https://files.pythonhosted.org/packages/e4/f6/c9022d94a81d63a296c9ba6d8587d98ba6a18a03ebd757530eec8e57affb/zeitgleich-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-09 09:58:42",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "zeitgleich"
}
