# ICOlyzer
Auxiliary set of tools to analyze data from the [ICOtronic system](https://github.com/MyTooliT/ICOc):
- [ICOplotter](#readme:section:icoplotter): Visualize data and calculate signal to noise ratio
- [ICOanalyzer](#readme:section:icoanalyzer): Measures packet loss
- [ICOconverter](#readme:section:icoconverter): Convert HDF5 measurement files into other formats
## Requirements
- [Python](https://www.python.org)
## Setup Instructions
### Install
You can install the latest version of the tools from [PyPI](https://pypi.org/project/icolyzer/) using the command:
```sh
pip install icolyzer
```
To install the latest development version use:
```sh
pip install --upgrade git+https://github.com/mytoolit/ICOlyzer
```
### Remove
```sh
pip uninstall -y icolyzer
```
<a name="readme:section:icoplotter"></a>
## ICOplotter
- Measures signal to noise ratio (SNR) value of a measurement
- Opens a plotter window visualizing the recorded signal
- Plots the power spectral density graph for the recorded signal
- Prints the IFT value of the signal
- Adds optional red lines for time periods containing missing data values
### Usage
The ICOplotter script looks for a `log.hdf5` file in the current working directory by default. After using the command:
```sh
icoplotter
```
the script will load this file and open the graph of the recording:
<img src="https://raw.githubusercontent.com/MyTooliT/ICOlyzer/main/assets/Plotter.webp" alt="Plotter window" style="zoom:40%;" />
The calculated SNR will be written to the standard output.
```
SNR:
X: -106.50 dB @ 9.40 kHz
```
With closing the plotter the script will finish and the command line will be ready for a new command. For running the script with another input-file use one of the parameters described below.
#### Positional Parameters
##### Filepath
To change the HDF filepath from `log.hdf5` in the current working directory, please specify the filepath as positional argument.
Example:
```sh
icoplotter HDF5/log-x.hdf5
```
#### Optional Parameters
##### `-h`, `--help`
Prints the help menu of the command
##### `-l`, `--loss`
Highlight periods containing lost data using red lines:
<img src="https://raw.githubusercontent.com/MyTooliT/ICOlyzer/main/assets/Plotter-Data-Loss.webp" alt="Data plot containing red lines that highlight periods with data loss" style="zoom:40%;" />
##### `-p`, `--print`
Store the plotter output as PDF file instead of displaying it on the screen. The output is stored in the same location as the the input file with the extension `.pdf`.
##### `-s`, `--scatter`
Use a scatter plot (instead of a line plot) to visualize the measurement data
<a name="readme:section:icoanalyzer"></a>
## ICOanalyzer
- Measures packet-loss of the recorded signal
- Print number of sample points/channel
- Measures how many datapoints are outside of a given minimum and maximum value
- Prints warnings about long durations (more than 1 second) between consecutive timestamps
### Usage
The ICOanalyzer script looks for the file `log.hdf5` in the current working directory (usually this will be the root of this repository) by default. After you use the command:
```sh
icoanalyzer
```
the script will run and analyze `log.hdf5`. The script will then list the packet loss and values outside of the given range (default is -1g and 1g):
```
Input: log.hdf5
Packet Loss: 0.61%
Data Points:
X-Axis: 282021 Samples - 127772 Samples were over 1g or below -1g (45.31%)
Measurement Date: 2021-12-20T11:00:21.489537
Runtime: 29.996 seconds
```
If you want to change the file the script analyzes or the minimum and maximum values you have to provide some extra parameters. Using multiple parameters is also possible.
#### Parameters
##### `-h`, `--help`
This argument calls the help menu of the script instead of running the script.
##### Filepath
With the `input` positional parameter:
```sh
icoanalyzer examples/log-xz.hdf5
```
you can change the file the script analyzes:
```
Input: examples/log-xz.hdf5
Packet Loss: 24.67%
Data Points:
X-Axis: 106992 Samples - 106992 Samples were over 1g or below -1g (100.0%)
Z-Axis: 106992 Samples - 106992 Samples were over 1g or below -1g (100.0%)
Measurement Date: 2022-01-03T07:52:29.573573
Runtime: 29.993 seconds
```
The tool also supports multiple input files. For example, if you want to analyze all `.hdf5` files in the current directory, then you can use the following command:
```sh
icoanalyzer *.hdf5
```
##### `-d`, `--details`
With `-d`:
```sh
icoanalyzer -d
```
you can activate a more detailed information about the packet loss:
```
Input: log.hdf5
64 Packets lost after 2.979 seconds - No values for 0.3 milliseconds
64 Packets lost after 3.274 seconds - No values for 0.3 milliseconds
64 Packets lost after 7.089 seconds - No values for 0.3 milliseconds
64 Packets lost after 11.192 seconds - No values for 0.3 milliseconds
64 Packets lost after 16.931 seconds - No values for 0.3 milliseconds
64 Packets lost after 22.149 seconds - No values for 0.3 milliseconds
64 Packets lost after 22.782 seconds - No values for 0.2 milliseconds
64 Packets lost after 25.781 seconds - No values for 0.3 milliseconds
64 Packets lost after 29.623 seconds - No values for 0.2 milliseconds
Packet Loss: 0.61%
Data Points:
X-Axis: 282021 Samples - 127772 Samples were over 1g or below -1g (45.31%)
Measurement Date: 2021-12-20T11:00:21.489537
Runtime: 29.996 seconds
```
Not only will it show the percentage of data loss for the file, it will now also show how many packets were lost in each packet loss event.
##### `-m`, `--min`
With "-m VALUE" you can change the minimal value for datapoints to be checked. All values below this parameter will be counted as out of the borders.
##### `-v`, `--max`
With "-v VALUE":
```sh
icoanalyzer -m -0.42 -v 2.55
```
you can change the maximum value for datapoints to be checked. All values above this parameter will be counted as out of the borders.
```
Input: log-x.hdf5
Packet Loss: 0.61%
Data Points:
X-Axis: 282021 Samples - 269420 Samples were over 2.55g or below -0.42g (95.53%)
Measurement Date: 2021-12-20T11:00:21.489537
Runtime: 29.996 seconds
```
<a name="readme:section:icoconverter"></a>
## ICOconverter
- Converts HDF5 measurement files (`.hdf5`) into `.csv` (Comma-Separated Values) or `.xlsx` (Excel) files
### Usage
The ICOconverter script looks for the file `log.hdf5` in the current working directory by default. After you use the command:
```
icoconverter
```
the script will run and analyze `log.hdf5`. The script will then convert the .hdf5 file and create `log.csv` and save it into the same directory. Attention: When there is already an file with the name of the converted .csv it will be overwritten.
```
> icoconverter
Starting the conversion process
Converting “log.hdf5” to “log.csv”
Finished the conversion process
```
If you want to change the file the script converts or the format it should convert to you have to provide some extra parameters. Using multiple parameters is also possible.
#### Parameters
##### `-h`, `--help`
This argument calls the help menu of the script instead of running the script.
##### Filepath
With the `inputs` positional parameter you can change the file the script converts.
```
icoconverter ~/Downloads/log.hdf5
Starting the conversion process
Converting “/Users/rene/Downloads/log.hdf5” to “/Users/rene/Downloads/log.csv”
Finished the conversion process
```
This path can be given as a relative or an absolute path. If the given path is a folder the script will convert ALL .hdf5 files inside the given folder.
```
icoconverter ~/Downloads/TEST-LOGS-HDF5
Starting the conversion process
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-z.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-z.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xy.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xy.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xyz.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xyz.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xz.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-xz.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-x.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-x.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-yz.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-yz.csv”
Converting “/Users/rene/Downloads/TEST-LOGS-HDF5/log-y.hdf5” to “/Users/rene/Downloads/TEST-LOGS-HDF5/log-y.csv”
Finished the conversion process
```
##### `-e`, `--excel`
With "-e" you can change the format the script converts to to excel sheets. Instead of .csv it now creates .xlsx files.
```
icoconverter ~/Downloads/*.hdf5 -e
Starting the conversion process
Starting the conversion of: /Users/rene/Downloads/log.hdf5
Starting the conversion of: /Users/rene/Downloads/Measurement_2022-03-31_10-20-16.hdf5
Finished the conversion process
```
Raw data
{
"_id": null,
"home_page": "",
"name": "icolyzer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "smart-tool,smh,stu,sth,tool-holder",
"author": "",
"author_email": "Clemens Burgstaller <burgstaller@ift.at>, Norbert Leder <norbert.leder@mytoolit.com>, Ren\u00e9 Schwaiger <rene.schwaiger@ift.at>, Thomas Hirschb\u00fcchler <hirschbuechler@ift.at>",
"download_url": "https://files.pythonhosted.org/packages/64/d5/0a555bd817eb1c101ffbb3d08771454a77d438df528d45b811c9113bfe4c/icolyzer-1.6.0.tar.gz",
"platform": null,
"description": "# ICOlyzer\n\nAuxiliary set of tools to analyze data from the [ICOtronic system](https://github.com/MyTooliT/ICOc):\n\n- [ICOplotter](#readme:section:icoplotter): Visualize data and calculate signal to noise ratio\n- [ICOanalyzer](#readme:section:icoanalyzer): Measures packet loss\n- [ICOconverter](#readme:section:icoconverter): Convert HDF5 measurement files into other formats\n\n## Requirements\n\n- [Python](https://www.python.org)\n\n## Setup Instructions\n\n### Install\n\nYou can install the latest version of the tools from [PyPI](https://pypi.org/project/icolyzer/) using the command:\n\n```sh\npip install icolyzer\n```\n\nTo install the latest development version use:\n\n```sh\npip install --upgrade git+https://github.com/mytoolit/ICOlyzer\n```\n\n### Remove\n\n```sh\npip uninstall -y icolyzer\n```\n\n<a name=\"readme:section:icoplotter\"></a>\n\n## ICOplotter\n\n- Measures signal to noise ratio (SNR) value of a measurement\n- Opens a plotter window visualizing the recorded signal\n- Plots the power spectral density graph for the recorded signal\n- Prints the IFT value of the signal\n- Adds optional red lines for time periods containing missing data values\n\n### Usage\n\nThe ICOplotter script looks for a `log.hdf5` file in the current working directory by default. After using the command:\n\n```sh\nicoplotter\n```\n\nthe script will load this file and open the graph of the recording:\n\n<img src=\"https://raw.githubusercontent.com/MyTooliT/ICOlyzer/main/assets/Plotter.webp\" alt=\"Plotter window\" style=\"zoom:40%;\" />\n\nThe calculated SNR will be written to the standard output.\n\n```\nSNR:\n X: -106.50 dB @ 9.40 kHz\n```\n\nWith closing the plotter the script will finish and the command line will be ready for a new command. For running the script with another input-file use one of the parameters described below.\n\n#### Positional Parameters\n\n##### Filepath\n\nTo change the HDF filepath from `log.hdf5` in the current working directory, please specify the filepath as positional argument.\n\nExample:\n\n```sh\nicoplotter HDF5/log-x.hdf5\n```\n\n#### Optional Parameters\n\n##### `-h`, `--help`\n\nPrints the help menu of the command\n\n##### `-l`, `--loss`\n\nHighlight periods containing lost data using red lines:\n\n<img src=\"https://raw.githubusercontent.com/MyTooliT/ICOlyzer/main/assets/Plotter-Data-Loss.webp\" alt=\"Data plot containing red lines that highlight periods with data loss\" style=\"zoom:40%;\" />\n\n##### `-p`, `--print`\n\nStore the plotter output as PDF file instead of displaying it on the screen. The output is stored in the same location as the the input file with the extension `.pdf`.\n\n##### `-s`, `--scatter`\n\nUse a scatter plot (instead of a line plot) to visualize the measurement data\n\n<a name=\"readme:section:icoanalyzer\"></a>\n\n## ICOanalyzer\n\n- Measures packet-loss of the recorded signal\n- Print number of sample points/channel\n- Measures how many datapoints are outside of a given minimum and maximum value\n- Prints warnings about long durations (more than 1 second) between consecutive timestamps\n\n### Usage\n\nThe ICOanalyzer script looks for the file `log.hdf5` in the current working directory (usually this will be the root of this repository) by default. After you use the command:\n\n```sh\nicoanalyzer\n```\n\nthe script will run and analyze `log.hdf5`. The script will then list the packet loss and values outside of the given range (default is -1g and 1g):\n\n```\nInput: log.hdf5\nPacket Loss: 0.61%\nData Points:\n X-Axis: 282021 Samples - 127772 Samples were over 1g or below -1g (45.31%)\nMeasurement Date: 2021-12-20T11:00:21.489537\nRuntime: 29.996 seconds\n```\n\nIf you want to change the file the script analyzes or the minimum and maximum values you have to provide some extra parameters. Using multiple parameters is also possible.\n\n#### Parameters\n\n##### `-h`, `--help`\n\nThis argument calls the help menu of the script instead of running the script.\n\n##### Filepath\n\nWith the `input` positional parameter:\n\n```sh\nicoanalyzer examples/log-xz.hdf5\n```\n\nyou can change the file the script analyzes:\n\n```\nInput: examples/log-xz.hdf5\nPacket Loss: 24.67%\nData Points:\n X-Axis: 106992 Samples - 106992 Samples were over 1g or below -1g (100.0%)\n Z-Axis: 106992 Samples - 106992 Samples were over 1g or below -1g (100.0%)\nMeasurement Date: 2022-01-03T07:52:29.573573\nRuntime: 29.993 seconds\n```\n\nThe tool also supports multiple input files. For example, if you want to analyze all `.hdf5` files in the current directory, then you can use the following command:\n\n```sh\nicoanalyzer *.hdf5\n```\n\n##### `-d`, `--details`\n\nWith `-d`:\n\n```sh\nicoanalyzer -d\n```\n\nyou can activate a more detailed information about the packet loss:\n\n```\nInput: log.hdf5\n 64 Packets lost after 2.979 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 3.274 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 7.089 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 11.192 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 16.931 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 22.149 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 22.782 seconds - No values for 0.2 milliseconds\n 64 Packets lost after 25.781 seconds - No values for 0.3 milliseconds\n 64 Packets lost after 29.623 seconds - No values for 0.2 milliseconds\nPacket Loss: 0.61%\nData Points:\n X-Axis: 282021 Samples - 127772 Samples were over 1g or below -1g (45.31%)\nMeasurement Date: 2021-12-20T11:00:21.489537\nRuntime: 29.996 seconds\n```\n\nNot only will it show the percentage of data loss for the file, it will now also show how many packets were lost in each packet loss event.\n\n##### `-m`, `--min`\n\nWith \"-m VALUE\" you can change the minimal value for datapoints to be checked. All values below this parameter will be counted as out of the borders.\n\n##### `-v`, `--max`\n\nWith \"-v VALUE\":\n\n```sh\nicoanalyzer -m -0.42 -v 2.55\n```\n\nyou can change the maximum value for datapoints to be checked. All values above this parameter will be counted as out of the borders.\n\n```\nInput: log-x.hdf5\nPacket Loss: 0.61%\nData Points:\n X-Axis: 282021 Samples - 269420 Samples were over 2.55g or below -0.42g (95.53%)\nMeasurement Date: 2021-12-20T11:00:21.489537\nRuntime: 29.996 seconds\n```\n\n<a name=\"readme:section:icoconverter\"></a>\n\n## ICOconverter\n\n- Converts HDF5 measurement files (`.hdf5`) into `.csv` (Comma-Separated Values) or `.xlsx` (Excel) files\n\n### Usage\n\nThe ICOconverter script looks for the file `log.hdf5` in the current working directory by default. After you use the command:\n\n```\nicoconverter\n```\n\nthe script will run and analyze `log.hdf5`. The script will then convert the .hdf5 file and create `log.csv` and save it into the same directory. Attention: When there is already an file with the name of the converted .csv it will be overwritten.\n\n```\n> icoconverter\nStarting the conversion process\nConverting \u201clog.hdf5\u201d to \u201clog.csv\u201d\nFinished the conversion process\n```\n\nIf you want to change the file the script converts or the format it should convert to you have to provide some extra parameters. Using multiple parameters is also possible.\n\n#### Parameters\n\n##### `-h`, `--help`\n\nThis argument calls the help menu of the script instead of running the script.\n\n##### Filepath\n\nWith the `inputs` positional parameter you can change the file the script converts.\n\n```\nicoconverter ~/Downloads/log.hdf5\nStarting the conversion process\nConverting \u201c/Users/rene/Downloads/log.hdf5\u201d to \u201c/Users/rene/Downloads/log.csv\u201d\nFinished the conversion process\n```\n\nThis path can be given as a relative or an absolute path. If the given path is a folder the script will convert ALL .hdf5 files inside the given folder.\n\n```\nicoconverter ~/Downloads/TEST-LOGS-HDF5\nStarting the conversion process\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-z.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-z.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xy.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xy.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xyz.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xyz.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xz.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-xz.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-x.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-x.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-yz.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-yz.csv\u201d\nConverting \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-y.hdf5\u201d to \u201c/Users/rene/Downloads/TEST-LOGS-HDF5/log-y.csv\u201d\nFinished the conversion process\n```\n\n##### `-e`, `--excel`\n\nWith \"-e\" you can change the format the script converts to to excel sheets. Instead of .csv it now creates .xlsx files.\n\n```\nicoconverter ~/Downloads/*.hdf5 -e\nStarting the conversion process\nStarting the conversion of: /Users/rene/Downloads/log.hdf5\nStarting the conversion of: /Users/rene/Downloads/Measurement_2022-03-31_10-20-16.hdf5\nFinished the conversion process\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "Auxiliary set of tools to work with measurement data produced by the ICOtronic system",
"version": "1.6.0",
"project_urls": {
"Source": "https://github.com/mytoolit/ICOlyzer/"
},
"split_keywords": [
"smart-tool",
"smh",
"stu",
"sth",
"tool-holder"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7a4f3da2bcb4c367827921d10c23600249837c34cb1f52866352606fb8a16920",
"md5": "5d97c5ec439a2426134cadc50cce5cd8",
"sha256": "cb4d13f1ff68d81f7336156a6d9d83c0d94cda75c30a6591b740786a8c5569a7"
},
"downloads": -1,
"filename": "icolyzer-1.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5d97c5ec439a2426134cadc50cce5cd8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 32816,
"upload_time": "2023-11-27T16:45:27",
"upload_time_iso_8601": "2023-11-27T16:45:27.440324Z",
"url": "https://files.pythonhosted.org/packages/7a/4f/3da2bcb4c367827921d10c23600249837c34cb1f52866352606fb8a16920/icolyzer-1.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "64d50a555bd817eb1c101ffbb3d08771454a77d438df528d45b811c9113bfe4c",
"md5": "b1c6b695e607295f8a367ed430b3a38a",
"sha256": "4547b0f7d2288ae41819546e90ad2a7102e3f74759aca50f6e794cf85d21d87c"
},
"downloads": -1,
"filename": "icolyzer-1.6.0.tar.gz",
"has_sig": false,
"md5_digest": "b1c6b695e607295f8a367ed430b3a38a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 35195,
"upload_time": "2023-11-27T16:45:29",
"upload_time_iso_8601": "2023-11-27T16:45:29.340462Z",
"url": "https://files.pythonhosted.org/packages/64/d5/0a555bd817eb1c101ffbb3d08771454a77d438df528d45b811c9113bfe4c/icolyzer-1.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-27 16:45:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mytoolit",
"github_project": "ICOlyzer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "matplotlib",
"specs": [
[
">=",
"3.5.1"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.20"
]
]
},
{
"name": "pandas",
"specs": [
[
">=",
"1.3.5"
]
]
},
{
"name": "tables",
"specs": [
[
">=",
"3.7.0"
]
]
},
{
"name": "openpyxl",
"specs": []
}
],
"lcname": "icolyzer"
}
JSON
