[](https://travis-ci.com/MideTechnology/idelib) [](https://codecov.io/gh/MideTechnology/idelib)
# _idelib_ README
_idelib_ is a Python API for accessing [enDAQ's](http://endaq.com) IDE recordings. The IDE format is
an [EBML](http://matroska-org.github.io/libebml/specs.html) encoded file using a
custom schema. This library utilizes our
[ebmlite](https://github.com/MideTechnology/ebmlite) to parse the files, and
provides classes that make reading data simple.
## IDE File Basics
### What's an IDE file?
An IDE file is a read-only hierarchical data format that stores recording
information generated by an [enDAQ sensor device](http://endaq.com/collections/endaq-sensors-shock-vibration-s-series). It contains both time-indexed
data from several different kinds of sensors (like acceleration, pressure,
temperature, etc.), as well as metadata about the recording device (like device
serial number, model number, device name, etc.) and recording settings.
### Accessing an IDE file
The top-level interface for an IDE file is the `Dataset` object, through which
one can access all of the above-listed information. When you open a file for
reading, for example, this is the type of object that is returned.
#### Opening an IDE File
You can open an IDE file like so:
```python
filename = "your_file.IDE"
with idelib.importFile(filename) as ds:
print(type(ds))
```
**Note**: a `Dataset` object perfoms _lazy-loading_, meaning that it only loads
information as is needed. As a result, it internally retains a handle to the
source file which after use needs to be closed. This can be accomplished by
either using `Dataset` as a _context manager_ (as seen above; this is the
recommended method), or by using `Dataset` as a normal value and calling the
`close()` method manually:
```python
filename = "your_file.IDE"
ds = idelib.importFile(filename)
# use `ds` here
ds.close() # remember to close the file after use!
```
### Getting recording data
#### Channels and Subchannels
IDE files organize recording data into _channels_ and _subchannels_. A channel
encapsulates data recorded by a particular individual sensor on the device
(e.g., XYZ acceleration from the ADXL375 DC Accelerometer); a subchannel, if
present, specifies a particular data stream within a channel (e.g., the
X-coordinate acceleration from the ADXL375 DC Accelerometer).
At the top-level, a `Dataset` object has a `channels` member, which is a dict
of all channels recorded in the file. The dict is keyed by channel id numbers,
with `Channel` objects in the values.
Each `Channel` object has a `subchannels` member, which is a list of
`Subchannel` objects. If the channel has no subchannels, this member will be `None`.
The below table lists current conventions for channels across all enDAQ sensors:
| (Abbreviated) Product No. | Description | Example Product Nos. |
|--------------------------:|--------------------------------------------------------------------------------|-------------------------|
| S-D | enDAQ S-series devices with a single digital accelerometer | S3-D16, S4-D40 |
| S-DD | enDAQ S-series devices with dual digital accelerometers | S1-D100D40, S2-D25D16 |
| S-ED | enDAQ S-series devices with an analog piezoelectric and digital accelerometer | S5-E25D40, S4-E100D40 |
| S-RD | enDAQ S-series devices with an analog piezoresistive and digital accelerometer | S4-R500D40, S5-R2000D40 |
| W-D | enDAQ W-series devices with a single digital accelerometer | W5-D40 |
| W-ED | enDAQ W-series devices with an analog piezoelectric and digital accelerometer | W8-E100D40, W8-E2000D40 |
| W-RD | enDAQ W-series devices with an analog piezoresistive and digital accelerometer | W8-R500D40, W8-R2000D40 |
| SSX | Midé Slam Stick X data recorders | SSX |
| SSC | Midé Slam Stick C data recorders | SSC |
| SSS | Midé Slam Stick S data recorders | SSS |
The below table lists channel ID numbers used in a recording file based on the
recording device’s product number (device series numbers and accelerometer
sensitivity ranges are omitted when applicable to all such devices):
| Sensor | Channel | Valid Devices | Suchannels |
|------------------------:|:--------|-----------------------------------------|------------------------------------|
| Main Accelerometer | 8 | S-RD, S-ED, SSS, SSX | X-, Y-, Z-axis Acceleration |
| 16/200g Accelerometer | 32 | S-DD, SSX, SSS, SSC, S-D16, S-D200 | X-, Y-, Z-axis Acceleration |
| 8/40g Accelerometer | 80 | S-RD, S-DD, S-ED, S-D40, S-D8 | X-, Y-, Z-axis Acceleration |
| IMU Gyroscope | 47 | All<sup>1</sup> | X-, Y-, Z-axis Rotation |
| Absolute Orientation | 65 | All<sup>1</sup> | X-, Y-, Z-, W-axis Quaternion; Acc |
| Relative Orientation | 70 | All<sup>1</sup> | X-, Y-, Z-, W-axis Quaternion |
| MPL3115 | 36 | S-D16, All<sup>1</sup> before Mid-2023 | Pressure, Temperature <sup>2</sup> |
| MS8607 Internal | 20 | All<sup>1</sup> after Mid-2023 | Pressure, Temperature, Humidity |
| MS8607 Control Pad | 59 | All<sup>1</sup> | Pressure, Temperature, Humidity |
| SI1133 | 76 | All<sup>1</sup> | Lux, UV |
| BMI270/BMG250 Gyroscope | 84 | All<sup>1</sup> after Mid-2023 | X-, Y-, Z-axis Rotation |
| CAM-M8Q GPS | 88 | W-D, W-ED, W-RD | Latitude, Longitude, Time, Speed |
<sup>1</sup> excluding early SSC/SSS/SSX models
<sup>2</sup> 1 Hz Internal Measurements
<sup>3</sup> 10 Hz Control Pad Measurements
To simply use all recording data, we can iterate through each subchannel in a dataset like so:
```python
for ch in ds.channels.values():
for sch in ch.subchannels:
print(sch)
```
#### EventArrays and raw data
Each `Channel` and `Subchannel` object has a `getSession()` method, which
returns an `EventArray` object. `EventArray` is a wrapper around a channel's
underlying recording data that loads data on demand from the source file. You
can index an `EventArray` (e.g., `eventarray[i]` for some index `i`) to get a
numpy `ndarray` of data. Data is organized in an n-dimensional array.
For subchannels, this will always be a 2-by-n array, where n is the number of
samples recorded; `eventarray[1]` indexes the samples, `eventarray[0]` indexes
the respective timestamps.
For channels, this will be a (c+1)-by-n array, where n is the number of samples
recorded and c is the number of subchannels; `eventarray[1:]` indexes the
samples, `eventarray[0]` indexes the respective timestamps.
### Getting metadata
`Dataset` makes available some basic metadata. Some useful pieces of information
are stored directly as members:
```python
>>> ds.filename
'C:\\Users\\Public\\SSX09546_019.IDE'
```
Other data is stored in the dict member recorderInfo:
```python
>>> ds.recorderInfo['RecorderSerial']
9546
>>> ds.recorderInfo['PartNumber']
'S3-E500D40'
```
`EventArray` also stores some sample-specific metadata, like the data's units:
```python
>>> eventarray.units
('Acceleration', u'g')
```
Raw data
{
"_id": null,
"home_page": "https://github.com/MideTechnology/idelib",
"name": "idelib",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": null,
"keywords": "ebml binary ide mide",
"author": "Mide Technology",
"author_email": "help@mide.com",
"download_url": "https://files.pythonhosted.org/packages/e6/ad/e4efe6a1fb4568d4e651f478cacf3fe0d6c35b612c6b846165dd5c7bce25/idelib-3.2.10.tar.gz",
"platform": null,
"description": "[](https://travis-ci.com/MideTechnology/idelib) [](https://codecov.io/gh/MideTechnology/idelib)\n\n\n# _idelib_ README\n\n_idelib_ is a Python API for accessing [enDAQ's](http://endaq.com) IDE recordings. The IDE format is\nan [EBML](http://matroska-org.github.io/libebml/specs.html) encoded file using a \ncustom schema. This library utilizes our \n[ebmlite](https://github.com/MideTechnology/ebmlite) to parse the files, and \nprovides classes that make reading data simple.\n\n## IDE File Basics\n\n### What's an IDE file?\n\nAn IDE file is a read-only hierarchical data format that stores recording \ninformation generated by an [enDAQ sensor device](http://endaq.com/collections/endaq-sensors-shock-vibration-s-series). It contains both time-indexed \ndata from several different kinds of sensors (like acceleration, pressure, \ntemperature, etc.), as well as metadata about the recording device (like device \nserial number, model number, device name, etc.) and recording settings.\n\n### Accessing an IDE file\n\nThe top-level interface for an IDE file is the `Dataset` object, through which \none can access all of the above-listed information. When you open a file for \nreading, for example, this is the type of object that is returned.\n\n#### Opening an IDE File\n\nYou can open an IDE file like so:\n\n```python\nfilename = \"your_file.IDE\"\nwith idelib.importFile(filename) as ds:\n print(type(ds))\n```\n\n**Note**: a `Dataset` object perfoms _lazy-loading_, meaning that it only loads \ninformation as is needed. As a result, it internally retains a handle to the \nsource file which after use needs to be closed. This can be accomplished by \neither using `Dataset` as a _context manager_ (as seen above; this is the \nrecommended method), or by using `Dataset` as a normal value and calling the \n`close()` method manually:\n\n```python\nfilename = \"your_file.IDE\"\nds = idelib.importFile(filename)\n# use `ds` here\nds.close() # remember to close the file after use!\n```\n\n### Getting recording data\n\n#### Channels and Subchannels\n\n\n\nIDE files organize recording data into _channels_ and _subchannels_. A channel \nencapsulates data recorded by a particular individual sensor on the device \n(e.g., XYZ acceleration from the ADXL375 DC Accelerometer); a subchannel, if \npresent, specifies a particular data stream within a channel (e.g., the \nX-coordinate acceleration from the ADXL375 DC Accelerometer).\n\nAt the top-level, a `Dataset` object has a `channels` member, which is a dict \nof all channels recorded in the file. The dict is keyed by channel id numbers, \nwith `Channel` objects in the values.\n\nEach `Channel` object has a `subchannels` member, which is a list of \n`Subchannel` objects. If the channel has no subchannels, this member will be `None`.\n\nThe below table lists current conventions for channels across all enDAQ sensors:\n\n| (Abbreviated) Product No. | Description | Example Product Nos. |\n|--------------------------:|--------------------------------------------------------------------------------|-------------------------|\n| S-D | enDAQ S-series devices with a single digital accelerometer | S3-D16, S4-D40 |\n| S-DD | enDAQ S-series devices with dual digital accelerometers | S1-D100D40, S2-D25D16 |\n| S-ED | enDAQ S-series devices with an analog piezoelectric and digital accelerometer | S5-E25D40, S4-E100D40 |\n| S-RD | enDAQ S-series devices with an analog piezoresistive and digital accelerometer | S4-R500D40, S5-R2000D40 |\n| W-D | enDAQ W-series devices with a single digital accelerometer | W5-D40 |\n| W-ED | enDAQ W-series devices with an analog piezoelectric and digital accelerometer | W8-E100D40, W8-E2000D40 |\n| W-RD | enDAQ W-series devices with an analog piezoresistive and digital accelerometer | W8-R500D40, W8-R2000D40 |\n| SSX | Mid\u00e9 Slam Stick X data recorders | SSX |\n| SSC | Mid\u00e9 Slam Stick C data recorders | SSC |\n| SSS | Mid\u00e9 Slam Stick S data recorders | SSS |\n\nThe below table lists channel ID numbers used in a recording file based on the \nrecording device\u2019s product number (device series numbers and accelerometer \nsensitivity ranges are omitted when applicable to all such devices):\n\n| Sensor | Channel | Valid Devices | Suchannels |\n|------------------------:|:--------|-----------------------------------------|------------------------------------|\n| Main Accelerometer | 8 | S-RD, S-ED, SSS, SSX | X-, Y-, Z-axis Acceleration |\n| 16/200g Accelerometer | 32 | S-DD, SSX, SSS, SSC, S-D16, S-D200 | X-, Y-, Z-axis Acceleration |\n| 8/40g Accelerometer | 80 | S-RD, S-DD, S-ED, S-D40, S-D8 | X-, Y-, Z-axis Acceleration |\n| IMU Gyroscope | 47 | All<sup>1</sup> | X-, Y-, Z-axis Rotation |\n| Absolute Orientation | 65 | All<sup>1</sup> | X-, Y-, Z-, W-axis Quaternion; Acc |\n| Relative Orientation | 70 | All<sup>1</sup> | X-, Y-, Z-, W-axis Quaternion |\n| MPL3115 | 36 | S-D16, All<sup>1</sup> before Mid-2023 | Pressure, Temperature <sup>2</sup> |\n| MS8607 Internal | 20 | All<sup>1</sup> after Mid-2023 | Pressure, Temperature, Humidity |\n| MS8607 Control Pad | 59 | All<sup>1</sup> | Pressure, Temperature, Humidity |\n| SI1133 | 76 | All<sup>1</sup> | Lux, UV |\n| BMI270/BMG250 Gyroscope | 84 | All<sup>1</sup> after Mid-2023 | X-, Y-, Z-axis Rotation |\n| CAM-M8Q GPS | 88 | W-D, W-ED, W-RD | Latitude, Longitude, Time, Speed |\n\n<sup>1</sup> excluding early SSC/SSS/SSX models\n\n<sup>2</sup> 1 Hz Internal Measurements\n\n<sup>3</sup> 10 Hz Control Pad Measurements\n\nTo simply use all recording data, we can iterate through each subchannel in a dataset like so:\n\n```python\nfor ch in ds.channels.values():\n for sch in ch.subchannels:\n print(sch)\n```\n\n#### EventArrays and raw data\n\nEach `Channel` and `Subchannel` object has a `getSession()` method, which \nreturns an `EventArray` object. `EventArray` is a wrapper around a channel's \nunderlying recording data that loads data on demand from the source file. You \ncan index an `EventArray` (e.g., `eventarray[i]` for some index `i`) to get a \nnumpy `ndarray` of data. Data is organized in an n-dimensional array.\n\nFor subchannels, this will always be a 2-by-n array, where n is the number of \nsamples recorded; `eventarray[1]` indexes the samples, `eventarray[0]` indexes \nthe respective timestamps.\n\nFor channels, this will be a (c+1)-by-n array, where n is the number of samples \nrecorded and c is the number of subchannels; `eventarray[1:]` indexes the \nsamples, `eventarray[0]` indexes the respective timestamps.\n\n### Getting metadata\n\n`Dataset` makes available some basic metadata. Some useful pieces of information \nare stored directly as members:\n\n```python\n>>> ds.filename\n'C:\\\\Users\\\\Public\\\\SSX09546_019.IDE'\n```\n\nOther data is stored in the dict member recorderInfo:\n\n```python\n>>> ds.recorderInfo['RecorderSerial']\n9546\n>>> ds.recorderInfo['PartNumber']\n'S3-E500D40'\n```\n\n`EventArray` also stores some sample-specific metadata, like the data's units:\n\n```python\n>>> eventarray.units\n('Acceleration', u'g')\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python API for accessing IDE data recordings",
"version": "3.2.10",
"project_urls": {
"Bug Tracker": "https://github.com/MideTechnology/idelib/issues",
"Documentation": "https://mide-technology-idelib.readthedocs-hosted.com/en/latest/",
"Homepage": "https://github.com/MideTechnology/idelib",
"Source Code": "https://github.com/MideTechnology/idelib"
},
"split_keywords": [
"ebml",
"binary",
"ide",
"mide"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ff059194f3fba3637168a193b30538a142261a3bb61b87e832392fd4291d7f80",
"md5": "3efdcf127c57701806963efcb26e5945",
"sha256": "a9cdccaa57f9bc7d8e595a731d4a8a43aaed70c05b67262372234625fdd17d1e"
},
"downloads": -1,
"filename": "idelib-3.2.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3efdcf127c57701806963efcb26e5945",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 99668,
"upload_time": "2024-06-25T13:24:04",
"upload_time_iso_8601": "2024-06-25T13:24:04.415272Z",
"url": "https://files.pythonhosted.org/packages/ff/05/9194f3fba3637168a193b30538a142261a3bb61b87e832392fd4291d7f80/idelib-3.2.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e6ade4efe6a1fb4568d4e651f478cacf3fe0d6c35b612c6b846165dd5c7bce25",
"md5": "5b35a437d072c3af4a48f31dd74e3876",
"sha256": "a80eba8791377cc8f50530a8d6d96840644a8349019ad8958f826a8b1a40d672"
},
"downloads": -1,
"filename": "idelib-3.2.10.tar.gz",
"has_sig": false,
"md5_digest": "5b35a437d072c3af4a48f31dd74e3876",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 97622,
"upload_time": "2024-06-25T13:24:07",
"upload_time_iso_8601": "2024-06-25T13:24:07.049210Z",
"url": "https://files.pythonhosted.org/packages/e6/ad/e4efe6a1fb4568d4e651f478cacf3fe0d6c35b612c6b846165dd5c7bce25/idelib-3.2.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-25 13:24:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MideTechnology",
"github_project": "idelib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "numpy",
"specs": [
[
">=",
"1.16.6"
]
]
},
{
"name": "ebmlite",
"specs": [
[
">=",
"3.1.0"
]
]
},
{
"name": "pytest",
"specs": [
[
">=",
"4.6"
]
]
},
{
"name": "pytest-xdist",
"specs": []
},
{
"name": "mock",
"specs": []
},
{
"name": "pytest-cov",
"specs": []
},
{
"name": "sphinx",
"specs": [
[
">=",
"5.0.2"
]
]
},
{
"name": "scipy",
"specs": []
},
{
"name": "setuptools",
"specs": []
}
],
"lcname": "idelib"
}
JSON
