# **EdgeModelKit**: Sensor Data Acquisition and Logging Library
EdgeModelKit is a Python library developed by **EdgeNeuron**, designed to simplify sensor data acquisition, logging, and real-time processing for IoT devices. It works seamlessly with the **DataLogger script** from the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron), making it ideal for edge computing and machine learning applications.
---
## **Features**
- **Serial Communication**: Supports data acquisition over serial ports with robust error handling.
- **Flexible Data Fetching**: Retrieve sensor data as Python lists or NumPy arrays.
- **Customizable Logging**: Log sensor data into CSV files with optional timestamps and counters.
- **Class-Based Organization**: Log data with class labels to prepare datasets for machine learning tasks.
- **Custom Preprocessing**: Apply custom preprocessing functions to sensor data before logging or inference.
- **Error Handling**: Gracefully handles data decoding errors and missing keys in sensor data packets.
---
## **Usage Prerequisites**
This library is designed to work in conjunction with the **DataLogger script** available in the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron). The DataLogger script configures your Arduino-based IoT device to send structured JSON sensor data over a serial connection.
Before using EdgeModelKit, ensure:
1. Your Arduino device is programmed with the **DataLogger script** from the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron).
2. The device is connected to your system via a serial interface.
---
## **Installation**
Install EdgeModelKit using pip:
```bash
pip install edgemodelkit
```
---
## **Quick Start**
### **1. Initialize the DataFetcher**
```python
from edgemodelkit import DataFetcher
# Initialize the DataFetcher with the desired serial port and baud rate
fetcher = DataFetcher(serial_port="COM3", baud_rate=9600)
```
### **2. Fetch Sensor Data**
```python
# Fetch data as a Python list
sensor_data = fetcher.fetch_data(return_as_numpy=False)
print("Sensor Data:", sensor_data)
# Fetch data as a NumPy array
sensor_data_numpy = fetcher.fetch_data(return_as_numpy=True)
print("Sensor Data (NumPy):", sensor_data_numpy)
```
### **3. Log Sensor Data**
```python
# Log 10 samples to a CSV file with timestamp and count columns
fetcher.log_sensor_data(class_label="ClassA", num_samples=10, add_timestamp=True, add_count=True)
```
---
## **CSV Logging Details**
The CSV file is generated automatically based on the sensor name (e.g., `TemperatureSensor_data_log.csv`) and contains the following:
- **Timestamp**: (Optional) Records the time when the data was logged.
- **Sample Count**: (Optional) A sequential counter for each data sample.
- **Data Columns**: Each element in the sensor data array is stored in separate columns (e.g., `data_value_1`, `data_value_2`, ...).
The data is saved under a folder named `Dataset`, with subfolders organized by `class_label` (if specified).
---
## **Real-Time Data Processing Example**
```python
from edgemodelkit import DataFetcher
fetcher = DataFetcher(serial_port="COM3", baud_rate=9600)
def custom_preprocess(data):
# Example: Normalize the data
return (data - min(data)) / (max(data) - min(data))
try:
while True:
# Fetch data as NumPy array
sensor_data = fetcher.fetch_data(return_as_numpy=True)
print("Received Data (Raw):", sensor_data)
# Apply custom preprocessing
processed_data = custom_preprocess(sensor_data)
print("Preprocessed Data:", processed_data)
# Perform custom processing (e.g., feed to a TensorFlow model)
# prediction = model.predict(processed_data)
# print("Prediction:", prediction)
finally:
fetcher.close_connection()
```
---
## **Using ModelPlayGround**
### **1. Initialize the ModelPlayGround**
```python
from edgemodelkit import ModelPlayGround
# Initialize the ModelPlayGround with the path to your .keras model
playground = ModelPlayGround()
playground.load_model(model_path="path_to_your_model.keras")
```
### **2. View Model Summary**
```python
# Display the model architecture
playground.model_summary()
```
### **3. View Model Statistics**
```python
# View model size and number of parameters
playground.model_stats()
```
### **4. Convert Model to TensorFlow Lite**
```python
# Convert the model to TFLite format with default quantization
playground.model_converter(quantization_type="default")
# Convert the model to TFLite format with float16 quantization
playground.model_converter(quantization_type="float16")
# Convert the model to TFLite format with int8 quantization
playground.model_converter(quantization_type="int8")
```
### **5. Test TFLite Model on Live Data**
```python
from edgemodelkit import DataFetcher
# Initialize a DataFetcher
fetcher = DataFetcher(serial_port="COM3", baud_rate=9600)
def custom_preprocess(data):
# Example: Normalize the data
return (data - min(data)) / (max(data) - min(data))
# Perform live testing of the TFLite model
playground_output = playground.edge_testing(
tflite_model_path="path_to_tflite_model.tflite",
data_fetcher=fetcher,
preprocess_func=custom_preprocess
)
print("Model Prediction: ", playground_output['ModelOutput'])
print("Sensor data: ", playground_output['SensorData'])
```
---
## **Disclaimer**
Currently, the `ModelPlayGround` class supports only `.keras` models for conversion and testing. Support for other model formats may be added in future updates.
---
## **Contributing**
We welcome contributions to EdgeModelKit! Feel free to submit bug reports, feature requests, or pull requests on our [GitHub repository](https://github.com/ConsentiumIoT/edgemodelkit).
---
## **License**
EdgeModelKit is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
---
## **Support**
For support and inquiries, contact us at **support@edgeneuronai.com** or visit our [GitHub repository](https://github.com/ConsentiumIoT/edgemodelkit).
---
## **About EdgeNeuron**
EdgeNeuron is a pioneer in edge computing solutions, enabling developers to build intelligent IoT applications with state-of-the-art tools and libraries. Learn more at [edgeneuronai.com](https://edgeneuronai.com).
Raw data
{
"_id": null,
"home_page": "https://github.com/ConsentiumIoT/edgemodelkit",
"name": "edgemodelkit",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "Python, Edge Computing, IoT, Sensor Data, Data Logging, EdgeNeuron, edgemodelkit, Machine Learning",
"author": "EdgeNeuron",
"author_email": "official@consentiumiot.com",
"download_url": "https://files.pythonhosted.org/packages/d0/8b/9f4440637c85ef9c95dce519025f4c26bd4c4c3330eaf09dfc5abee4fd5b/edgemodelkit-1.1.6.tar.gz",
"platform": null,
"description": "# **EdgeModelKit**: Sensor Data Acquisition and Logging Library\n\nEdgeModelKit is a Python library developed by **EdgeNeuron**, designed to simplify sensor data acquisition, logging, and real-time processing for IoT devices. It works seamlessly with the **DataLogger script** from the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron), making it ideal for edge computing and machine learning applications.\n\n---\n\n## **Features**\n\n- **Serial Communication**: Supports data acquisition over serial ports with robust error handling. \n- **Flexible Data Fetching**: Retrieve sensor data as Python lists or NumPy arrays. \n- **Customizable Logging**: Log sensor data into CSV files with optional timestamps and counters. \n- **Class-Based Organization**: Log data with class labels to prepare datasets for machine learning tasks. \n- **Custom Preprocessing**: Apply custom preprocessing functions to sensor data before logging or inference. \n- **Error Handling**: Gracefully handles data decoding errors and missing keys in sensor data packets. \n\n---\n\n## **Usage Prerequisites**\n\nThis library is designed to work in conjunction with the **DataLogger script** available in the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron). The DataLogger script configures your Arduino-based IoT device to send structured JSON sensor data over a serial connection.\n\nBefore using EdgeModelKit, ensure: \n1. Your Arduino device is programmed with the **DataLogger script** from the [EdgeNeuron Arduino library](https://github.com/ConsentiumIoT/EdgeNeuron). \n2. The device is connected to your system via a serial interface. \n\n---\n\n## **Installation**\n\nInstall EdgeModelKit using pip:\n\n```bash\npip install edgemodelkit\n```\n\n---\n\n## **Quick Start**\n\n### **1. Initialize the DataFetcher**\n\n```python\nfrom edgemodelkit import DataFetcher\n\n# Initialize the DataFetcher with the desired serial port and baud rate\nfetcher = DataFetcher(serial_port=\"COM3\", baud_rate=9600)\n```\n\n### **2. Fetch Sensor Data**\n\n```python\n# Fetch data as a Python list\nsensor_data = fetcher.fetch_data(return_as_numpy=False)\nprint(\"Sensor Data:\", sensor_data)\n\n# Fetch data as a NumPy array\nsensor_data_numpy = fetcher.fetch_data(return_as_numpy=True)\nprint(\"Sensor Data (NumPy):\", sensor_data_numpy)\n```\n\n### **3. Log Sensor Data**\n\n```python\n# Log 10 samples to a CSV file with timestamp and count columns\nfetcher.log_sensor_data(class_label=\"ClassA\", num_samples=10, add_timestamp=True, add_count=True)\n```\n\n---\n\n## **CSV Logging Details**\n\nThe CSV file is generated automatically based on the sensor name (e.g., `TemperatureSensor_data_log.csv`) and contains the following: \n\n- **Timestamp**: (Optional) Records the time when the data was logged. \n- **Sample Count**: (Optional) A sequential counter for each data sample. \n- **Data Columns**: Each element in the sensor data array is stored in separate columns (e.g., `data_value_1`, `data_value_2`, ...). \n\nThe data is saved under a folder named `Dataset`, with subfolders organized by `class_label` (if specified). \n\n---\n\n## **Real-Time Data Processing Example**\n\n```python\nfrom edgemodelkit import DataFetcher\n\nfetcher = DataFetcher(serial_port=\"COM3\", baud_rate=9600)\n\ndef custom_preprocess(data):\n # Example: Normalize the data\n return (data - min(data)) / (max(data) - min(data))\n\ntry:\n while True:\n # Fetch data as NumPy array\n sensor_data = fetcher.fetch_data(return_as_numpy=True)\n print(\"Received Data (Raw):\", sensor_data)\n\n # Apply custom preprocessing\n processed_data = custom_preprocess(sensor_data)\n print(\"Preprocessed Data:\", processed_data)\n\n # Perform custom processing (e.g., feed to a TensorFlow model)\n # prediction = model.predict(processed_data)\n # print(\"Prediction:\", prediction)\nfinally:\n fetcher.close_connection()\n```\n\n---\n\n## **Using ModelPlayGround**\n\n### **1. Initialize the ModelPlayGround**\n\n```python\nfrom edgemodelkit import ModelPlayGround\n\n# Initialize the ModelPlayGround with the path to your .keras model\nplayground = ModelPlayGround()\nplayground.load_model(model_path=\"path_to_your_model.keras\")\n```\n\n### **2. View Model Summary**\n\n```python\n# Display the model architecture\nplayground.model_summary()\n```\n\n### **3. View Model Statistics**\n\n```python\n# View model size and number of parameters\nplayground.model_stats()\n```\n\n### **4. Convert Model to TensorFlow Lite**\n\n```python\n# Convert the model to TFLite format with default quantization\nplayground.model_converter(quantization_type=\"default\")\n\n# Convert the model to TFLite format with float16 quantization\nplayground.model_converter(quantization_type=\"float16\")\n\n# Convert the model to TFLite format with int8 quantization\nplayground.model_converter(quantization_type=\"int8\")\n```\n\n### **5. Test TFLite Model on Live Data**\n\n```python\nfrom edgemodelkit import DataFetcher\n\n# Initialize a DataFetcher\nfetcher = DataFetcher(serial_port=\"COM3\", baud_rate=9600)\n\ndef custom_preprocess(data):\n # Example: Normalize the data\n return (data - min(data)) / (max(data) - min(data))\n\n# Perform live testing of the TFLite model\nplayground_output = playground.edge_testing(\n tflite_model_path=\"path_to_tflite_model.tflite\",\n data_fetcher=fetcher,\n preprocess_func=custom_preprocess\n)\nprint(\"Model Prediction: \", playground_output['ModelOutput'])\nprint(\"Sensor data: \", playground_output['SensorData'])\n```\n\n---\n\n## **Disclaimer**\n\nCurrently, the `ModelPlayGround` class supports only `.keras` models for conversion and testing. Support for other model formats may be added in future updates.\n\n---\n\n## **Contributing**\n\nWe welcome contributions to EdgeModelKit! Feel free to submit bug reports, feature requests, or pull requests on our [GitHub repository](https://github.com/ConsentiumIoT/edgemodelkit).\n\n---\n\n## **License**\n\nEdgeModelKit is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n---\n\n## **Support**\n\nFor support and inquiries, contact us at **support@edgeneuronai.com** or visit our [GitHub repository](https://github.com/ConsentiumIoT/edgemodelkit).\n\n---\n\n## **About EdgeNeuron**\n\nEdgeNeuron is a pioneer in edge computing solutions, enabling developers to build intelligent IoT applications with state-of-the-art tools and libraries. Learn more at [edgeneuronai.com](https://edgeneuronai.com).\n\n",
"bugtrack_url": null,
"license": null,
"summary": "edgemodelkit: A Python library for seamless sensor data acquisition and logging.",
"version": "1.1.6",
"project_urls": {
"Bug Tracker": "https://github.com/ConsentiumIoT/edgemodelkit/issues",
"Documentation": "https://github.com/ConsentiumIoT/edgemodelkit#readme",
"Homepage": "https://github.com/ConsentiumIoT/edgemodelkit",
"Source Code": "https://github.com/ConsentiumIoT/edgemodelkit"
},
"split_keywords": [
"python",
" edge computing",
" iot",
" sensor data",
" data logging",
" edgeneuron",
" edgemodelkit",
" machine learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "761538988dae430899fc0b22baf14136ef1ba0ebfd39698e0aea278419f82c88",
"md5": "c6ebea5970daf75841b24a860576228f",
"sha256": "b0408cb54607fc78019d5f4f748427d6f8dfdfdc08d3390b86c6f9faea0089fe"
},
"downloads": -1,
"filename": "edgemodelkit-1.1.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c6ebea5970daf75841b24a860576228f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 8546,
"upload_time": "2025-01-12T14:19:07",
"upload_time_iso_8601": "2025-01-12T14:19:07.423527Z",
"url": "https://files.pythonhosted.org/packages/76/15/38988dae430899fc0b22baf14136ef1ba0ebfd39698e0aea278419f82c88/edgemodelkit-1.1.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d08b9f4440637c85ef9c95dce519025f4c26bd4c4c3330eaf09dfc5abee4fd5b",
"md5": "adc7d08d906c2f6e93be9a75baf41d9e",
"sha256": "e8211cb3660a85741c650ebdc8ae87e17df470ceb9cad4c4215ef089b1fa8157"
},
"downloads": -1,
"filename": "edgemodelkit-1.1.6.tar.gz",
"has_sig": false,
"md5_digest": "adc7d08d906c2f6e93be9a75baf41d9e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 9210,
"upload_time": "2025-01-12T14:19:08",
"upload_time_iso_8601": "2025-01-12T14:19:08.645425Z",
"url": "https://files.pythonhosted.org/packages/d0/8b/9f4440637c85ef9c95dce519025f4c26bd4c4c3330eaf09dfc5abee4fd5b/edgemodelkit-1.1.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-12 14:19:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ConsentiumIoT",
"github_project": "edgemodelkit",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "edgemodelkit"
}
JSON
