# emon_tools
[](https://pypi.org/project/emon_tools/)
[](https://codecov.io/gh/vemonitor/emon_tools)
[](https://pepy.tech/project/emon_tools)
`emon-tools` is a Python library that provides tools and APIs for interacting with [EmonCMS](https://emoncms.org/) and processing time-series data. It is designed to simplify data retrieval, analysis, and validation, making it easier to work with energy monitoring data.
## Installation
### Global Installation
To install all modules and dependencies globally:
1. Via Pip
```
pip install emon-tools
```
### Module-Specific Installation
You can install specific modules and their dependencies as needed. For example:
- To enable pandas time-series output functionality:
```
pip install emon-tools[time_series]
```
- To include graph plotting capabilities:
```
pip install emon-tools[plot]
```
## Modules
`emon_tools` is modular, allowing you to install and use individual components as needed.
### 1. emon_fina
The `emon_fina` module facilitates the analysis and processing of time-series data, particularly from PhpFina file formats.
#### Features
- Data Reading: Efficiently read data from PhpFina file formats.
- Time-Series Analysis: Compute daily statistics such as min, max, mean, and more.
- Filtering: Validate and filter data based on custom thresholds.
- Utilities: Timestamp manipulation and interval computation tools.
#### Usage Example:
The examples below demonstrate how to retrieve and analyze data from PhpFina timeseries .dat files. For additional examples, refer to the [`emon_fina` Jupiter NoteBook](https://github.com/vemonitor/emon_tools/blob/main/notebook/emon_fina.ipynb).
Every PhpFina timeseries feed engine is acompagned with `.meta` file who contain meta values of actual status of `.dat` file. Meta data is readed on initialize objects
##### Retrieving data
`FinaData` initialization:
```python
from emon_tools.emon_fina import FinaData
fdf = FinaData(
feed_id=1,
data_dir="/path/to/phpfina/files
)
```
Values output can be set as:
> In above example we get 8 days (8 * 24 * 3600) from meta `time_start` value.
1. 1D numpy array by timestamp
Retrieve data values from the Fina data file for a specified time window.
```python
values = fdf.get_fina_values(
start=fr.meta.start_time,
step=10,
window=8 * 24 * 3600
)
```
2. 1D numpy array by srting datetime
Retrieve values from the Fina data file based on a specified date range.
```python
ts = fdf.get_fina_values_by_date(
start_date='2019-12-12 00:00:00',
end_date='2019-12-13 00:00:00',
step=10
)
```
3. 2D TimeSeries numpy array by timestamp
Retrieve a 2D time series array of timestamps and values from the Fina data file.
```python
ts = fdf.get_fina_time_series(
start=fr.meta.start_time,
step=10,
window=8 * 24 * 3600
)
```
4. 2D TimeSeries numpy array by srting datetime
Retrieve a 2D time series array of timestamps and values for a specific date range.
```python
ts = fdf.get_fina_time_series_by_date(
start_date='2019-12-12 00:00:00',
end_date='2019-12-13 00:00:00',
step=10
)
```
5. pandas DataFrame TimeSeries
`FinaDataFrame` initialization:
```python
from emon_tools.fina_time_series import FinaDataFrame
fdf = FinaDataFrame(
feed_id=1,
data_dir="/path/to/phpfina/files
)
```
Retrieve time series data within a specified time window
and return it as a Pandas DataFrame.
```python
ts = fdf.get_fina_df_time_series(
start=fr.meta.start_time,
step=10,
window=8 * 24 * 3600
)
```
Retrieve time series data by specifying a date range and convert it to a Pandas DataFrame.
```python
ts = fdf.get_fina_time_series_by_date(
start_date='2019-12-12 00:00:00',
end_date='2019-12-13 00:00:00',
step=10
)
```
And optionaly ploted dirrectly.
`FinaDataFrame` initialization:
```python
from emon_tools.fina_plot import PlotData
PlotData.plot(data=ts)
```
##### Compute Daily Statistics
`FinaDataFrame` initialization:
```python
from emon_tools.emon_fina import FinaStats
from emon_tools.emon_fina import StatsType
stats = FinaStats(
feed_id=1,
data_dir="/path/to/phpfina/files
)
```
Once initialized, you can access the metadata of the PhpFina `.meta` file. For example, a file with `feed_id=1` might return:
```python
stats.meta
{
"interval": 10,
"start_time": 1575981140,
"npoints": 4551863,
"end_time": 1621499760
}
```
On grabing phpfina timeseries feed engine, missed data points are set as Nan values,
We can get file integrity daily statistics to compute real and total values of phpfina `.dat` file
```python
# Compute daily statistics
daily_stats = stats.get_stats(stats_type=StatsType.INTEGRITY)
```
<img src="https://github.com/vemonitor/emon_tools/blob/main/img/integrity_stats.png" with="100%">
Or we can get daily values statistics from your phpfina timeseries feed engine file
```python
# Compute daily statistics
daily_stats = stats.get_stats(stats_type=StatsType.VALUES)
```
<img src="https://github.com/vemonitor/emon_tools/blob/main/img/values_stats.png" with="100%">
Phpfina timeseries feed engine file can contain bad data, in this case we can limit values from statistics without bad values.
Here statistics are calculated only with values between -50 and 50.
```python
# Compute daily statistics
daily_stats = stats.get_stats(
stats_type=StatsType.VALUES,
min_value=-50,
max_value=50
)
```
<img src="https://github.com/vemonitor/emon_tools/blob/main/img/values_stats_limited.png" with="100%">
You can limit daily statistics from desired window, by setting `start_time` and/or `steps_window` properties.
In above example we get daily stats values for 8 days from timestamp value 1575981140
```python
# Compute daily statistics
daily_stats = stats.get_stats(
start_time=1575981140,
steps_window=8 * 24 * 3600,
stats_type=StatsType.VALUES,
min_value=-50,
max_value=50
)
```
## Running Tests
To ensure everything is functioning correctly, run the test suite:
```
pytest -v
```
## Contributing
Contributions are welcome! To contribute:
1. Fork the repository.
2. Create a feature branch.
3. Submit a pull request with a clear description.
## License
This project is licensed under the MIT License. See LICENSE for more details.
....
Raw data
{
"_id": null,
"home_page": "https://github.com/vemonitor/emon-tools",
"name": "emon-tools",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "EmonCms, Python, PHPFINA",
"author": "Eli Serra",
"author_email": "mano8@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/75/13/efe7502ff24220d31a31bbf5e7bd63ebd1262e0e00a8f11b4fdc023c5a99/emon_tools-0.1.1.tar.gz",
"platform": null,
"description": "# emon_tools\n\n\n[](https://pypi.org/project/emon_tools/)\n[](https://codecov.io/gh/vemonitor/emon_tools)\n[](https://pepy.tech/project/emon_tools)\n\n`emon-tools` is a Python library that provides tools and APIs for interacting with [EmonCMS](https://emoncms.org/) and processing time-series data. It is designed to simplify data retrieval, analysis, and validation, making it easier to work with energy monitoring data.\n\n## Installation\n\n### Global Installation\nTo install all modules and dependencies globally:\n1. Via Pip\n```\npip install emon-tools\n```\n\n### Module-Specific Installation\n\nYou can install specific modules and their dependencies as needed. For example: \n- To enable pandas time-series output functionality:\n\n```\npip install emon-tools[time_series]\n```\n\n- To include graph plotting capabilities:\n\n```\npip install emon-tools[plot]\n```\n\n## Modules\n`emon_tools` is modular, allowing you to install and use individual components as needed.\n\n### 1. emon_fina\n\nThe `emon_fina` module facilitates the analysis and processing of time-series data, particularly from PhpFina file formats.\n\n#### Features\n\n - Data Reading: Efficiently read data from PhpFina file formats.\n - Time-Series Analysis: Compute daily statistics such as min, max, mean, and more.\n - Filtering: Validate and filter data based on custom thresholds.\n - Utilities: Timestamp manipulation and interval computation tools.\n\n#### Usage Example:\n\nThe examples below demonstrate how to retrieve and analyze data from PhpFina timeseries .dat files. For additional examples, refer to the [`emon_fina` Jupiter NoteBook](https://github.com/vemonitor/emon_tools/blob/main/notebook/emon_fina.ipynb).\n\nEvery PhpFina timeseries feed engine is acompagned with `.meta` file who contain meta values of actual status of `.dat` file. Meta data is readed on initialize objects\n\n##### Retrieving data\n\n`FinaData` initialization:\n\n```python\nfrom emon_tools.emon_fina import FinaData\n\nfdf = FinaData(\n feed_id=1,\n data_dir=\"/path/to/phpfina/files\n)\n```\n\nValues output can be set as:\n> In above example we get 8 days (8 * 24 * 3600) from meta `time_start` value.\n\n1. 1D numpy array by timestamp\n\nRetrieve data values from the Fina data file for a specified time window.\n\n```python\nvalues = fdf.get_fina_values(\n start=fr.meta.start_time,\n step=10,\n window=8 * 24 * 3600\n)\n```\n\n2. 1D numpy array by srting datetime\n\nRetrieve values from the Fina data file based on a specified date range.\n\n```python\nts = fdf.get_fina_values_by_date(\n start_date='2019-12-12 00:00:00',\n end_date='2019-12-13 00:00:00',\n step=10\n)\n```\n\n3. 2D TimeSeries numpy array by timestamp\n\nRetrieve a 2D time series array of timestamps and values from the Fina data file.\n\n```python\nts = fdf.get_fina_time_series(\n start=fr.meta.start_time,\n step=10,\n window=8 * 24 * 3600\n)\n```\n\n\n\n4. 2D TimeSeries numpy array by srting datetime\n\nRetrieve a 2D time series array of timestamps and values for a specific date range.\n\n```python\nts = fdf.get_fina_time_series_by_date(\n start_date='2019-12-12 00:00:00',\n end_date='2019-12-13 00:00:00',\n step=10\n)\n```\n\n5. pandas DataFrame TimeSeries\n\n`FinaDataFrame` initialization:\n\n```python\nfrom emon_tools.fina_time_series import FinaDataFrame\n\nfdf = FinaDataFrame(\n feed_id=1,\n data_dir=\"/path/to/phpfina/files\n)\n```\n\nRetrieve time series data within a specified time window\nand return it as a Pandas DataFrame.\n\n```python\nts = fdf.get_fina_df_time_series(\n start=fr.meta.start_time,\n step=10,\n window=8 * 24 * 3600\n)\n```\n\nRetrieve time series data by specifying a date range and convert it to a Pandas DataFrame.\n\n```python\nts = fdf.get_fina_time_series_by_date(\n start_date='2019-12-12 00:00:00',\n end_date='2019-12-13 00:00:00',\n step=10\n)\n```\nAnd optionaly ploted dirrectly.\n\n`FinaDataFrame` initialization:\n\n```python\nfrom emon_tools.fina_plot import PlotData\n\nPlotData.plot(data=ts)\n```\n\n##### Compute Daily Statistics\n\n`FinaDataFrame` initialization:\n\n```python\nfrom emon_tools.emon_fina import FinaStats\nfrom emon_tools.emon_fina import StatsType\n\nstats = FinaStats(\n feed_id=1,\n data_dir=\"/path/to/phpfina/files\n)\n```\n\nOnce initialized, you can access the metadata of the PhpFina `.meta` file. For example, a file with `feed_id=1` might return:\n\n```python\nstats.meta\n {\n \"interval\": 10,\n \"start_time\": 1575981140,\n \"npoints\": 4551863,\n \"end_time\": 1621499760\n }\n```\n\nOn grabing phpfina timeseries feed engine, missed data points are set as Nan values,\nWe can get file integrity daily statistics to compute real and total values of phpfina `.dat` file\n\n```python\n# Compute daily statistics\ndaily_stats = stats.get_stats(stats_type=StatsType.INTEGRITY)\n```\n\n<img src=\"https://github.com/vemonitor/emon_tools/blob/main/img/integrity_stats.png\" with=\"100%\">\n\nOr we can get daily values statistics from your phpfina timeseries feed engine file\n\n```python\n# Compute daily statistics\ndaily_stats = stats.get_stats(stats_type=StatsType.VALUES)\n```\n\n<img src=\"https://github.com/vemonitor/emon_tools/blob/main/img/values_stats.png\" with=\"100%\">\n\nPhpfina timeseries feed engine file can contain bad data, in this case we can limit values from statistics without bad values.\nHere statistics are calculated only with values between -50 and 50.\n\n```python\n# Compute daily statistics\ndaily_stats = stats.get_stats(\n stats_type=StatsType.VALUES,\n min_value=-50,\n max_value=50\n)\n```\n\n<img src=\"https://github.com/vemonitor/emon_tools/blob/main/img/values_stats_limited.png\" with=\"100%\">\n\nYou can limit daily statistics from desired window, by setting `start_time` and/or `steps_window` properties.\nIn above example we get daily stats values for 8 days from timestamp value 1575981140\n```python\n# Compute daily statistics\ndaily_stats = stats.get_stats(\n start_time=1575981140,\n steps_window=8 * 24 * 3600,\n stats_type=StatsType.VALUES,\n min_value=-50,\n max_value=50\n)\n```\n\n## Running Tests\n\nTo ensure everything is functioning correctly, run the test suite:\n\n```\npytest -v\n```\n\n## Contributing\n\nContributions are welcome! To contribute:\n1. Fork the repository.\n2. Create a feature branch.\n3. Submit a pull request with a clear description.\n\n## License\nThis project is licensed under the MIT License. See LICENSE for more details.\n\n....\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A numpy subclass to read emoncms PHPFINA feeds as numpy array",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/vemonitor/emon-tools"
},
"split_keywords": [
"emoncms",
" python",
" phpfina"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "263b271b3cce9e1695aa0f4d58b09a3cdbd190fcc832d7218846206a38f5dc27",
"md5": "e48fb200c9f05be54265465bf9825a43",
"sha256": "a2a11a0dd0d4e9a4ffcc24f3c2c070ef0ee319bfb2842e5e661e98f9f032d1f4"
},
"downloads": -1,
"filename": "emon_tools-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e48fb200c9f05be54265465bf9825a43",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 37082,
"upload_time": "2024-12-29T06:34:21",
"upload_time_iso_8601": "2024-12-29T06:34:21.619205Z",
"url": "https://files.pythonhosted.org/packages/26/3b/271b3cce9e1695aa0f4d58b09a3cdbd190fcc832d7218846206a38f5dc27/emon_tools-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7513efe7502ff24220d31a31bbf5e7bd63ebd1262e0e00a8f11b4fdc023c5a99",
"md5": "6143809e416a927b5cf1b8b586d554b0",
"sha256": "29589cdd4f307e6e44d57902d6843f76b39dd564b0bf8d5234337b8e48966a13"
},
"downloads": -1,
"filename": "emon_tools-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "6143809e416a927b5cf1b8b586d554b0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 32486,
"upload_time": "2024-12-29T06:34:23",
"upload_time_iso_8601": "2024-12-29T06:34:23.744143Z",
"url": "https://files.pythonhosted.org/packages/75/13/efe7502ff24220d31a31bbf5e7bd63ebd1262e0e00a8f11b4fdc023c5a99/emon_tools-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-29 06:34:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "vemonitor",
"github_project": "emon-tools",
"github_not_found": true,
"lcname": "emon-tools"
}
JSON
