# PIdata
An easy to use python package for extracting data from an OSI PI historian/server via the OSI PI SDK. Based on this blog post: https://pisquare.osisoft.com/people/rborges/blog/2016/08/01/pi-and-python-pithon
## What makes PIdata great:
1. Written in python/ironpython
2. Simple to use (specifically for people new to python)
3. Customisable (exposing the PI SDK as python functions/objects)
4. Integrates well with existing python handling tools (e.g. pandas dfs)
5. Does not reinvent existing tools as these are what makes python great
## Prerequisites:
OSI PI installed on your machine, connected the historian (probably already the case if you were looking for this package)
## Installation:
pip install PIdata
## Basic usage:
To pull hourly averages for the '24T1345.PV' tag:
import pidata
df = pidata.pull.aggregated_vals(['24T1345.PV'], start_time='1/1/2020', end_time='2/2/2020', interval='1h')
## Similar projects:
FernandoRodriguezP/OSIsoftPy
onamission21/AF-SDK-for-Python
alyasaud/PITHON
## Contributing
I find this code very useful. If you do too, please star the repo on github and share with your colleagues.
Many thanks to the people that have contributed their code thus far. Please feel free to submit a pull request, report a bug or request a feature.
## Functions: Data pull functions (`pidata.pull`)
#### `pidata.pull.aggregated_vals`
Will return a pandas dataframe of aggregated values (averaged values by default) between `start_time` and `end_time`, within the given interval
Arguments:
tags : list or list like
start_time : Time of the first data point. Default: '-30d' (thirty days ago)
end_time : Time of the last data point. Default: '' (empty/current time)
interval : Time between data points. Default: '12h'
method : Instead of returning the average value over the interval, the returned values can be specified as one of the following:
Total
Average (Default)
Minimum
Maximum
Range
StdDev - Standard deviation.
PopulationStdDev - Population standard deviation.
Count
PercentGood - Percentage of data with good value.
TotalWithUOM
All
AllForNonNumeric
Please see: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/T_OSIsoft_AF_Data_AFSummaryTypes.htm
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.recorded_vals`
Will return a pandas dataframe of recorded vals between `start_time` and `end_time`, with an the given interval
Arguments:
tags : list or list like
start_time : Time of the first data point. Default: '-30d' (thirty days ago)
end_time : Time of the last data point. Default: '' (empty/current time)
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.interp_vals`
Will return a pandas dataframe of averaged vals between `start_time` and `end_time`, with an the given interval
Arguments:
tags : list or list like
start_time : Time of the first data point. Default: '-30d' (thirty days ago)
end_time : Time of the last data point. Default: '' (empty/current time)
interval : Time between data points. Default: '12h'
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.current_vals`
Returns the last recorded values at the time of running the function
Arguments:
tags : list or list like
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.batch_aggregated_vals`
Puprose: fetch large averaged data in batches
Arguments:
tags : list of tags to download
start_time : start date time in string format where batch fetch begin
end_time : end data time in string format where batch ends
interval : period over which to average data e.g. '4H', '2D'
method : aggregation method that will be given to aggregated_vals
period : time period to define batch size e.g. 'days','months'
increment : number of time periods in a batch
verbose : verbose output of progress (default = False)
save_csv : save progress files. Default is False.
filename : name of file without the extension. Function will add suffix
return_df : whether or not to return the data as a pandas dataframe (default=True)
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.batch_recorded_vals`
Puprose: fetch large averaged data in batches
Arguments:
tags : list of tags to download
start_time : start date time in string format where batch fetch begin
end_time : end data time in string format where batch ends
period : time period to define batch size e.g. 'days','months'
increment : number of time periods in a batch
verbose : verbose output of progress (default = False)
save_csv : save progress files. Default is False.
filename : name of file without the extension. Function will add suffix
return_df : whether or not to return the data as a pandas dataframe (default=True)
server : Name of the PI server to use. Uses the default if none is provided
#### `pidata.pull.recorded_vals_dict`
A dictionary version of the recorded vals function for better efficiency for large amounts of data.
#### `pidata.pull.batch_recorded_vals_dict`
Same as `batch_recorded_vals` but returns a dictionary.
## Functions: Utility functions (`pidata.utils`)
#### `pidata.utils.strip_timestamp`
Internal function. Converts PI timestamp format to python datetime format.
#### `pidata.utils.validate_tags`
Will check each PI Tag in list tag and return list of tags found or NOT found (depending on parameter `return_found`)
tags: list of PI query filters
returns: list of all tag names that match the PI queries if return_found=True (default) OR list of the given PI queries that did not match any tags if return_found=False
For example, you can wrap your tag list in the `validate_tags` function to ensure you don't get a "tag not found" error in one of the other functions.
from pidata.pull import aggregated_vals
from pidata.utils import validate_tags
aggregated_vals(validate_tags(['tag1', 'tag2', 'tag3',]), start_time='1/1/2020')
## How do I change to a different (non default) PI Server:
For any function that requires access to PI Server, use the `server` argument to pass the PI Server by name (string). If the `server` argument is ommitted, PI Server will be set to the default server. This is the recommended method of changing servers.
df = pidata.pull.aggregated_vals(['24T1345.PV'], start_time='1/1/2020', server='PI.SERVER.NAME')
Alternatively, you can change the default server name by importing `piServers` from `pidata.pull`. This might not change the default server for functions in `pidata.utils`.
from pidata.pull import piServers
piServers.DefaultPIServer = 'PI.SERVER.NAME'
## Exposing the SDK:
TODO
Raw data
{
"_id": null,
"home_page": "",
"name": "PIdata",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "OSI,PI,Connector,Historian,Process,Control,Automation,Engineering",
"author": "",
"author_email": "Venanzio Petrarolo <petrarolov@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/89/60/3daa6cf16b7948ece46f1d1917a14175b33febc434cd1dd384c47c42be38/PIdata-0.3.1.tar.gz",
"platform": null,
"description": "# PIdata\r\nAn easy to use python package for extracting data from an OSI PI historian/server via the OSI PI SDK. Based on this blog post: https://pisquare.osisoft.com/people/rborges/blog/2016/08/01/pi-and-python-pithon\r\n\r\n## What makes PIdata great:\r\n1. Written in python/ironpython\r\n2. Simple to use (specifically for people new to python)\r\n3. Customisable (exposing the PI SDK as python functions/objects)\r\n4. Integrates well with existing python handling tools (e.g. pandas dfs)\r\n5. Does not reinvent existing tools as these are what makes python great\r\n\r\n## Prerequisites: \r\nOSI PI installed on your machine, connected the historian (probably already the case if you were looking for this package)\r\n\r\n## Installation:\r\n\r\n pip install PIdata\r\n\r\n## Basic usage:\r\nTo pull hourly averages for the '24T1345.PV' tag:\r\n\r\n import pidata\r\n\r\n df = pidata.pull.aggregated_vals(['24T1345.PV'], start_time='1/1/2020', end_time='2/2/2020', interval='1h')\r\n\r\n## Similar projects:\r\nFernandoRodriguezP/OSIsoftPy\r\n\r\nonamission21/AF-SDK-for-Python\r\n\r\nalyasaud/PITHON\r\n\r\n## Contributing\r\nI find this code very useful. If you do too, please star the repo on github and share with your colleagues. \r\nMany thanks to the people that have contributed their code thus far. Please feel free to submit a pull request, report a bug or request a feature.\r\n\r\n\r\n\r\n## Functions: Data pull functions (`pidata.pull`)\r\n\r\n#### `pidata.pull.aggregated_vals`\r\nWill return a pandas dataframe of aggregated values (averaged values by default) between `start_time` and `end_time`, within the given interval\r\n \r\n Arguments: \r\n tags : list or list like\r\n start_time : Time of the first data point. Default: '-30d' (thirty days ago)\r\n end_time : Time of the last data point. Default: '' (empty/current time)\r\n interval : Time between data points. Default: '12h'\r\n method : Instead of returning the average value over the interval, the returned values can be specified as one of the following: \r\n Total\r\n Average (Default)\r\n Minimum\r\n Maximum\r\n Range\r\n StdDev - Standard deviation.\r\n PopulationStdDev - Population standard deviation.\r\n Count\r\n PercentGood - Percentage of data with good value. \r\n TotalWithUOM\r\n All\r\n AllForNonNumeric\r\n Please see: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/T_OSIsoft_AF_Data_AFSummaryTypes.htm\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n\r\n#### `pidata.pull.recorded_vals`\r\nWill return a pandas dataframe of recorded vals between `start_time` and `end_time`, with an the given interval\r\n \r\n Arguments: \r\n tags : list or list like\r\n start_time : Time of the first data point. Default: '-30d' (thirty days ago)\r\n end_time : Time of the last data point. Default: '' (empty/current time)\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n\r\n#### `pidata.pull.interp_vals`\r\nWill return a pandas dataframe of averaged vals between `start_time` and `end_time`, with an the given interval\r\n \r\n Arguments: \r\n tags : list or list like\r\n start_time : Time of the first data point. Default: '-30d' (thirty days ago)\r\n end_time : Time of the last data point. Default: '' (empty/current time)\r\n interval : Time between data points. Default: '12h'\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n\r\n#### `pidata.pull.current_vals`\r\nReturns the last recorded values at the time of running the function\r\n\r\n Arguments: \r\n tags : list or list like\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n\r\n#### `pidata.pull.batch_aggregated_vals`\r\nPuprose: fetch large averaged data in batches\r\n \r\n Arguments:\r\n tags : list of tags to download\r\n start_time : start date time in string format where batch fetch begin\r\n end_time : end data time in string format where batch ends\r\n interval : period over which to average data e.g. '4H', '2D'\r\n method : aggregation method that will be given to aggregated_vals\r\n period : time period to define batch size e.g. 'days','months'\r\n increment : number of time periods in a batch\r\n verbose : verbose output of progress (default = False)\r\n save_csv : save progress files. Default is False.\r\n filename : name of file without the extension. Function will add suffix\r\n return_df : whether or not to return the data as a pandas dataframe (default=True)\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n\r\n#### `pidata.pull.batch_recorded_vals`\r\nPuprose: fetch large averaged data in batches\r\n \r\n Arguments: \r\n tags : list of tags to download\r\n start_time : start date time in string format where batch fetch begin\r\n end_time : end data time in string format where batch ends\r\n period : time period to define batch size e.g. 'days','months'\r\n increment : number of time periods in a batch\r\n verbose : verbose output of progress (default = False)\r\n save_csv : save progress files. Default is False.\r\n filename : name of file without the extension. Function will add suffix\r\n return_df : whether or not to return the data as a pandas dataframe (default=True)\r\n server : Name of the PI server to use. Uses the default if none is provided\r\n \r\n#### `pidata.pull.recorded_vals_dict`\r\nA dictionary version of the recorded vals function for better efficiency for large amounts of data.\r\n\r\n#### `pidata.pull.batch_recorded_vals_dict`\r\nSame as `batch_recorded_vals` but returns a dictionary. \r\n\r\n## Functions: Utility functions (`pidata.utils`)\r\n\r\n#### `pidata.utils.strip_timestamp`\r\nInternal function. Converts PI timestamp format to python datetime format. \r\n\r\n#### `pidata.utils.validate_tags`\r\nWill check each PI Tag in list tag and return list of tags found or NOT found (depending on parameter `return_found`)\r\n \r\n tags: list of PI query filters\r\n returns: list of all tag names that match the PI queries if return_found=True (default) OR list of the given PI queries that did not match any tags if return_found=False\r\n\r\nFor example, you can wrap your tag list in the `validate_tags` function to ensure you don't get a \"tag not found\" error in one of the other functions. \r\n\r\n from pidata.pull import aggregated_vals\r\n from pidata.utils import validate_tags\r\n\r\n aggregated_vals(validate_tags(['tag1', 'tag2', 'tag3',]), start_time='1/1/2020') \r\n\r\n\r\n## How do I change to a different (non default) PI Server:\r\nFor any function that requires access to PI Server, use the `server` argument to pass the PI Server by name (string). If the `server` argument is ommitted, PI Server will be set to the default server. This is the recommended method of changing servers. \r\n\r\n df = pidata.pull.aggregated_vals(['24T1345.PV'], start_time='1/1/2020', server='PI.SERVER.NAME')\r\n\r\nAlternatively, you can change the default server name by importing `piServers` from `pidata.pull`. This might not change the default server for functions in `pidata.utils`.\r\n\r\n from pidata.pull import piServers\r\n piServers.DefaultPIServer = 'PI.SERVER.NAME'\r\n\r\n## Exposing the SDK:\r\nTODO\r\n",
"bugtrack_url": null,
"license": "",
"summary": "An easy-to-use connector for the OSI PI historian",
"version": "0.3.1",
"project_urls": {
"Homepage": "https://github.com/Petrarolo/PIdata",
"Issues": "https://github.com/Petrarolo/PIdata/issues"
},
"split_keywords": [
"osi",
"pi",
"connector",
"historian",
"process",
"control",
"automation",
"engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "de5a8db9435935535d70bcc67a4dff9276c28a1e448e851a628253ff00c46b82",
"md5": "ccd1f0a14051874dc921752d9cb156d6",
"sha256": "faa192cd1e551d486750730ace2e584d2bdc791ad65c2c907698bf56e7e76ebf"
},
"downloads": -1,
"filename": "PIdata-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ccd1f0a14051874dc921752d9cb156d6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9653,
"upload_time": "2024-03-09T05:17:32",
"upload_time_iso_8601": "2024-03-09T05:17:32.271488Z",
"url": "https://files.pythonhosted.org/packages/de/5a/8db9435935535d70bcc67a4dff9276c28a1e448e851a628253ff00c46b82/PIdata-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "89603daa6cf16b7948ece46f1d1917a14175b33febc434cd1dd384c47c42be38",
"md5": "cedef9c369b92ebeea55e0989b454ed2",
"sha256": "293c5f2995ef14d1c85eaa626d40f0279cd24b9c4da7fff1bf4f61843549b9e5"
},
"downloads": -1,
"filename": "PIdata-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "cedef9c369b92ebeea55e0989b454ed2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 7582,
"upload_time": "2024-03-09T05:20:27",
"upload_time_iso_8601": "2024-03-09T05:20:27.635265Z",
"url": "https://files.pythonhosted.org/packages/89/60/3daa6cf16b7948ece46f1d1917a14175b33febc434cd1dd384c47c42be38/PIdata-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-09 05:20:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Petrarolo",
"github_project": "PIdata",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pidata"
}
JSON
