# MISOReports
A comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes.
As of 2024-12-22, MISOReports supports reports from
[MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/), [MISORTWDBIReporter](https://api.misoenergy.org/MISORTWDBIReporter/),
and [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/), totalling to well over 120 different reports.
With MISOReports, you can skip all of the intermediate URL generation/parsing/typing steps and get any supported report's data
as a dataframe with just a few lines of code. You can also choose to retrieve the raw data directly and use that instead.
For documentation and information on currently supported reports, please check out [DOCUMENTATION.md](DOCUMENTATION.md).
## Features
MISOReports supports these features and more:
- Downloading reports by datetime for reports that offer a datetime option
- Downloading live reports for reports without a date option
- Downloading raw report content in any of their supported formats (csv, xml, json, xls, xlsx, etc.)
- Generating target URLs for the report of your choice
## Examples
### Example 1:
Download a single-table report with datetime option from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).
#### Input:
```python
import datetime
from MISOReports.MISOReports import MISOReports
# Downloads the data offered at
# https://docs.misoenergy.org/marketreports/20241030_da_expost_ramp_mcp.xlsx.
df = MISOReports.get_df(
report_name="da_expost_ramp_mcp",
ddatetime=datetime.datetime(year=2024, month=10, day=30),
)
print(df)
```
#### Output:
```
Hour Ending Reserve Zone 1 - DA MCP Ramp Up Ex-Post 1 Hour ... Reserve Zone 8 - DA MCP Ramp Up Ex-Post 1 Hour Reserve Zone 8 - DA MCP Ramp Down Ex-Post 1 Hour
0 1 0.00 ... 0.00 0.0
1 2 0.00 ... 0.00 0.0
2 3 0.00 ... 0.00 0.0
3 4 0.00 ... 0.00 0.0
4 5 0.00 ... 0.00 0.0
5 6 0.17 ... 0.17 0.0
6 7 1.48 ... 1.48 0.0
7 8 0.00 ... 0.00 0.0
8 9 0.00 ... 0.00 0.0
9 10 0.00 ... 0.00 0.0
10 11 0.00 ... 0.00 0.0
11 12 1.08 ... 1.08 0.0
12 13 1.81 ... 1.81 0.0
13 14 2.56 ... 2.56 0.0
14 15 3.13 ... 3.13 0.0
15 16 5.00 ... 5.00 0.0
16 17 5.00 ... 5.00 0.0
17 18 12.85 ... 12.85 0.0
18 19 5.17 ... 5.17 0.0
19 20 0.00 ... 0.00 0.0
20 21 0.00 ... 0.00 0.0
21 22 0.00 ... 0.00 0.0
22 23 0.00 ... 0.00 0.0
23 24 0.00 ... 0.00 0.0
[24 rows x 17 columns]
```
### Example 2:
Download a multi-table report from [MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/).
#### Input:
```python
from MISOReports.MISOReports import MISOReports
# Downloads the data offered at
# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=gettotalload&returnType=csv.
df = MISOReports.get_df(
report_name="totalload",
)
# For multi-table reports, use a for-loop
# to iterate across the tables.
for i, table_name in enumerate(df["table_names"]):
print(table_name)
print(df["dataframes"].iloc[i].head(3))
print()
print()
```
#### Output:
```
ClearedMW
Load_Hour Load_Value
0 1 65871.0
1 2 65521.0
2 3 64474.0
MediumTermLoadForecast
Hour_End Load_Forecast
0 1 68614.0
1 2 66566.0
2 3 66620.0
FiveMinTotalLoad
Load_Time Load_Value
0 1900-01-01 00:00:00 68899.0
1 1900-01-01 00:05:00 68690.0
2 1900-01-01 00:10:00 68327.0
```
### Example 3:
Download a multi-table report from [MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/).
#### Input:
```python
from MISOReports.MISOReports import MISOReports
# Downloads the data offered at
# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=getlmpconsolidatedtable&returnType=csv.
df = MISOReports.get_df(
report_name="lmpconsolidatedtable",
)
# For multi-table reports, use a for-loop
# to iterate across the tables.
for i, table_name in enumerate(df["table_names"]):
print(table_name)
print(df["dataframes"].iloc[i].head(3))
print()
print()
```
#### Output:
```
Metadata
Type Timing
0 FiveMinLMP 1900-01-01 16:45:00
1 HourlyIntegratedLmp 1900-01-01 16:00:00
2 DayAheadExAnteLmp 1900-01-01 17:00:00
Data
Name LMP - FiveMinLMP MLC - FiveMinLMP MCC - FiveMinLMP REGMCP - FiveMinLMP ... MLC - DayAheadExAnteLmp MCC - DayAheadExAnteLmp LMP - DayAheadExPostLmp MLC - DayAheadExPostLmp MCC - DayAheadExPostLmp
1 EES.PERVL2_CT 17.49 -1.69 -12.64 15.0 ... -1.15 -6.1 21.0 -1.15 -6.1
2 EES.RICE1 17.91 -1.25 -12.66 15.0 ... -0.06 -6.21 21.98 -0.06 -6.21
3 EES.RVRBEND1 18.42 -0.98 -12.42 15.0 ... -0.38 -5.83 22.04 -0.38 -5.83
[3 rows x 20 columns]
```
### Example 4:
Download a single-table report along with its text content from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).
#### Input:
```python
from MISOReports.MISOReports import MISOReports
# Downloads the data offered at
# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=getNAI&returnType=csv.
data = MISOReports.get_data(
report_name="NAI",
file_extension="csv",
)
print("Text Content:")
print(data.response.text)
print()
print("Dataframe:")
print(data.df)
```
#### Output:
```
Text Content:
RefId,22-Dec-2024 - Interval 16:40 EST
Name,Value
MISO,2212.89
Dataframe:
Name Value
0 MISO 2212.89
```
### Example 5:
Download a single-table report with datetime option from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).
#### Input:
```python
import datetime
from MISOReports.MISOReports import MISOReports
# Downloads the data offered at
# https://docs.misoenergy.org/marketreports/MISOdaily3042024.xml.
# Note: the above link's 304 represents
# the number of days past the start of the year,
# 2024, which aligns with the ddatetime given below.
data = MISOReports.get_data(
report_name="MISOdaily",
ddatetime=datetime.datetime(year=2024, month=10, day=30),
)
print(data.df)
```
#### Output:
```
PostedValue Hour Data_Code Data_Date Data_Type UTCOffset PostingType
0 64975 1 2024-10-30 DF 5 Daily
1 63868 2 2024-10-30 DF 5 Daily
2 62750 3 2024-10-30 DF 5 Daily
3 62581 4 2024-10-30 DF 5 Daily
4 63869 5 2024-10-30 DF 5 Daily
.. ... ... ... ... ... ... ...
619 1935 20 TVA 2024-10-28 FLOW 5 Daily
620 2304 21 TVA 2024-10-28 FLOW 5 Daily
621 2379 22 TVA 2024-10-28 FLOW 5 Daily
622 2343 23 TVA 2024-10-28 FLOW 5 Daily
623 2364 24 TVA 2024-10-28 FLOW 5 Daily
[624 rows x 7 columns]
```
## Contributing
Please take a look at our [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/BrianWeiHaoMa/MISOReports",
"name": "MISOReports",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "python, data-science, scraper, download, pandas, reports, electricity, tables, data-tables, data-scraping, miso, energy-markets, electricity-markets",
"author": "Brian Wei Hao Ma, Ryan B. Green, William Sun",
"author_email": "brianmaytc@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/b6/13/86258094df9dfe0a076900c8963cf298a1a94eb16604a3808f4b8661d92a/misoreports-0.0.0.tar.gz",
"platform": null,
"description": "# MISOReports\r\nA comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes. \r\n\r\nAs of 2024-12-22, MISOReports supports reports from \r\n[MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/), [MISORTWDBIReporter](https://api.misoenergy.org/MISORTWDBIReporter/),\r\nand [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/), totalling to well over 120 different reports.\r\n\r\nWith MISOReports, you can skip all of the intermediate URL generation/parsing/typing steps and get any supported report's data\r\nas a dataframe with just a few lines of code. You can also choose to retrieve the raw data directly and use that instead. \r\n\r\nFor documentation and information on currently supported reports, please check out [DOCUMENTATION.md](DOCUMENTATION.md).\r\n\r\n## Features\r\nMISOReports supports these features and more:\r\n- Downloading reports by datetime for reports that offer a datetime option\r\n- Downloading live reports for reports without a date option\r\n- Downloading raw report content in any of their supported formats (csv, xml, json, xls, xlsx, etc.)\r\n- Generating target URLs for the report of your choice\r\n\r\n## Examples\r\n\r\n### Example 1:\r\nDownload a single-table report with datetime option from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).\r\n\r\n#### Input:\r\n```python\r\nimport datetime\r\nfrom MISOReports.MISOReports import MISOReports\r\n\r\n# Downloads the data offered at\r\n# https://docs.misoenergy.org/marketreports/20241030_da_expost_ramp_mcp.xlsx.\r\ndf = MISOReports.get_df(\r\n report_name=\"da_expost_ramp_mcp\",\r\n ddatetime=datetime.datetime(year=2024, month=10, day=30),\r\n)\r\n\r\nprint(df)\r\n```\r\n\r\n#### Output:\r\n```\r\n Hour Ending Reserve Zone 1 - DA MCP Ramp Up Ex-Post 1 Hour ... Reserve Zone 8 - DA MCP Ramp Up Ex-Post 1 Hour Reserve Zone 8 - DA MCP Ramp Down Ex-Post 1 Hour\r\n0 1 0.00 ... 0.00 0.0\r\n1 2 0.00 ... 0.00 0.0\r\n2 3 0.00 ... 0.00 0.0\r\n3 4 0.00 ... 0.00 0.0\r\n4 5 0.00 ... 0.00 0.0\r\n5 6 0.17 ... 0.17 0.0\r\n6 7 1.48 ... 1.48 0.0\r\n7 8 0.00 ... 0.00 0.0\r\n8 9 0.00 ... 0.00 0.0\r\n9 10 0.00 ... 0.00 0.0\r\n10 11 0.00 ... 0.00 0.0\r\n11 12 1.08 ... 1.08 0.0\r\n12 13 1.81 ... 1.81 0.0\r\n13 14 2.56 ... 2.56 0.0\r\n14 15 3.13 ... 3.13 0.0\r\n15 16 5.00 ... 5.00 0.0\r\n16 17 5.00 ... 5.00 0.0\r\n17 18 12.85 ... 12.85 0.0\r\n18 19 5.17 ... 5.17 0.0\r\n19 20 0.00 ... 0.00 0.0\r\n20 21 0.00 ... 0.00 0.0\r\n21 22 0.00 ... 0.00 0.0\r\n22 23 0.00 ... 0.00 0.0\r\n23 24 0.00 ... 0.00 0.0\r\n\r\n[24 rows x 17 columns]\r\n```\r\n\r\n### Example 2:\r\nDownload a multi-table report from [MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/).\r\n\r\n#### Input:\r\n```python\r\nfrom MISOReports.MISOReports import MISOReports\r\n\r\n# Downloads the data offered at\r\n# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=gettotalload&returnType=csv.\r\ndf = MISOReports.get_df(\r\n report_name=\"totalload\",\r\n)\r\n\r\n# For multi-table reports, use a for-loop\r\n# to iterate across the tables.\r\nfor i, table_name in enumerate(df[\"table_names\"]):\r\n print(table_name)\r\n print(df[\"dataframes\"].iloc[i].head(3))\r\n print()\r\n print()\r\n```\r\n\r\n#### Output:\r\n```\r\nClearedMW\r\n Load_Hour Load_Value\r\n0 1 65871.0\r\n1 2 65521.0\r\n2 3 64474.0\r\n\r\n\r\nMediumTermLoadForecast\r\n Hour_End Load_Forecast\r\n0 1 68614.0\r\n1 2 66566.0\r\n2 3 66620.0\r\n\r\n\r\nFiveMinTotalLoad\r\n Load_Time Load_Value\r\n0 1900-01-01 00:00:00 68899.0\r\n1 1900-01-01 00:05:00 68690.0\r\n2 1900-01-01 00:10:00 68327.0\r\n```\r\n\r\n### Example 3:\r\nDownload a multi-table report from [MISORTWDDataBroker](https://api.misoenergy.org/MISORTWDDataBroker/).\r\n\r\n#### Input:\r\n```python\r\nfrom MISOReports.MISOReports import MISOReports\r\n\r\n# Downloads the data offered at\r\n# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=getlmpconsolidatedtable&returnType=csv.\r\ndf = MISOReports.get_df(\r\n report_name=\"lmpconsolidatedtable\",\r\n)\r\n\r\n# For multi-table reports, use a for-loop\r\n# to iterate across the tables.\r\nfor i, table_name in enumerate(df[\"table_names\"]):\r\n print(table_name)\r\n print(df[\"dataframes\"].iloc[i].head(3))\r\n print()\r\n print()\r\n```\r\n\r\n#### Output:\r\n```\r\nMetadata\r\n Type Timing\r\n0 FiveMinLMP 1900-01-01 16:45:00\r\n1 HourlyIntegratedLmp 1900-01-01 16:00:00\r\n2 DayAheadExAnteLmp 1900-01-01 17:00:00\r\n\r\n\r\nData\r\n Name LMP - FiveMinLMP MLC - FiveMinLMP MCC - FiveMinLMP REGMCP - FiveMinLMP ... MLC - DayAheadExAnteLmp MCC - DayAheadExAnteLmp LMP - DayAheadExPostLmp MLC - DayAheadExPostLmp MCC - DayAheadExPostLmp\r\n1 EES.PERVL2_CT 17.49 -1.69 -12.64 15.0 ... -1.15 -6.1 21.0 -1.15 -6.1\r\n2 EES.RICE1 17.91 -1.25 -12.66 15.0 ... -0.06 -6.21 21.98 -0.06 -6.21\r\n3 EES.RVRBEND1 18.42 -0.98 -12.42 15.0 ... -0.38 -5.83 22.04 -0.38 -5.83\r\n\r\n[3 rows x 20 columns]\r\n```\r\n\r\n### Example 4:\r\nDownload a single-table report along with its text content from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).\r\n\r\n#### Input:\r\n```python\r\nfrom MISOReports.MISOReports import MISOReports\r\n\r\n# Downloads the data offered at\r\n# https://api.misoenergy.org/MISORTWDDataBroker/DataBrokerServices.asmx?messageType=getNAI&returnType=csv.\r\ndata = MISOReports.get_data(\r\n report_name=\"NAI\",\r\n file_extension=\"csv\",\r\n)\r\n\r\nprint(\"Text Content:\")\r\nprint(data.response.text)\r\nprint()\r\n\r\nprint(\"Dataframe:\")\r\nprint(data.df)\r\n```\r\n\r\n#### Output:\r\n```\r\nText Content:\r\nRefId,22-Dec-2024 - Interval 16:40 EST\r\n\r\nName,Value\r\nMISO,2212.89\r\n\r\n\r\nDataframe:\r\n Name Value\r\n0 MISO 2212.89\r\n```\r\n\r\n### Example 5:\r\nDownload a single-table report with datetime option from [MISO Market Reports](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports/).\r\n\r\n#### Input:\r\n```python\r\nimport datetime\r\nfrom MISOReports.MISOReports import MISOReports\r\n\r\n# Downloads the data offered at\r\n# https://docs.misoenergy.org/marketreports/MISOdaily3042024.xml.\r\n# Note: the above link's 304 represents\r\n# the number of days past the start of the year, \r\n# 2024, which aligns with the ddatetime given below.\r\ndata = MISOReports.get_data(\r\n report_name=\"MISOdaily\",\r\n ddatetime=datetime.datetime(year=2024, month=10, day=30),\r\n)\r\n\r\nprint(data.df)\r\n```\r\n\r\n#### Output:\r\n```\r\n PostedValue Hour Data_Code Data_Date Data_Type UTCOffset PostingType\r\n0 64975 1 2024-10-30 DF 5 Daily\r\n1 63868 2 2024-10-30 DF 5 Daily\r\n2 62750 3 2024-10-30 DF 5 Daily\r\n3 62581 4 2024-10-30 DF 5 Daily\r\n4 63869 5 2024-10-30 DF 5 Daily\r\n.. ... ... ... ... ... ... ...\r\n619 1935 20 TVA 2024-10-28 FLOW 5 Daily\r\n620 2304 21 TVA 2024-10-28 FLOW 5 Daily\r\n621 2379 22 TVA 2024-10-28 FLOW 5 Daily\r\n622 2343 23 TVA 2024-10-28 FLOW 5 Daily\r\n623 2364 24 TVA 2024-10-28 FLOW 5 Daily\r\n\r\n[624 rows x 7 columns]\r\n```\r\n\r\n## Contributing\r\nPlease take a look at our [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute.\r\n\r\n## License\r\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes.",
"version": "0.0.0",
"project_urls": {
"Homepage": "https://github.com/BrianWeiHaoMa/MISOReports"
},
"split_keywords": [
"python",
" data-science",
" scraper",
" download",
" pandas",
" reports",
" electricity",
" tables",
" data-tables",
" data-scraping",
" miso",
" energy-markets",
" electricity-markets"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fdbecb2ed98d2c02bcd7330d1544119d8da45240c1793096d862dac4aed4c757",
"md5": "ceb7dbd81419ae84abd8b7b29f45aa0b",
"sha256": "979786b9b102148dc75d0c4a28f1c1bf8990197bc2e7a3be01b8df6ec64f1eca"
},
"downloads": -1,
"filename": "MISOReports-0.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ceb7dbd81419ae84abd8b7b29f45aa0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 30479,
"upload_time": "2024-12-24T21:10:49",
"upload_time_iso_8601": "2024-12-24T21:10:49.439249Z",
"url": "https://files.pythonhosted.org/packages/fd/be/cb2ed98d2c02bcd7330d1544119d8da45240c1793096d862dac4aed4c757/MISOReports-0.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b61386258094df9dfe0a076900c8963cf298a1a94eb16604a3808f4b8661d92a",
"md5": "b9affeb1b99770b5547614ade961a0bb",
"sha256": "58e68db46a26e66974eb45d03161f3df81118f3537eecdbd1071c0f6394f9ea3"
},
"downloads": -1,
"filename": "misoreports-0.0.0.tar.gz",
"has_sig": false,
"md5_digest": "b9affeb1b99770b5547614ade961a0bb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 43615,
"upload_time": "2024-12-24T21:10:54",
"upload_time_iso_8601": "2024-12-24T21:10:54.427538Z",
"url": "https://files.pythonhosted.org/packages/b6/13/86258094df9dfe0a076900c8963cf298a1a94eb16604a3808f4b8661d92a/misoreports-0.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-24 21:10:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "BrianWeiHaoMa",
"github_project": "MISOReports",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "misoreports"
}
JSON
