| Name | nemdata JSON |
| Version |
0.3.7
JSON |
| download |
| home_page | None |
| Summary | Simple CLI for downloading data for Australia's NEM from AEMO. |
| upload_time | 2024-10-20 03:50:52 |
| maintainer | None |
| docs_url | None |
| author | Adam Green |
| requires_python | <3.11,>=3.10 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# nem-data
A simple & opinionated Python command line tool to access Australian National Energy Market (NEM) data provided by the Australian Energy Market Operator (AEMO). It features:
- access to the NEMDE dataset as well as the predispatch, unit-scada, trading-price, demand and interconnectors tables from MMSDM,
- a cache to not re-download data that already exists in `~/nem-data/data`,
- adds `interval-start` and `interval-end` columns.
It is designed for use by researchers & data scientists - this tool supports my personal research work. It is not designed for production use - see [NEMOSIS](https://github.com/UNSW-CEEM/NEMOSIS) for a production grade package.
See [A Hackers Guide to AEMO & NEM Data](https://adgefficiency.com/hackers-aemo/) for more context on the data provided by AEMO.
## Setup
```bash
$ pip install nemdata
```
## Use
### CLI
```shell-session
$ nemdata --help
Usage: nemdata [OPTIONS]
Downloads NEM data from AEMO.
Options:
-t, --table TEXT Available tables: nemde, dispatch-price,
predispatch, unit-scada, trading-price, demand,
interconnectors, p5min, predispatch-sensitivities,
predispatch-demand.
-s, --start TEXT Start date (YYYY-MM or YYYY-MM-DD for NEMDE).
-e, --end TEXT End date (incusive) (YYYY-MM or YYYY-MM-DD for
NEMDE).
--dry-run / --no-dry-run Whether to save downloaded data to disk.
--help Show this message and exit.
```
Download NEMDE data for the first three days in January 2018:
```shell-session
$ nemdata -t nemde --start 2018-01-01 --end 2018-01-03
```
Download trading price data from MMSDM for January to March 2018:
```shell-session
$ nemdata -t trading-price -s 2018-01 -e 2018-03
```
### Python
Download trading price data from MMSDM for January to Feburary 2020:
```python
import nemdata
data = nemdata.download(start="2020-01", end="2020-02", table="trading-price")
```
Load this data back into a pandas DataFrame:
```python
data = nemdata.load()['trading-price']
```
At this point, `data` will have the trading price for all regions.
## Data
Downloaded into into `$HOME/nem-data/data/`:
```shell-session
$ nemdata -t trading-price -s 2020-01 -e 2020-02
$ tree ~/nem-data
/Users/adam/nem-data
└── data
└── trading-price
├── 2020-01
│ ├── PUBLIC_DVD_TRADINGPRICE_202001010000.CSV
│ ├── clean.csv
│ ├── clean.parquet
│ └── raw.zip
└── 2020-02
├── PUBLIC_DVD_TRADINGPRICE_202002010000.CSV
├── clean.csv
├── clean.parquet
└── raw.zip
```
A few things happen during data processing:
- rows of the raw CSV are removed to create a rectangular, single table CSV,
- `interval-start` and `interval-end` timezone aware datetime columns are added,
- when using `nemdata.loader.loader` for `trading-price`, all data is resampled to a 5 minute frequency (both before and after the 30 to 5 minute settlement interval change).
Raw data
{
"_id": null,
"home_page": null,
"name": "nemdata",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.11,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Adam Green",
"author_email": "adam.green@adgefficiency.com",
"download_url": "https://files.pythonhosted.org/packages/d5/a3/bcdac4efeca036781df84d4445e561dfbb279be3c6d57a07448f739ec471/nemdata-0.3.7.tar.gz",
"platform": null,
"description": "# nem-data\n\nA simple & opinionated Python command line tool to access Australian National Energy Market (NEM) data provided by the Australian Energy Market Operator (AEMO). It features:\n\n- access to the NEMDE dataset as well as the predispatch, unit-scada, trading-price, demand and interconnectors tables from MMSDM,\n- a cache to not re-download data that already exists in `~/nem-data/data`,\n- adds `interval-start` and `interval-end` columns.\n\nIt is designed for use by researchers & data scientists - this tool supports my personal research work. It is not designed for production use - see [NEMOSIS](https://github.com/UNSW-CEEM/NEMOSIS) for a production grade package.\n\nSee [A Hackers Guide to AEMO & NEM Data](https://adgefficiency.com/hackers-aemo/) for more context on the data provided by AEMO.\n\n\n## Setup\n\n```bash\n$ pip install nemdata\n```\n\n\n## Use\n\n### CLI\n\n```shell-session\n$ nemdata --help\nUsage: nemdata [OPTIONS]\n\n Downloads NEM data from AEMO.\n\nOptions:\n -t, --table TEXT Available tables: nemde, dispatch-price,\n predispatch, unit-scada, trading-price, demand,\n interconnectors, p5min, predispatch-sensitivities,\n predispatch-demand.\n -s, --start TEXT Start date (YYYY-MM or YYYY-MM-DD for NEMDE).\n -e, --end TEXT End date (incusive) (YYYY-MM or YYYY-MM-DD for\n NEMDE).\n --dry-run / --no-dry-run Whether to save downloaded data to disk.\n --help Show this message and exit.\n```\n\nDownload NEMDE data for the first three days in January 2018:\n\n```shell-session\n$ nemdata -t nemde --start 2018-01-01 --end 2018-01-03\n```\n\nDownload trading price data from MMSDM for January to March 2018:\n\n```shell-session\n$ nemdata -t trading-price -s 2018-01 -e 2018-03\n```\n\n### Python\n\nDownload trading price data from MMSDM for January to Feburary 2020:\n\n```python\nimport nemdata\n\ndata = nemdata.download(start=\"2020-01\", end=\"2020-02\", table=\"trading-price\")\n```\n\nLoad this data back into a pandas DataFrame:\n\n```python\ndata = nemdata.load()['trading-price']\n```\n\nAt this point, `data` will have the trading price for all regions.\n\n\n## Data\n\nDownloaded into into `$HOME/nem-data/data/`:\n\n```shell-session\n$ nemdata -t trading-price -s 2020-01 -e 2020-02\n$ tree ~/nem-data\n/Users/adam/nem-data\n\u2514\u2500\u2500 data\n \u2514\u2500\u2500 trading-price\n \u251c\u2500\u2500 2020-01\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 PUBLIC_DVD_TRADINGPRICE_202001010000.CSV\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 clean.csv\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 clean.parquet\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 raw.zip\n \u2514\u2500\u2500 2020-02\n \u251c\u2500\u2500 PUBLIC_DVD_TRADINGPRICE_202002010000.CSV\n \u251c\u2500\u2500 clean.csv\n \u251c\u2500\u2500 clean.parquet\n \u2514\u2500\u2500 raw.zip\n```\n\nA few things happen during data processing:\n\n- rows of the raw CSV are removed to create a rectangular, single table CSV,\n- `interval-start` and `interval-end` timezone aware datetime columns are added,\n- when using `nemdata.loader.loader` for `trading-price`, all data is resampled to a 5 minute frequency (both before and after the 30 to 5 minute settlement interval change).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Simple CLI for downloading data for Australia's NEM from AEMO.",
"version": "0.3.7",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0d09fe3ad4b6009f2c30af1b47cbdd62f34d834a3308e58431a44ff18113bc93",
"md5": "df67ceb314b382504cb539a5820bb1e5",
"sha256": "5ea6af34718354fd8c4dd0d05a7a68b50d859955641d2bcce35d4422d2812248"
},
"downloads": -1,
"filename": "nemdata-0.3.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "df67ceb314b382504cb539a5820bb1e5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.11,>=3.10",
"size": 10758,
"upload_time": "2024-10-20T03:50:51",
"upload_time_iso_8601": "2024-10-20T03:50:51.218691Z",
"url": "https://files.pythonhosted.org/packages/0d/09/fe3ad4b6009f2c30af1b47cbdd62f34d834a3308e58431a44ff18113bc93/nemdata-0.3.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d5a3bcdac4efeca036781df84d4445e561dfbb279be3c6d57a07448f739ec471",
"md5": "bb5feba3086c0474ae0ea1031f377d5b",
"sha256": "a6200b9b9e609b171a1216f435303c40e0d792956a9b183d2778574ab80a8402"
},
"downloads": -1,
"filename": "nemdata-0.3.7.tar.gz",
"has_sig": false,
"md5_digest": "bb5feba3086c0474ae0ea1031f377d5b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.11,>=3.10",
"size": 8444,
"upload_time": "2024-10-20T03:50:52",
"upload_time_iso_8601": "2024-10-20T03:50:52.115826Z",
"url": "https://files.pythonhosted.org/packages/d5/a3/bcdac4efeca036781df84d4445e561dfbb279be3c6d57a07448f739ec471/nemdata-0.3.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-20 03:50:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "nemdata"
}
