# Elexon Data Portal
[](https://zenodo.org/badge/latestdoi/189842391) [](https://notebooks.gesis.org/binder/v2/gh/OSUKED/ElexonDataPortal/master?urlpath=lab%2Ftree%2Fnbs%2F08-quick-start.ipynb) [](https://badge.fury.io/py/ElexonDataPortal)
The `ElexonDataPortal` library is a Python Client for retrieving data from the Elexon/BMRS API. The library significantly reduces the complexity of interfacing with the Elexon/BMRS API through the standardisation of parameter names and orchestration of multiple queries when making requests over a date range. To use the `ElexonDataPortal` you will have to register for an Elexon API key which can be done [here](https://www.elexonportal.co.uk/registration/newuser).
<br>
<br>
### Installation
The library can be easily installed from PyPi, this can be done using:
```bash
pip install ElexonDataPortal
```
<br>
<br>
### Getting Started
We'll begin by initialising the API `Client`. The key parameter to pass here is the `api_key`, alternatively this can be set by specifying the environment variable `BMRS_API_KEY` which will then be loaded automatically.
```python
from ElexonDataPortal import api
client = api.Client('your_api_key_here')
```
<br>
Now that the client has been initialised we can make a request!
One of the key abstractions within the `ElexonDataPortal` library is the handling of multiple requests over a date range specified through the `start_date` and `end_date` parameters. Each response will be automatically cleaned and parsed, then concatenated into a single Pandas DataFrame. If a settlement period and date column can be identified in the returned data then a new column will be added with the local datetime for each data-point. N.b. that if passed as a string the start and end datetimes will be assumed to be in the local timezone for the UK
```python
start_date = '2020-01-01'
end_date = '2020-01-01 1:30'
df_B1610 = client.get_B1610(start_date, end_date)
df_B1610.head(3)
```
| | documentType | businessType | processType | timeSeriesID | curveType | settlementDate | powerSystemResourceType | registeredResourceEICCode | marketGenerationUnitEICCode | marketGenerationBMUId | marketGenerationNGCBMUId | bMUnitID | nGCBMUnitID | activeFlag | documentID | documentRevNum | resolution | start | end | settlementPeriod | quantity | local_datetime |
|---:|:------------------|:---------------|:--------------|:----------------------|:----------------------------|:-----------------|:--------------------------|:----------------------------|:------------------------------|:------------------------|:---------------------------|:------------|:--------------|:-------------|:------------------------|-----------------:|:-------------|:-----------|:-----------|-------------------:|-----------:|:--------------------------|
| 0 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-212 | Sequential fixed size block | 2020-01-01 | Generation | 48W000CAS-BEU01F | 48W000CAS-BEU01F | M_CAS-BEU01 | CAS-BEU01 | M_CAS-BEU01 | CAS-BEU01 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 18.508 | 2020-01-01 00:00:00+00:00 |
| 1 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-355 | Sequential fixed size block | 2020-01-01 | Generation | 48W00000STLGW-3A | 48W00000STLGW-3A | T_STLGW-3 | STLGW-3 | T_STLGW-3 | STLGW-3 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 28.218 | 2020-01-01 00:00:00+00:00 |
| 2 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-278 | Sequential fixed size block | 2020-01-01 | Generation | 48W00000GNFSW-1H | 48W00000GNFSW-1H | T_GNFSW-1 | GNFSW-1 | T_GNFSW-1 | GNFSW-1 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 29.44 | 2020-01-01 00:00:00+00:00 |
<br>
If you've previously written your own code for extracting data from the Elexon/BMRS API then you may be wondering where some of the normal parameters you pass have gone. The reduction in the parameters passed are due to 4 core drivers:
* Standardisation of date range parameter names
* Removal of the need to specify `ServiceType`
* Automatic passing of `APIKey` after client initialisation
* Shipped with sensible defaults for all remaining parameters
The full list of data streams that are able to be requested can be found [here](#data-stream-descriptions). If you wish to make requests using the raw methods these are available through the `ElexonDataportal.dev.raw` module.
Further information can be found in the [Quick Start guide](https://osuked.github.io/ElexonDataPortal/08-quick-start/).
<br>
<br>
### What's Changed in v2
The latest release of the library includes a full rewrite of the code-base. We have endeavoured to make the new API as intuitive as possible but that has required breaking changes from v1, if you wish to continue using the historic library use `pip install ElexonDataPortal==1.0.4`. N.b v1 will not be maintained going forward, you are advised to change over to v2.0.0+.
The key feature changes are:
* Coverage of more BMRS streams
* Automated default values
* Cleaner client API
* A larger range of request types are compatible with the date range orchestrator
<br>
<br>
### Programmatic Library Generation
One of the core features within the `ElexonDataPortal` library is that it is *self-generating*, by which we mean it can rebuild itself (including any new API request methods) from scratch using only the `endpoints.csv` spreadsheet. As well as generating the Python Client library a `BMRS_API.yaml` file is created, this provides an OpenAPI specification representation of the Elexon/BMRS API. In turn this allows us to automatically generate documentation, as well as run tests on the API itself to ensure that everything is working as expected - during this process we identified and corrected several small errors in the API documentation provided by Elexon.
To rebuild the library simply run the following in the root directory:
```bash
python -m ElexonDataPortal.rebuild
```
<br>
N.b. If you wish to develop the library further or use any of the programmatic library generation functionality then please install the development version of the library using:
```bash
pip install ElexonDataPortal[dev]
```
If you are not installing into a fresh environment it is recommended you install `pyyaml` and `geopandas` using conda to avoid any dependency conflicts. In future we are looking to release `ElexonDataPortal` as a conda package to avoid these issues.
<br>
<br>
### Data Stream Descriptions
The following table describes the data streams that are currently retreivable through the API. The client method to retrieve data from a given stream follows the naming convention `get_{stream-name}`.
| Stream | Description |
|:-----------------------|:---------------------------------------------------------------|
| B0610 | Actual Total Load per Bidding Zone |
| B0620 | Day-Ahead Total Load Forecast per Bidding Zone |
| B0630 | Week-Ahead Total Load Forecast per Bidding Zone |
| B0640 | Month-Ahead Total Load Forecast Per Bidding Zone |
| B0650 | Year Ahead Total Load Forecast per Bidding Zone |
| B0710 | Planned Unavailability of Consumption Units |
| B0720 | Changes In Actual Availability Of Consumption Units |
| B0810 | Year Ahead Forecast Margin |
| B0910 | Expansion and Dismantling Projects |
| B1010 | Planned Unavailability In The Transmission Grid |
| B1020 | Changes In Actual Availability In The Transmission Grid |
| B1030 | Changes In Actual Availability of Offshore Grid Infrastructure |
| B1320 | Congestion Management Measures Countertrading |
| B1330 | Congestion Management Measures Costs of Congestion Management |
| B1410 | Installed Generation Capacity Aggregated |
| B1420 | Installed Generation Capacity per Unit |
| B1430 | Day-Ahead Aggregated Generation |
| B1440 | Generation forecasts for Wind and Solar |
| B1510 | Planned Unavailability of Generation Units |
| B1520 | Changes In Actual Availability of Generation Units |
| B1530 | Planned Unavailability of Production Units |
| B1540 | Changes In Actual Availability of Production Units |
| B1610 | Actual Generation Output per Generation Unit |
| B1620 | Actual Aggregated Generation per Type |
| B1630 | Actual Or Estimated Wind and Solar Power Generation |
| B1720 | Amount Of Balancing Reserves Under Contract Service |
| B1730 | Prices Of Procured Balancing Reserves Service |
| B1740 | Accepted Aggregated Offers |
| B1750 | Activated Balancing Energy |
| B1760 | Prices Of Activated Balancing Energy |
| B1770 | Imbalance Prices |
| B1780 | Aggregated Imbalance Volumes |
| B1790 | Financial Expenses and Income For Balancing |
| B1810 | Cross-Border Balancing Volumes of Exchanged Bids and Offers |
| B1820 | Cross-Border Balancing Prices |
| B1830 | Cross-border Balancing Energy Activated |
| BOD | Bid Offer Level Data |
| CDN | Credit Default Notice Data |
| DERSYSDATA | Derived System Data |
| DETSYSPRICES | Detailed System Prices |
| DEVINDOD | Daily Energy Volume Data |
| DISBSAD | Balancing Services Adjustment Action Data |
| FORDAYDEM | Forecast Day and Day Ahead Demand Data |
| FREQ | Rolling System Frequency |
| FUELHH | Half Hourly Outturn Generation by Fuel Type |
| MELIMBALNGC | Forecast Day and Day Ahead Margin and Imbalance Data |
| MID | Market Index Data |
| MessageDetailRetrieval | REMIT Flow - Message List Retrieval |
| MessageListRetrieval | REMIT Flow - Message List Retrieval |
| NETBSAD | Balancing Service Adjustment Data |
| NONBM | Non BM STOR Instructed Volume Data |
| PHYBMDATA | Physical Data |
| SYSDEM | System Demand |
| SYSWARN | System Warnings |
| TEMP | Temperature Data |
| WINDFORFUELHH | Wind Generation Forecast and Out-turn Data |
Raw data
{
"_id": null,
"home_page": "https://github.com/OSUKED/ElexonDataPortal",
"name": "ElexonDataPortal",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Ayrton Bourn",
"author_email": "ayrtonbourn@outlook.com",
"download_url": "https://files.pythonhosted.org/packages/e7/b4/2279238a904948fa670e3fd066ec352cdea74a89d7eed9684f3b0e93b3fb/ElexonDataPortal-2.0.13.tar.gz",
"platform": null,
"description": "# Elexon Data Portal\n\n[](https://zenodo.org/badge/latestdoi/189842391) [](https://notebooks.gesis.org/binder/v2/gh/OSUKED/ElexonDataPortal/master?urlpath=lab%2Ftree%2Fnbs%2F08-quick-start.ipynb) [](https://badge.fury.io/py/ElexonDataPortal)\n\nThe `ElexonDataPortal` library is a Python Client for retrieving data from the Elexon/BMRS API. The library significantly reduces the complexity of interfacing with the Elexon/BMRS API through the standardisation of parameter names and orchestration of multiple queries when making requests over a date range. To use the `ElexonDataPortal` you will have to register for an Elexon API key which can be done [here](https://www.elexonportal.co.uk/registration/newuser). \n\n<br>\n<br>\n\n### Installation\n\nThe library can be easily installed from PyPi, this can be done using:\n\n```bash\npip install ElexonDataPortal\n```\n\n<br>\n<br>\n\n### Getting Started\n\nWe'll begin by initialising the API `Client`. The key parameter to pass here is the `api_key`, alternatively this can be set by specifying the environment variable `BMRS_API_KEY` which will then be loaded automatically.\n\n```python\nfrom ElexonDataPortal import api\n\nclient = api.Client('your_api_key_here')\n```\n\n<br>\n\nNow that the client has been initialised we can make a request! \n\nOne of the key abstractions within the `ElexonDataPortal` library is the handling of multiple requests over a date range specified through the `start_date` and `end_date` parameters. Each response will be automatically cleaned and parsed, then concatenated into a single Pandas DataFrame. If a settlement period and date column can be identified in the returned data then a new column will be added with the local datetime for each data-point. N.b. that if passed as a string the start and end datetimes will be assumed to be in the local timezone for the UK\n\n```python\nstart_date = '2020-01-01'\nend_date = '2020-01-01 1:30'\n\ndf_B1610 = client.get_B1610(start_date, end_date)\n\ndf_B1610.head(3)\n```\n\n| | documentType | businessType | processType | timeSeriesID | curveType | settlementDate | powerSystemResourceType | registeredResourceEICCode | marketGenerationUnitEICCode | marketGenerationBMUId | marketGenerationNGCBMUId | bMUnitID | nGCBMUnitID | activeFlag | documentID | documentRevNum | resolution | start | end | settlementPeriod | quantity | local_datetime |\n|---:|:------------------|:---------------|:--------------|:----------------------|:----------------------------|:-----------------|:--------------------------|:----------------------------|:------------------------------|:------------------------|:---------------------------|:------------|:--------------|:-------------|:------------------------|-----------------:|:-------------|:-----------|:-----------|-------------------:|-----------:|:--------------------------|\n| 0 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-212 | Sequential fixed size block | 2020-01-01 | Generation | 48W000CAS-BEU01F | 48W000CAS-BEU01F | M_CAS-BEU01 | CAS-BEU01 | M_CAS-BEU01 | CAS-BEU01 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 18.508 | 2020-01-01 00:00:00+00:00 |\n| 1 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-355 | Sequential fixed size block | 2020-01-01 | Generation | 48W00000STLGW-3A | 48W00000STLGW-3A | T_STLGW-3 | STLGW-3 | T_STLGW-3 | STLGW-3 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 28.218 | 2020-01-01 00:00:00+00:00 |\n| 2 | Actual generation | Production | Realised | ELX-EMFIP-AGOG-TS-278 | Sequential fixed size block | 2020-01-01 | Generation | 48W00000GNFSW-1H | 48W00000GNFSW-1H | T_GNFSW-1 | GNFSW-1 | T_GNFSW-1 | GNFSW-1 | Y | ELX-EMFIP-AGOG-22495386 | 1 | PT30M | 2020-01-01 | 2020-01-01 | 1 | 29.44 | 2020-01-01 00:00:00+00:00 |\n\n<br>\n\nIf you've previously written your own code for extracting data from the Elexon/BMRS API then you may be wondering where some of the normal parameters you pass have gone. The reduction in the parameters passed are due to 4 core drivers:\n\n* Standardisation of date range parameter names\n* Removal of the need to specify `ServiceType`\n* Automatic passing of `APIKey` after client initialisation\n* Shipped with sensible defaults for all remaining parameters\n\nThe full list of data streams that are able to be requested can be found [here](#data-stream-descriptions). If you wish to make requests using the raw methods these are available through the `ElexonDataportal.dev.raw` module.\n\nFurther information can be found in the [Quick Start guide](https://osuked.github.io/ElexonDataPortal/08-quick-start/).\n\n<br>\n<br>\n\n### What's Changed in v2\n\nThe latest release of the library includes a full rewrite of the code-base. We have endeavoured to make the new API as intuitive as possible but that has required breaking changes from v1, if you wish to continue using the historic library use `pip install ElexonDataPortal==1.0.4`. N.b v1 will not be maintained going forward, you are advised to change over to v2.0.0+. \n\nThe key feature changes are:\n\n* Coverage of more BMRS streams \n* Automated default values\n* Cleaner client API\n* A larger range of request types are compatible with the date range orchestrator\n\n<br>\n<br>\n\n### Programmatic Library Generation\n\nOne of the core features within the `ElexonDataPortal` library is that it is *self-generating*, by which we mean it can rebuild itself (including any new API request methods) from scratch using only the `endpoints.csv` spreadsheet. As well as generating the Python Client library a `BMRS_API.yaml` file is created, this provides an OpenAPI specification representation of the Elexon/BMRS API. In turn this allows us to automatically generate documentation, as well as run tests on the API itself to ensure that everything is working as expected - during this process we identified and corrected several small errors in the API documentation provided by Elexon. \n\nTo rebuild the library simply run the following in the root directory: \n\n```bash\npython -m ElexonDataPortal.rebuild\n```\n\n<br>\n\nN.b. If you wish to develop the library further or use any of the programmatic library generation functionality then please install the development version of the library using:\n\n```bash\npip install ElexonDataPortal[dev]\n```\n\nIf you are not installing into a fresh environment it is recommended you install `pyyaml` and `geopandas` using conda to avoid any dependency conflicts. In future we are looking to release `ElexonDataPortal` as a conda package to avoid these issues.\n\n<br>\n<br>\n\n### Data Stream Descriptions\n\nThe following table describes the data streams that are currently retreivable through the API. The client method to retrieve data from a given stream follows the naming convention `get_{stream-name}`.\n\n| Stream | Description |\n|:-----------------------|:---------------------------------------------------------------|\n| B0610 | Actual Total Load per Bidding Zone |\n| B0620 | Day-Ahead Total Load Forecast per Bidding Zone |\n| B0630 | Week-Ahead Total Load Forecast per Bidding Zone |\n| B0640 | Month-Ahead Total Load Forecast Per Bidding Zone |\n| B0650 | Year Ahead Total Load Forecast per Bidding Zone |\n| B0710 | Planned Unavailability of Consumption Units |\n| B0720 | Changes In Actual Availability Of Consumption Units |\n| B0810 | Year Ahead Forecast Margin |\n| B0910 | Expansion and Dismantling Projects |\n| B1010 | Planned Unavailability In The Transmission Grid |\n| B1020 | Changes In Actual Availability In The Transmission Grid |\n| B1030 | Changes In Actual Availability of Offshore Grid Infrastructure |\n| B1320 | Congestion Management Measures Countertrading |\n| B1330 | Congestion Management Measures Costs of Congestion Management |\n| B1410 | Installed Generation Capacity Aggregated |\n| B1420 | Installed Generation Capacity per Unit |\n| B1430 | Day-Ahead Aggregated Generation |\n| B1440 | Generation forecasts for Wind and Solar |\n| B1510 | Planned Unavailability of Generation Units |\n| B1520 | Changes In Actual Availability of Generation Units |\n| B1530 | Planned Unavailability of Production Units |\n| B1540 | Changes In Actual Availability of Production Units |\n| B1610 | Actual Generation Output per Generation Unit |\n| B1620 | Actual Aggregated Generation per Type |\n| B1630 | Actual Or Estimated Wind and Solar Power Generation |\n| B1720 | Amount Of Balancing Reserves Under Contract Service |\n| B1730 | Prices Of Procured Balancing Reserves Service |\n| B1740 | Accepted Aggregated Offers |\n| B1750 | Activated Balancing Energy |\n| B1760 | Prices Of Activated Balancing Energy |\n| B1770 | Imbalance Prices |\n| B1780 | Aggregated Imbalance Volumes |\n| B1790 | Financial Expenses and Income For Balancing |\n| B1810 | Cross-Border Balancing Volumes of Exchanged Bids and Offers |\n| B1820 | Cross-Border Balancing Prices |\n| B1830 | Cross-border Balancing Energy Activated |\n| BOD | Bid Offer Level Data |\n| CDN | Credit Default Notice Data |\n| DERSYSDATA | Derived System Data |\n| DETSYSPRICES | Detailed System Prices |\n| DEVINDOD | Daily Energy Volume Data |\n| DISBSAD | Balancing Services Adjustment Action Data |\n| FORDAYDEM | Forecast Day and Day Ahead Demand Data |\n| FREQ | Rolling System Frequency |\n| FUELHH | Half Hourly Outturn Generation by Fuel Type |\n| MELIMBALNGC | Forecast Day and Day Ahead Margin and Imbalance Data |\n| MID | Market Index Data |\n| MessageDetailRetrieval | REMIT Flow - Message List Retrieval |\n| MessageListRetrieval | REMIT Flow - Message List Retrieval |\n| NETBSAD | Balancing Service Adjustment Data |\n| NONBM | Non BM STOR Instructed Volume Data |\n| PHYBMDATA | Physical Data |\n| SYSDEM | System Demand |\n| SYSWARN | System Warnings |\n| TEMP | Temperature Data |\n| WINDFORFUELHH | Wind Generation Forecast and Out-turn Data |\n\n",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "2.0.13",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "ebdcbe343d51e1f75a69e495896b9498",
"sha256": "d12288a67bfc13960e291076eb4419e16bc58fc0205d873aeb050af63de98ad6"
},
"downloads": -1,
"filename": "ElexonDataPortal-2.0.13-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ebdcbe343d51e1f75a69e495896b9498",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 45885,
"upload_time": "2022-05-14T17:33:33",
"upload_time_iso_8601": "2022-05-14T17:33:33.860252Z",
"url": "https://files.pythonhosted.org/packages/9f/51/1e765db0d0bb4c534811e08858b4a30929be50bffd4cf2aec2faba82c895/ElexonDataPortal-2.0.13-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "e063e86dd11b737f2c98cefc52ac396b",
"sha256": "ad032bcf682afeacb1abc42d2fb19e858ae90a50056d427dbfc39f5364b3f270"
},
"downloads": -1,
"filename": "ElexonDataPortal-2.0.13.tar.gz",
"has_sig": false,
"md5_digest": "e063e86dd11b737f2c98cefc52ac396b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 38589,
"upload_time": "2022-05-14T17:33:35",
"upload_time_iso_8601": "2022-05-14T17:33:35.720552Z",
"url": "https://files.pythonhosted.org/packages/e7/b4/2279238a904948fa670e3fd066ec352cdea74a89d7eed9684f3b0e93b3fb/ElexonDataPortal-2.0.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-05-14 17:33:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "OSUKED",
"github_project": "ElexonDataPortal",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "elexondataportal"
}
JSON
