# MIDAS Open parser
[](https://pypi.org/project/midas-open-parser/)
`midas_open_parser` is a Python library specifically designed to parse BADC-CSV files from the MIDAS Open dataset.
It provides a tailored and extensible framework for handling the metadata labels and structures found in the MIDAS Open dataset.
The main goal of the `midas_open_parser` project is to provide tools to ingest the MIDAS Open dataset into various destinations.
## About MIDAS Open and BADC-CSV
### MIDAS Open Dataset
The Met Office Integrated Data Archive System (MIDAS) Open dataset is a collection of surface observations from the UK land surface observing network. It includes various meteorological parameters such as temperature, wind, precipitation, and cloud cover, among others. The dataset is maintained and distributed by the Met Office, the national meteorological service for the UK.
For more information about the MIDAS Open dataset, please refer to the following resources:
- [MIDAS Open User Guide](https://help.ceda.ac.uk/article/4982-midas-open-user-guide)
### BADC-CSV Format
The BADC-CSV (British Atmospheric Data Centre Comma-Separated Values) format is a text-based format used to store and distribute data from the MIDAS Open dataset. It is a structured CSV format that includes a metadata section describing the data, followed by the actual data records.
The metadata section follows various conventions, such as the Climate and Forecast (CF) conventions, to provide detailed information about the data variables, units, measurement methods, and other relevant metadata. This metadata is crucial for properly interpreting and understanding the data.
For more information about the BADC-CSV format, please refer to the following resources:
- [Short description](https://help.ceda.ac.uk/article/105-badc-csv)
- [BADC-CSV GitHub repository](https://github.com/cedadev/badc-csv/tree/main)
- [The BADC Text File - Guide for users, and producers](https://github.com/cedadev/badc-csv/blob/main/new_ASCII_file_format_guide.md)
## Features
- Parse BADC-CSV files and extract data records
- Extract and process metadata from BADC-CSV files
- Support for handling global and field-level metadata
- Extensible architecture for adding new metadata label handlers
- Error handling for unknown metadata labels
- Command-line interface for running the parser
## Installation
You can install badc-csv-parser using pip:
```bash
pip install badc-csv-parser
```
## Usage
Here's an example of how to use badc-csv-parser:
```python
from badc_csv_parser import parse_badc_csv, parse_badc_csv_metadata
from midas import extract_midas_metadata
# Parse the BADC-CSV file metadata
metadata = parse_badc_csv_metadata('path/to/file.csv')
# Extract MIDAS metadata
midas_metadata = extract_midas_metadata(metadata)
# Access global metadata
print("Global Metadata:")
for field, values in midas_metadata['global'].items():
print(f"{field}: {values}")
# Access field metadata
print("\nField Metadata:")
for field_name, field_data in midas_metadata.items():
if field_name != 'global':
print(f"{field_name}: {field_data}")
# Parse the BADC-CSV data records
data_rows = parse_badc_csv('path/to/file.csv')
for row in data_rows:
print(row)
```
You can also use the command-line interface:
```python
python -m badc_csv_parser.test path/to/file.csv
```
## License
This project is licensed under the MIT License.
Raw data
{
"_id": null,
"home_page": null,
"name": "midas-open-parser",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "atmospheric-science, midas-open, badc-csv, parser, environmental-data",
"author": null,
"author_email": "TempoTerra <vanuan+midasparser@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/83/ac/c2c8c89cea02452fa6f6a71d49fd944aa76d07e9b546689d405c0db872a5/midas_open_parser-0.1.25.tar.gz",
"platform": null,
"description": "# MIDAS Open parser\n\n[](https://pypi.org/project/midas-open-parser/)\n\n\n`midas_open_parser` is a Python library specifically designed to parse BADC-CSV files from the MIDAS Open dataset.\n\nIt provides a tailored and extensible framework for handling the metadata labels and structures found in the MIDAS Open dataset.\n\nThe main goal of the `midas_open_parser` project is to provide tools to ingest the MIDAS Open dataset into various destinations.\n\n## About MIDAS Open and BADC-CSV\n\n### MIDAS Open Dataset\n\nThe Met Office Integrated Data Archive System (MIDAS) Open dataset is a collection of surface observations from the UK land surface observing network. It includes various meteorological parameters such as temperature, wind, precipitation, and cloud cover, among others. The dataset is maintained and distributed by the Met Office, the national meteorological service for the UK.\n\nFor more information about the MIDAS Open dataset, please refer to the following resources:\n\n- [MIDAS Open User Guide](https://help.ceda.ac.uk/article/4982-midas-open-user-guide)\n\n### BADC-CSV Format\n\nThe BADC-CSV (British Atmospheric Data Centre Comma-Separated Values) format is a text-based format used to store and distribute data from the MIDAS Open dataset. It is a structured CSV format that includes a metadata section describing the data, followed by the actual data records.\n\nThe metadata section follows various conventions, such as the Climate and Forecast (CF) conventions, to provide detailed information about the data variables, units, measurement methods, and other relevant metadata. This metadata is crucial for properly interpreting and understanding the data.\n\nFor more information about the BADC-CSV format, please refer to the following resources:\n\n- [Short description](https://help.ceda.ac.uk/article/105-badc-csv)\n- [BADC-CSV GitHub repository](https://github.com/cedadev/badc-csv/tree/main)\n- [The BADC Text File - Guide for users, and producers](https://github.com/cedadev/badc-csv/blob/main/new_ASCII_file_format_guide.md)\n\n## Features\n\n- Parse BADC-CSV files and extract data records\n- Extract and process metadata from BADC-CSV files\n- Support for handling global and field-level metadata\n- Extensible architecture for adding new metadata label handlers\n- Error handling for unknown metadata labels\n- Command-line interface for running the parser\n\n## Installation\n\nYou can install badc-csv-parser using pip:\n\n```bash\npip install badc-csv-parser\n```\n\n## Usage\n\nHere's an example of how to use badc-csv-parser:\n\n```python\nfrom badc_csv_parser import parse_badc_csv, parse_badc_csv_metadata\nfrom midas import extract_midas_metadata\n\n# Parse the BADC-CSV file metadata\nmetadata = parse_badc_csv_metadata('path/to/file.csv')\n\n# Extract MIDAS metadata\nmidas_metadata = extract_midas_metadata(metadata)\n\n# Access global metadata\nprint(\"Global Metadata:\")\nfor field, values in midas_metadata['global'].items():\n print(f\"{field}: {values}\")\n\n# Access field metadata\nprint(\"\\nField Metadata:\")\nfor field_name, field_data in midas_metadata.items():\n if field_name != 'global':\n print(f\"{field_name}: {field_data}\")\n\n# Parse the BADC-CSV data records\ndata_rows = parse_badc_csv('path/to/file.csv')\nfor row in data_rows:\n print(row)\n```\n\n\nYou can also use the command-line interface:\n\n```python\npython -m badc_csv_parser.test path/to/file.csv\n```\n\n## License\n\nThis project is licensed under the MIT License.\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python library for parsing BADC-CSV files from the MIDAS Open dataset",
"version": "0.1.25",
"project_urls": {
"Bug Tracker": "https://github.com/tempoterra/midas_open_parser/issues",
"Documentation": "https://tempoterra.github.io/midas_open_parser",
"Source Code": "https://github.com/tempoterra/midas_open_parser"
},
"split_keywords": [
"atmospheric-science",
" midas-open",
" badc-csv",
" parser",
" environmental-data"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6ff3d90f6c8dcc1673aaf7a61ca085af72b8da15b61c667f2f0fecab340bf72e",
"md5": "491d7d1ab10a2e2031fdf5ea51a25c4f",
"sha256": "8a9d38219ab07dea9227a5feb7e36300181aefac6c6ce2f2fa3a7c23110b3133"
},
"downloads": -1,
"filename": "midas_open_parser-0.1.25-py3-none-any.whl",
"has_sig": false,
"md5_digest": "491d7d1ab10a2e2031fdf5ea51a25c4f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 7484,
"upload_time": "2024-05-23T16:37:27",
"upload_time_iso_8601": "2024-05-23T16:37:27.652867Z",
"url": "https://files.pythonhosted.org/packages/6f/f3/d90f6c8dcc1673aaf7a61ca085af72b8da15b61c667f2f0fecab340bf72e/midas_open_parser-0.1.25-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "83acc2c8c89cea02452fa6f6a71d49fd944aa76d07e9b546689d405c0db872a5",
"md5": "1de65cd43f5478fe234c7de4c7a944fc",
"sha256": "3830a493ce25bc813a5063e4889039ecc6fd983ae3687d3b1da8dc28d94df4fe"
},
"downloads": -1,
"filename": "midas_open_parser-0.1.25.tar.gz",
"has_sig": false,
"md5_digest": "1de65cd43f5478fe234c7de4c7a944fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 6248,
"upload_time": "2024-05-23T16:37:28",
"upload_time_iso_8601": "2024-05-23T16:37:28.850414Z",
"url": "https://files.pythonhosted.org/packages/83/ac/c2c8c89cea02452fa6f6a71d49fd944aa76d07e9b546689d405c0db872a5/midas_open_parser-0.1.25.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-23 16:37:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tempoterra",
"github_project": "midas_open_parser",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "sphinx",
"specs": [
[
">=",
"4.5.0"
]
]
},
{
"name": "sphinx-rtd-theme",
"specs": [
[
">=",
"1.0.0"
]
]
},
{
"name": "myst-parser",
"specs": [
[
">=",
"0.18.0"
]
]
}
],
"lcname": "midas-open-parser"
}
JSON
