# EM27 Metadata
This repository is the single source of truth for our EM27 measurement logistics: "Where has each station been on each day of measurements?" We selected this format over putting it in a database due to various reasons:
- Easy to read, modify and extend by selective group members using GitHub permissions
- Changes to this are more obvious here than in database logs
- Versioning (easy to revert mistakes)
- Automatic testing of the files integrities
- Easy import as a statically typed Python library
<br/>
## What does this data look like?
There is a set of locations in **`data/locations.json`**:
```json
[
{
"location_id": "BRU",
"details": "Industriegelände an der Brudermühlstraße",
"lon": 11.547,
"lat": 48.111,
"alt": 528
},
{
"location_id": "DLR",
"details": "DLR in Wessling",
"lon": 11.279,
"lat": 48.086,
"alt": 592
}
]
```
There is a set of sensors in **`data/sensors.json`** that measure at these location sites:
```json
[
{
"sensor_id": "ma",
"serial_number": 61,
"utc_offsets": [
{ "from_date": "20150826", "to_date": "20220623", "utc_offset": 0 },
{ "from_date": "20220624", "to_date": "20220926", "utc_offset": 2 },
{ "from_date": "20220927", "to_date": "20221231", "utc_offset": 0 }
],
"different_pressure_data_source": [
{
"from_date": "20220626",
"to_date": "20220926",
"source": "LMU-MIM01-height-adjusted"
},
{
"from_date": "20220626",
"to_date": "20220926",
"source": "mc"
}
],
"pressure_calibration_factors": [
{ "from_date": "20150826", "to_date": "20221231", "factor": 1 }
],
"locations": [
{ "from_date": "20181019", "to_date": "20181019", "location": "TUM_LAB" },
{ "from_date": "20181020", "to_date": "20181030", "location": "LMU" },
{ "from_date": "20181031", "to_date": "20220515", "location": "TUM_I" }
]
}
]
```
There is a set of campaigns in **`data/campaigns.json`**:
```json
[
{
"campaign_id": "muccnet",
"from_date": "20190913",
"to_date": "21000101",
"stations": [
{ "sensor": "ma", "default_location": "TUM_I", "direction": "center" },
{ "sensor": "mb", "default_location": "FEL", "direction": "east" },
{ "sensor": "mc", "default_location": "GRAE", "direction": "west" },
{ "sensor": "md", "default_location": "OBE", "direction": "north" },
{ "sensor": "me", "default_location": "TAU", "direction": "south" }
]
}
]
```
<br/>
## How to add new measurement days?
1. Possibly add new locations in `data/locations.json`
2. Extend the list of locations in `data/sensors.json`
3. Possibly add new campaign setups in `data/campaigns.json`
<br/>
## How can I know whether my changes were correct?
Whenever you make changes in the repository on GitHub, the integrity of the files will automatically be checked. You can check whether all tests have passed [here](https://github.com/tum-esm/em27-location-data/actions). If some have failed you can ask Moritz Makowski.
A list of all integrity checks can be found in [`tests/README.md`](https://github.com/tum-esm/em27-location-data/tree/main/tests).
<br/>
## How to use it in your codebase?
1. Install python library
```bash
poetry add tum_esm_em27_metadata
# or
pip install tum_esm_em27_metadata
```
2. Create a personal access token for a GitHub account that has read access to the metadata repository: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
3. Use the metadata anywhere
```python
import tum_esm_em27_metadata
em27_metadata_interface = tum_esm_em27_metadata.load_from_github(
github_repository = "org-name/repo-name",
access_token = "ghp_..."
)
metadata = em27_metadata_interface.get(
sensor_id = "ma", date = "20220601"
) # is of type tum_esm_em27_metadata.types.SensorDataContext
print(metadata.dict())
```
... prints out:
```json
{
"sensor_id": "ma",
"serial_number": 61,
"utc_offset": 0,
"pressure_data_source": "ma",
"pressure_calibration_factor": 1,
"date": "20220601",
"location": {
"location_id": "TUM_I",
"details": "TUM Dach Innenstadt",
"lon": 11.569,
"lat": 48.151,
"alt": 539
}
}
```
<br/>
## For Developers: Publish the Package to PyPI
```bash
poetry build
poetry publish
```
Raw data
{
"_id": null,
"home_page": "https://github.com/tum-esm/em27-metadata",
"name": "tum-esm-em27-metadata",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Moritz Makowski",
"author_email": "moritz.makowski@tum.de",
"download_url": "https://files.pythonhosted.org/packages/ef/fa/563656af297050dfd69b920490a6fa076242ac9b850c5bc389eacbefe338/tum_esm_em27_metadata-1.0.1.tar.gz",
"platform": null,
"description": "# EM27 Metadata\n\nThis repository is the single source of truth for our EM27 measurement logistics: \"Where has each station been on each day of measurements?\" We selected this format over putting it in a database due to various reasons:\n\n- Easy to read, modify and extend by selective group members using GitHub permissions\n- Changes to this are more obvious here than in database logs\n- Versioning (easy to revert mistakes)\n- Automatic testing of the files integrities\n- Easy import as a statically typed Python library\n\n<br/>\n\n## What does this data look like?\n\nThere is a set of locations in **`data/locations.json`**:\n\n```json\n[\n {\n \"location_id\": \"BRU\",\n \"details\": \"Industriegel\u00e4nde an der Bruderm\u00fchlstra\u00dfe\",\n \"lon\": 11.547,\n \"lat\": 48.111,\n \"alt\": 528\n },\n {\n \"location_id\": \"DLR\",\n \"details\": \"DLR in Wessling\",\n \"lon\": 11.279,\n \"lat\": 48.086,\n \"alt\": 592\n }\n]\n```\n\nThere is a set of sensors in **`data/sensors.json`** that measure at these location sites:\n\n```json\n[\n {\n \"sensor_id\": \"ma\",\n \"serial_number\": 61,\n \"utc_offsets\": [\n { \"from_date\": \"20150826\", \"to_date\": \"20220623\", \"utc_offset\": 0 },\n { \"from_date\": \"20220624\", \"to_date\": \"20220926\", \"utc_offset\": 2 },\n { \"from_date\": \"20220927\", \"to_date\": \"20221231\", \"utc_offset\": 0 }\n ],\n \"different_pressure_data_source\": [\n {\n \"from_date\": \"20220626\",\n \"to_date\": \"20220926\",\n \"source\": \"LMU-MIM01-height-adjusted\"\n },\n {\n \"from_date\": \"20220626\",\n \"to_date\": \"20220926\",\n \"source\": \"mc\"\n }\n ],\n \"pressure_calibration_factors\": [\n { \"from_date\": \"20150826\", \"to_date\": \"20221231\", \"factor\": 1 }\n ],\n \"locations\": [\n { \"from_date\": \"20181019\", \"to_date\": \"20181019\", \"location\": \"TUM_LAB\" },\n { \"from_date\": \"20181020\", \"to_date\": \"20181030\", \"location\": \"LMU\" },\n { \"from_date\": \"20181031\", \"to_date\": \"20220515\", \"location\": \"TUM_I\" }\n ]\n }\n]\n```\n\nThere is a set of campaigns in **`data/campaigns.json`**:\n\n```json\n[\n {\n \"campaign_id\": \"muccnet\",\n \"from_date\": \"20190913\",\n \"to_date\": \"21000101\",\n \"stations\": [\n { \"sensor\": \"ma\", \"default_location\": \"TUM_I\", \"direction\": \"center\" },\n { \"sensor\": \"mb\", \"default_location\": \"FEL\", \"direction\": \"east\" },\n { \"sensor\": \"mc\", \"default_location\": \"GRAE\", \"direction\": \"west\" },\n { \"sensor\": \"md\", \"default_location\": \"OBE\", \"direction\": \"north\" },\n { \"sensor\": \"me\", \"default_location\": \"TAU\", \"direction\": \"south\" }\n ]\n }\n]\n```\n\n<br/>\n\n## How to add new measurement days?\n\n1. Possibly add new locations in `data/locations.json`\n2. Extend the list of locations in `data/sensors.json`\n3. Possibly add new campaign setups in `data/campaigns.json`\n\n<br/>\n\n## How can I know whether my changes were correct?\n\nWhenever you make changes in the repository on GitHub, the integrity of the files will automatically be checked. You can check whether all tests have passed [here](https://github.com/tum-esm/em27-location-data/actions). If some have failed you can ask Moritz Makowski.\n\nA list of all integrity checks can be found in [`tests/README.md`](https://github.com/tum-esm/em27-location-data/tree/main/tests).\n\n<br/>\n\n## How to use it in your codebase?\n\n1. Install python library\n\n```bash\npoetry add tum_esm_em27_metadata\n# or\npip install tum_esm_em27_metadata\n```\n\n2. Create a personal access token for a GitHub account that has read access to the metadata repository: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token\n\n3. Use the metadata anywhere\n\n```python\nimport tum_esm_em27_metadata\n\nem27_metadata_interface = tum_esm_em27_metadata.load_from_github(\n github_repository = \"org-name/repo-name\",\n access_token = \"ghp_...\"\n)\n\nmetadata = em27_metadata_interface.get(\n sensor_id = \"ma\", date = \"20220601\"\n) # is of type tum_esm_em27_metadata.types.SensorDataContext\n\nprint(metadata.dict())\n```\n\n... prints out:\n\n```json\n{\n \"sensor_id\": \"ma\",\n \"serial_number\": 61,\n \"utc_offset\": 0,\n \"pressure_data_source\": \"ma\",\n \"pressure_calibration_factor\": 1,\n \"date\": \"20220601\",\n \"location\": {\n \"location_id\": \"TUM_I\",\n \"details\": \"TUM Dach Innenstadt\",\n \"lon\": 11.569,\n \"lat\": 48.151,\n \"alt\": 539\n }\n}\n```\n\n<br/>\n\n## For Developers: Publish the Package to PyPI\n\n```bash\npoetry build\npoetry publish\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "single source of truth for ESM's EM27 measurement logistics",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/tum-esm/em27-metadata",
"Repository": "https://github.com/tum-esm/em27-metadata"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db3aad552f1c4a5e0afbf6bb41a5f9bfc204cc2c2c9e01fef45a77a3a79d2c81",
"md5": "3dc9c4df667079cd99f4f615a7c92371",
"sha256": "b91580f6a95114b043b0294187cb35aea197608e77df39d65945a7d533e907ba"
},
"downloads": -1,
"filename": "tum_esm_em27_metadata-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3dc9c4df667079cd99f4f615a7c92371",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<4.0",
"size": 6376,
"upload_time": "2023-03-19T21:39:04",
"upload_time_iso_8601": "2023-03-19T21:39:04.446064Z",
"url": "https://files.pythonhosted.org/packages/db/3a/ad552f1c4a5e0afbf6bb41a5f9bfc204cc2c2c9e01fef45a77a3a79d2c81/tum_esm_em27_metadata-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "effa563656af297050dfd69b920490a6fa076242ac9b850c5bc389eacbefe338",
"md5": "c9dfca513deca02993d455fb09bad78c",
"sha256": "d135c7a0185b02441385cbf8d76681605fe4891dc8c8bc4e8ecd541bafa7e41d"
},
"downloads": -1,
"filename": "tum_esm_em27_metadata-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "c9dfca513deca02993d455fb09bad78c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<4.0",
"size": 6583,
"upload_time": "2023-03-19T21:39:06",
"upload_time_iso_8601": "2023-03-19T21:39:06.238152Z",
"url": "https://files.pythonhosted.org/packages/ef/fa/563656af297050dfd69b920490a6fa076242ac9b850c5bc389eacbefe338/tum_esm_em27_metadata-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-19 21:39:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tum-esm",
"github_project": "em27-metadata",
"github_not_found": true,
"lcname": "tum-esm-em27-metadata"
}
JSON
