# Metamist
[](https://codecov.io/gh/populationgenomics/metamist)
## Introduction
**Metamist** is a database designed for storing **de-identified** -omics metadata.
## Purpose
The project provides an interface to interact with the -omics database via the Python client as well as the GraphQL + HTTP APIs.
## Features
- **Project-Based Resource Organization**: Every resource in Metamist is associated with a specific project.
- **Access Control**: Access to resources is controlled through a basic permissions system. Users can have one of the following permissions for a project.
- `reader` permission allows viewing of all metadata within a project
- `contributor` permission allows creating comments on metadata within a project
- `writer` permission allows writing of all metadata within a project.
## High-Level Architecture
It comprises three key components:
1. **System-Versioned MariaDB Database**: A robust database system for managing -omics metadata.
2. **Python Web API**: This component is responsible for:
- Managing permissions.
- Storing frequently used queries.
- Providing a GraphQL/HTTP API for efficient querying of the database.
3. **Installable Python Library**: Wraps the Python Web API using the OpenAPI generator, facilitating easier interaction with the system.
4. **Web client UI** A react application that allows viewing the metadata for projects.
## Usage
There are a few different ways that you may need to interact with metamist.
### Working with production metamist data
If you only need to interact with production metamist data, then you do not need to clone this repository or have a working local installation of metamist. You can install the python client from pypi:
```bash
pip install metamist
```
And use it in your python scripts:
```python
from metamist.graphql import gql, query
def main():
my_query = gql("""
{
myProjects {
id
name
}
}
""")
result = query(my_query)
print(result)
if __name__ == "__main__":
main()
```
#### Further resources
- [GraphiQL UI](https://sample-metadata.populationgenomics.org.au/graphql), for exploration of the GraphQL API
- [Python client documentation](https://sample-metadata.populationgenomics.org.au/documentation)
- [Swagger HTTP API documentation](https://sample-metadata.populationgenomics.org.au/swagger)
### Testing scripts against a local database
When writing more complex scripts it is a good idea to test them against a local version of metamist. To do this, follow the [local installation](docs/installation.md) instructions to set up the API. Setting up the web client is optional and only necessary if you need to view your data in the UI.
Once you have the API set up locally, set the `SM_ENVIRONMENT` variable to `local`
```bash
export SM_ENVIRONMENT=local
```
This will tell the metamist python client to point to your local metamist install.
### Developing or contributing to metamist
If you are working on developing features, fixing bugs or want to contribute to metamist in other ways then you will need a local installation of metamist. Follow the [local installation](docs/installation.md) instructions.
### Schema
As of Jan 15, 2024 this schema should reflect the data structure on the tables:
You can also find this at [DbDiagram](https://dbdiagram.io/d/Metamist-Schema-v7-7-6600c875ae072629ced6a1fc).
The codebase contains the following modules worth noting:
- `models` -> General data models + enums
- `db/python/tables` -> Interaction with MariaDB / BigQuery
- `db/python/layers` -> Logic
- `api/graphql` : GraphQL API schema and models
- `api/routes`: HTTP + OpenAPI
And metamist maintains two clients:
- `web` -> React app that consumes a generated Typescript API + GraphQL
- `metamist` -> autogenerated Python API
## License
This project is licensed under the MIT License. You can see it in the [LICENSE](LICENSE) file in the root directory of this source tree.
Raw data
{
"_id": null,
"home_page": "https://github.com/populationgenomics/metamist",
"name": "metamist",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "bioinformatics",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4d/3f/81b8384fb2c1e839af9b90b8b3089a23ddf950b2507e5c531649048dcae3/metamist-7.11.2.tar.gz",
"platform": null,
"description": "# Metamist\n\n[](https://codecov.io/gh/populationgenomics/metamist)\n\n\n## Introduction\n\n**Metamist** is a database designed for storing **de-identified** -omics metadata.\n\n## Purpose\n\nThe project provides an interface to interact with the -omics database via the Python client as well as the GraphQL + HTTP APIs.\n\n## Features\n\n- **Project-Based Resource Organization**: Every resource in Metamist is associated with a specific project.\n- **Access Control**: Access to resources is controlled through a basic permissions system. Users can have one of the following permissions for a project.\n - `reader` permission allows viewing of all metadata within a project\n - `contributor` permission allows creating comments on metadata within a project\n - `writer` permission allows writing of all metadata within a project.\n\n\n## High-Level Architecture\n\nIt comprises three key components:\n\n1. **System-Versioned MariaDB Database**: A robust database system for managing -omics metadata.\n\n2. **Python Web API**: This component is responsible for:\n - Managing permissions.\n - Storing frequently used queries.\n - Providing a GraphQL/HTTP API for efficient querying of the database.\n\n3. **Installable Python Library**: Wraps the Python Web API using the OpenAPI generator, facilitating easier interaction with the system.\n4. **Web client UI** A react application that allows viewing the metadata for projects.\n\n## Usage\n\nThere are a few different ways that you may need to interact with metamist.\n\n### Working with production metamist data\n\nIf you only need to interact with production metamist data, then you do not need to clone this repository or have a working local installation of metamist. You can install the python client from pypi:\n\n```bash\npip install metamist\n```\n\nAnd use it in your python scripts:\n\n```python\nfrom metamist.graphql import gql, query\n\ndef main():\n my_query = gql(\"\"\"\n {\n myProjects {\n id\n name\n }\n }\n \"\"\")\n\n result = query(my_query)\n print(result)\n\n\nif __name__ == \"__main__\":\n main()\n\n```\n\n#### Further resources\n\n- [GraphiQL UI](https://sample-metadata.populationgenomics.org.au/graphql), for exploration of the GraphQL API\n- [Python client documentation](https://sample-metadata.populationgenomics.org.au/documentation)\n- [Swagger HTTP API documentation](https://sample-metadata.populationgenomics.org.au/swagger)\n\n\n### Testing scripts against a local database\n\nWhen writing more complex scripts it is a good idea to test them against a local version of metamist. To do this, follow the [local installation](docs/installation.md) instructions to set up the API. Setting up the web client is optional and only necessary if you need to view your data in the UI.\n\nOnce you have the API set up locally, set the `SM_ENVIRONMENT` variable to `local`\n\n```bash\nexport SM_ENVIRONMENT=local\n```\n\nThis will tell the metamist python client to point to your local metamist install.\n\n\n### Developing or contributing to metamist\n\nIf you are working on developing features, fixing bugs or want to contribute to metamist in other ways then you will need a local installation of metamist. Follow the [local installation](docs/installation.md) instructions.\n\n\n\n### Schema\n\nAs of Jan 15, 2024 this schema should reflect the data structure on the tables:\n\n\n\nYou can also find this at [DbDiagram](https://dbdiagram.io/d/Metamist-Schema-v7-7-6600c875ae072629ced6a1fc).\n\nThe codebase contains the following modules worth noting:\n\n- `models` -> General data models + enums\n- `db/python/tables` -> Interaction with MariaDB / BigQuery\n- `db/python/layers` -> Logic\n- `api/graphql` : GraphQL API schema and models\n- `api/routes`: HTTP + OpenAPI\n\nAnd metamist maintains two clients:\n\n- `web` -> React app that consumes a generated Typescript API + GraphQL\n- `metamist` -> autogenerated Python API\n\n\n## License\n\nThis project is licensed under the MIT License. You can see it in the [LICENSE](LICENSE) file in the root directory of this source tree.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python API for interacting with the Sample API system",
"version": "7.11.2",
"project_urls": {
"Homepage": "https://github.com/populationgenomics/metamist"
},
"split_keywords": [
"bioinformatics"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "4d3f81b8384fb2c1e839af9b90b8b3089a23ddf950b2507e5c531649048dcae3",
"md5": "b70bff2afce293dc39603fd61fc3cb93",
"sha256": "66086270335763f7b71b52b115b38bd1c85be8ea51be7688d7c8aa222009488f"
},
"downloads": -1,
"filename": "metamist-7.11.2.tar.gz",
"has_sig": false,
"md5_digest": "b70bff2afce293dc39603fd61fc3cb93",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 239720,
"upload_time": "2025-10-23T00:09:55",
"upload_time_iso_8601": "2025-10-23T00:09:55.414794Z",
"url": "https://files.pythonhosted.org/packages/4d/3f/81b8384fb2c1e839af9b90b8b3089a23ddf950b2507e5c531649048dcae3/metamist-7.11.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-23 00:09:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "populationgenomics",
"github_project": "metamist",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "aiohappyeyeballs",
"specs": [
[
"==",
"2.6.1"
]
]
},
{
"name": "aiohttp",
"specs": [
[
"==",
"3.12.15"
]
]
},
{
"name": "aiomysql",
"specs": [
[
"==",
"0.2.0"
]
]
},
{
"name": "aiosignal",
"specs": [
[
"==",
"1.4.0"
]
]
},
{
"name": "annotated-types",
"specs": [
[
"==",
"0.7.0"
]
]
},
{
"name": "anyio",
"specs": [
[
"==",
"4.10.0"
]
]
},
{
"name": "async-lru",
"specs": [
[
"==",
"2.0.5"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"25.3.0"
]
]
},
{
"name": "azure-core",
"specs": [
[
"==",
"1.35.0"
]
]
},
{
"name": "azure-storage-blob",
"specs": [
[
"==",
"12.26.0"
]
]
},
{
"name": "azure-storage-file-datalake",
"specs": [
[
"==",
"12.21.0"
]
]
},
{
"name": "backoff",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "boto3",
"specs": [
[
"==",
"1.40.24"
]
]
},
{
"name": "botocore",
"specs": [
[
"==",
"1.40.24"
]
]
},
{
"name": "cachetools",
"specs": [
[
"==",
"5.5.2"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2025.8.3"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"1.17.1"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.3"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.2.1"
]
]
},
{
"name": "cloudpathlib",
"specs": [
[
"==",
"0.22.0"
]
]
},
{
"name": "cpg-utils",
"specs": [
[
"==",
"5.4.2"
]
]
},
{
"name": "cryptography",
"specs": [
[
"==",
"45.0.7"
]
]
},
{
"name": "databases",
"specs": [
[
"==",
"0.9.0"
]
]
},
{
"name": "deprecated",
"specs": [
[
"==",
"1.2.18"
]
]
},
{
"name": "dnspython",
"specs": [
[
"==",
"2.7.0"
]
]
},
{
"name": "duckdb",
"specs": [
[
"==",
"1.1.3"
]
]
},
{
"name": "email-validator",
"specs": [
[
"==",
"2.3.0"
]
]
},
{
"name": "fastapi",
"specs": [
[
"==",
"0.116.1"
]
]
},
{
"name": "fastapi-cli",
"specs": [
[
"==",
"0.0.10"
]
]
},
{
"name": "fastapi-cloud-cli",
"specs": [
[
"==",
"0.1.5"
]
]
},
{
"name": "frozendict",
"specs": [
[
"==",
"2.4.6"
]
]
},
{
"name": "frozenlist",
"specs": [
[
"==",
"1.7.0"
]
]
},
{
"name": "fsspec",
"specs": [
[
"==",
"2024.10.0"
]
]
},
{
"name": "google-api-core",
"specs": [
[
"==",
"2.25.1"
]
]
},
{
"name": "google-auth",
"specs": [
[
"==",
"2.40.3"
]
]
},
{
"name": "google-cloud-appengine-logging",
"specs": [
[
"==",
"1.6.2"
]
]
},
{
"name": "google-cloud-artifact-registry",
"specs": [
[
"==",
"1.15.0"
]
]
},
{
"name": "google-cloud-audit-log",
"specs": [
[
"==",
"0.3.2"
]
]
},
{
"name": "google-cloud-bigquery",
"specs": [
[
"==",
"3.11.4"
]
]
},
{
"name": "google-cloud-core",
"specs": [
[
"==",
"2.4.3"
]
]
},
{
"name": "google-cloud-logging",
"specs": [
[
"==",
"2.7.0"
]
]
},
{
"name": "google-cloud-pubsub",
"specs": [
[
"==",
"2.18.3"
]
]
},
{
"name": "google-cloud-secret-manager",
"specs": [
[
"==",
"2.23.0"
]
]
},
{
"name": "google-cloud-storage",
"specs": [
[
"==",
"1.43.0"
]
]
},
{
"name": "google-crc32c",
"specs": [
[
"==",
"1.7.1"
]
]
},
{
"name": "google-resumable-media",
"specs": [
[
"==",
"2.7.2"
]
]
},
{
"name": "googleapis-common-protos",
"specs": [
[
"==",
"1.70.0"
]
]
},
{
"name": "graphql-core",
"specs": [
[
"==",
"3.2.6"
]
]
},
{
"name": "greenlet",
"specs": [
[
"==",
"3.2.4"
]
]
},
{
"name": "grpc-google-iam-v1",
"specs": [
[
"==",
"0.12.7"
]
]
},
{
"name": "grpcio",
"specs": [
[
"==",
"1.74.0"
]
]
},
{
"name": "grpcio-status",
"specs": [
[
"==",
"1.62.3"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.16.0"
]
]
},
{
"name": "httpcore",
"specs": [
[
"==",
"1.0.9"
]
]
},
{
"name": "httptools",
"specs": [
[
"==",
"0.6.4"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.28.1"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "isodate",
"specs": [
[
"==",
"0.7.2"
]
]
},
{
"name": "itsdangerous",
"specs": [
[
"==",
"2.2.0"
]
]
},
{
"name": "jinja2",
"specs": [
[
"==",
"3.1.6"
]
]
},
{
"name": "jmespath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"4.0.0"
]
]
},
{
"name": "markupsafe",
"specs": [
[
"==",
"3.0.2"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "multidict",
"specs": [
[
"==",
"6.6.4"
]
]
},
{
"name": "orjson",
"specs": [
[
"==",
"3.11.3"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"25.0"
]
]
},
{
"name": "propcache",
"specs": [
[
"==",
"0.3.2"
]
]
},
{
"name": "proto-plus",
"specs": [
[
"==",
"1.26.1"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"4.25.8"
]
]
},
{
"name": "pyarrow",
"specs": [
[
"==",
"18.1.0"
]
]
},
{
"name": "pyasn1",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "pyasn1-modules",
"specs": [
[
"==",
"0.4.2"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.22"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.11.7"
]
]
},
{
"name": "pydantic-core",
"specs": [
[
"==",
"2.33.2"
]
]
},
{
"name": "pydantic-extra-types",
"specs": [
[
"==",
"2.10.5"
]
]
},
{
"name": "pydantic-settings",
"specs": [
[
"==",
"2.10.1"
]
]
},
{
"name": "pygments",
"specs": [
[
"==",
"2.19.2"
]
]
},
{
"name": "pymysql",
"specs": [
[
"==",
"1.1.2"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.1.1"
]
]
},
{
"name": "python-multipart",
"specs": [
[
"==",
"0.0.20"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"==",
"6.0.2"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.5"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"14.1.0"
]
]
},
{
"name": "rich-toolkit",
"specs": [
[
"==",
"0.15.1"
]
]
},
{
"name": "rignore",
"specs": [
[
"==",
"0.6.4"
]
]
},
{
"name": "rsa",
"specs": [
[
"==",
"4.9.1"
]
]
},
{
"name": "s3transfer",
"specs": [
[
"==",
"0.13.1"
]
]
},
{
"name": "sentry-sdk",
"specs": [
[
"==",
"2.36.0"
]
]
},
{
"name": "shellingham",
"specs": [
[
"==",
"1.5.4"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "slack-sdk",
"specs": [
[
"==",
"3.20.2"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "sqlalchemy",
"specs": [
[
"==",
"2.0.43"
]
]
},
{
"name": "starlette",
"specs": [
[
"==",
"0.47.3"
]
]
},
{
"name": "strawberry-graphql",
"specs": [
[
"==",
"0.268.1"
]
]
},
{
"name": "tabulate",
"specs": [
[
"==",
"0.9.0"
]
]
},
{
"name": "toml",
"specs": [
[
"==",
"0.10.2"
]
]
},
{
"name": "typer",
"specs": [
[
"==",
"0.17.3"
]
]
},
{
"name": "typing-extensions",
"specs": [
[
"==",
"4.15.0"
]
]
},
{
"name": "typing-inspection",
"specs": [
[
"==",
"0.4.1"
]
]
},
{
"name": "ujson",
"specs": [
[
"==",
"5.11.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.5.0"
]
]
},
{
"name": "uvicorn",
"specs": [
[
"==",
"0.34.3"
]
]
},
{
"name": "uvloop",
"specs": [
[
"==",
"0.21.0"
]
]
},
{
"name": "watchfiles",
"specs": [
[
"==",
"1.1.0"
]
]
},
{
"name": "websockets",
"specs": [
[
"==",
"15.0.1"
]
]
},
{
"name": "wrapt",
"specs": [
[
"==",
"1.17.3"
]
]
},
{
"name": "yarl",
"specs": [
[
"==",
"1.20.1"
]
]
}
],
"lcname": "metamist"
}
