metamist
# Metamist
[](https://codecov.io/gh/populationgenomics/metamist)
## Introduction
**Metamist** is a database designed for storing **de-identified** -omics metadata.
## Purpose
The project provides an interface to interact with the -omics database via the Python client as well as the GraphQL + HTTP APIs.
## Features
- **Project-Based Resource Organization**: Every resource in Metamist is associated with a specific project.
- **Access Control**: Access to resources is controlled through membership in specific Google Groups:
- `dataset-sample-metadata-main-read`: For read-only access.
- `dataset-sample-metadata-main-write`: For write access.
- **Efficiency Note**: Members of Google Groups are cached in a blob to optimize performance, as group-membership identity lookups can be slow.
## High-Level Architecture
It comprises three key components:
1. **System-Versioned MariaDB Database**: A robust database system for managing -omics metadata.
2. **Python Web API**: This component is responsible for:
- Managing permissions.
- Storing frequently used queries.
- Providing a GraphQL/HTTP API for efficient querying of the database.
3. **Installable Python Library**: Wraps the Python Web API using the OpenAPI generator, facilitating easier interaction with the system.
### Schema
As of Jan 15, 2024 this schema should reflect the data structure on the tables:
You can also find this at [DbDiagram](https://dbdiagram.io/d/Metamist-Schema-v7-7-6600c875ae072629ced6a1fc).
The codebase contains the following modules worth noting:
- `models` -> General data models + enums
- `db/python/tables` -> Interaction with MariaDB / BigQuery
- `db/python/layers` -> Logic
- `api/graphql` : GraphQL
- `api/routes`: HTTP + OpenAPI
And metamist maintains two clients:
- `web` -> React app that consumes a generated Typescript API + GraphQL
- `metamist` -> autogenerated Python API
## Installation and Running Locally
- [Installation and developer setup](docs/installation.md)
## License
This project is licensed under the MIT License. You can see it in the [LICENSE](LICENSE) file in the root directory of this source tree.
Raw data
{
"_id": null,
"home_page": "https://github.com/populationgenomics/metamist",
"name": "metamist",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "bioinformatics",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/2e/d8/feab3a293913cabc6bef8b45521bb6b0f72015428faed3856a883ab7733e/metamist-7.8.0.tar.gz",
"platform": null,
"description": "# Metamist\n\n[](https://codecov.io/gh/populationgenomics/metamist)\n\n\n## Introduction\n\n**Metamist** is a database designed for storing **de-identified** -omics metadata.\n\n## Purpose\n\nThe project provides an interface to interact with the -omics database via the Python client as well as the GraphQL + HTTP APIs.\n\n## Features\n\n- **Project-Based Resource Organization**: Every resource in Metamist is associated with a specific project.\n- **Access Control**: Access to resources is controlled through membership in specific Google Groups:\n - `dataset-sample-metadata-main-read`: For read-only access.\n - `dataset-sample-metadata-main-write`: For write access.\n- **Efficiency Note**: Members of Google Groups are cached in a blob to optimize performance, as group-membership identity lookups can be slow.\n\n## High-Level Architecture\n\nIt comprises three key components:\n\n1. **System-Versioned MariaDB Database**: A robust database system for managing -omics metadata.\n\n2. **Python Web API**: This component is responsible for:\n - Managing permissions.\n - Storing frequently used queries.\n - Providing a GraphQL/HTTP API for efficient querying of the database.\n\n3. **Installable Python Library**: Wraps the Python Web API using the OpenAPI generator, facilitating easier interaction with the system.\n\n### Schema\n\nAs of Jan 15, 2024 this schema should reflect the data structure on the tables:\n\n\n\nYou can also find this at [DbDiagram](https://dbdiagram.io/d/Metamist-Schema-v7-7-6600c875ae072629ced6a1fc).\n\nThe codebase contains the following modules worth noting:\n\n- `models` -> General data models + enums\n- `db/python/tables` -> Interaction with MariaDB / BigQuery\n- `db/python/layers` -> Logic\n- `api/graphql` : GraphQL\n- `api/routes`: HTTP + OpenAPI\n\nAnd metamist maintains two clients:\n\n- `web` -> React app that consumes a generated Typescript API + GraphQL\n- `metamist` -> autogenerated Python API\n\n## Installation and Running Locally\n\n- [Installation and developer setup](docs/installation.md)\n\n## License\n\nThis project is licensed under the MIT License. You can see it in the [LICENSE](LICENSE) file in the root directory of this source tree.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python API for interacting with the Sample API system",
"version": "7.8.0",
"project_urls": {
"Homepage": "https://github.com/populationgenomics/metamist"
},
"split_keywords": [
"bioinformatics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2ed8feab3a293913cabc6bef8b45521bb6b0f72015428faed3856a883ab7733e",
"md5": "c3858a6bfbd7cd0f96482f06c98744f9",
"sha256": "e61c83f4cef275a465d858e52f3d2bf929fd3baecf01b17b4b166d18ec92531d"
},
"downloads": -1,
"filename": "metamist-7.8.0.tar.gz",
"has_sig": false,
"md5_digest": "c3858a6bfbd7cd0f96482f06c98744f9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 218024,
"upload_time": "2025-01-15T05:18:45",
"upload_time_iso_8601": "2025-01-15T05:18:45.434890Z",
"url": "https://files.pythonhosted.org/packages/2e/d8/feab3a293913cabc6bef8b45521bb6b0f72015428faed3856a883ab7733e/metamist-7.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-15 05:18:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "populationgenomics",
"github_project": "metamist",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "aiohappyeyeballs",
"specs": [
[
"==",
"2.4.4"
]
]
},
{
"name": "aiohttp",
"specs": [
[
"==",
"3.11.11"
]
]
},
{
"name": "aiomysql",
"specs": [
[
"==",
"0.2.0"
]
]
},
{
"name": "aiosignal",
"specs": [
[
"==",
"1.3.2"
]
]
},
{
"name": "annotated-types",
"specs": [
[
"==",
"0.7.0"
]
]
},
{
"name": "anyio",
"specs": [
[
"==",
"4.8.0"
]
]
},
{
"name": "async-lru",
"specs": [
[
"==",
"2.0.4"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"24.3.0"
]
]
},
{
"name": "azure-core",
"specs": [
[
"==",
"1.32.0"
]
]
},
{
"name": "azure-storage-blob",
"specs": [
[
"==",
"12.24.0"
]
]
},
{
"name": "azure-storage-file-datalake",
"specs": [
[
"==",
"12.18.0"
]
]
},
{
"name": "backoff",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "boto3",
"specs": [
[
"==",
"1.35.95"
]
]
},
{
"name": "botocore",
"specs": [
[
"==",
"1.35.95"
]
]
},
{
"name": "cachetools",
"specs": [
[
"==",
"5.5.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.12.14"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"1.17.1"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.1"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.1.8"
]
]
},
{
"name": "cloudpathlib",
"specs": [
[
"==",
"0.20.0"
]
]
},
{
"name": "cpg-utils",
"specs": [
[
"==",
"5.2.0"
]
]
},
{
"name": "cryptography",
"specs": [
[
"==",
"44.0.0"
]
]
},
{
"name": "databases",
"specs": [
[
"==",
"0.9.0"
]
]
},
{
"name": "deprecated",
"specs": [
[
"==",
"1.2.15"
]
]
},
{
"name": "dnspython",
"specs": [
[
"==",
"2.7.0"
]
]
},
{
"name": "duckdb",
"specs": [
[
"==",
"1.1.3"
]
]
},
{
"name": "email-validator",
"specs": [
[
"==",
"2.2.0"
]
]
},
{
"name": "fastapi",
"specs": [
[
"==",
"0.115.6"
]
]
},
{
"name": "fastapi-cli",
"specs": [
[
"==",
"0.0.7"
]
]
},
{
"name": "frozendict",
"specs": [
[
"==",
"2.4.6"
]
]
},
{
"name": "frozenlist",
"specs": [
[
"==",
"1.5.0"
]
]
},
{
"name": "fsspec",
"specs": [
[
"==",
"2024.10.0"
]
]
},
{
"name": "google-api-core",
"specs": [
[
"==",
"2.24.0"
]
]
},
{
"name": "google-auth",
"specs": [
[
"==",
"2.37.0"
]
]
},
{
"name": "google-cloud-appengine-logging",
"specs": [
[
"==",
"1.5.0"
]
]
},
{
"name": "google-cloud-audit-log",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "google-cloud-bigquery",
"specs": [
[
"==",
"3.11.4"
]
]
},
{
"name": "google-cloud-core",
"specs": [
[
"==",
"2.4.1"
]
]
},
{
"name": "google-cloud-logging",
"specs": [
[
"==",
"2.7.0"
]
]
},
{
"name": "google-cloud-pubsub",
"specs": [
[
"==",
"2.18.3"
]
]
},
{
"name": "google-cloud-secret-manager",
"specs": [
[
"==",
"2.22.0"
]
]
},
{
"name": "google-cloud-storage",
"specs": [
[
"==",
"1.43.0"
]
]
},
{
"name": "google-crc32c",
"specs": [
[
"==",
"1.6.0"
]
]
},
{
"name": "google-resumable-media",
"specs": [
[
"==",
"2.7.2"
]
]
},
{
"name": "googleapis-common-protos",
"specs": [
[
"==",
"1.66.0"
]
]
},
{
"name": "graphql-core",
"specs": [
[
"==",
"3.2.5"
]
]
},
{
"name": "greenlet",
"specs": [
[
"==",
"3.1.1"
]
]
},
{
"name": "grpc-google-iam-v1",
"specs": [
[
"==",
"0.12.7"
]
]
},
{
"name": "grpcio",
"specs": [
[
"==",
"1.69.0"
]
]
},
{
"name": "grpcio-status",
"specs": [
[
"==",
"1.62.3"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "httpcore",
"specs": [
[
"==",
"1.0.7"
]
]
},
{
"name": "httptools",
"specs": [
[
"==",
"0.6.4"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.28.1"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "isodate",
"specs": [
[
"==",
"0.7.2"
]
]
},
{
"name": "itsdangerous",
"specs": [
[
"==",
"2.2.0"
]
]
},
{
"name": "jinja2",
"specs": [
[
"==",
"3.1.5"
]
]
},
{
"name": "jmespath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "markupsafe",
"specs": [
[
"==",
"3.0.2"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "multidict",
"specs": [
[
"==",
"6.1.0"
]
]
},
{
"name": "orjson",
"specs": [
[
"==",
"3.10.14"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"24.2"
]
]
},
{
"name": "propcache",
"specs": [
[
"==",
"0.2.1"
]
]
},
{
"name": "proto-plus",
"specs": [
[
"==",
"1.25.0"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"4.25.5"
]
]
},
{
"name": "pyarrow",
"specs": [
[
"==",
"18.1.0"
]
]
},
{
"name": "pyasn1",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "pyasn1-modules",
"specs": [
[
"==",
"0.4.1"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.22"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.10.4"
]
]
},
{
"name": "pydantic-core",
"specs": [
[
"==",
"2.27.2"
]
]
},
{
"name": "pydantic-extra-types",
"specs": [
[
"==",
"2.10.1"
]
]
},
{
"name": "pydantic-settings",
"specs": [
[
"==",
"2.7.1"
]
]
},
{
"name": "pygments",
"specs": [
[
"==",
"2.19.1"
]
]
},
{
"name": "pymysql",
"specs": [
[
"==",
"1.1.1"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "python-multipart",
"specs": [
[
"==",
"0.0.20"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"==",
"6.0.2"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.9.4"
]
]
},
{
"name": "rich-toolkit",
"specs": [
[
"==",
"0.12.0"
]
]
},
{
"name": "rsa",
"specs": [
[
"==",
"4.9"
]
]
},
{
"name": "s3transfer",
"specs": [
[
"==",
"0.10.4"
]
]
},
{
"name": "shellingham",
"specs": [
[
"==",
"1.5.4"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "slack-sdk",
"specs": [
[
"==",
"3.20.2"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "sqlalchemy",
"specs": [
[
"==",
"2.0.36"
]
]
},
{
"name": "starlette",
"specs": [
[
"==",
"0.41.3"
]
]
},
{
"name": "strawberry-graphql",
"specs": [
[
"==",
"0.257.0"
]
]
},
{
"name": "tabulate",
"specs": [
[
"==",
"0.9.0"
]
]
},
{
"name": "toml",
"specs": [
[
"==",
"0.10.2"
]
]
},
{
"name": "typer",
"specs": [
[
"==",
"0.15.1"
]
]
},
{
"name": "typing-extensions",
"specs": [
[
"==",
"4.12.2"
]
]
},
{
"name": "ujson",
"specs": [
[
"==",
"5.10.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.3.0"
]
]
},
{
"name": "uvicorn",
"specs": [
[
"==",
"0.29.0"
]
]
},
{
"name": "uvloop",
"specs": [
[
"==",
"0.21.0"
]
]
},
{
"name": "watchfiles",
"specs": [
[
"==",
"1.0.3"
]
]
},
{
"name": "websockets",
"specs": [
[
"==",
"14.1"
]
]
},
{
"name": "wrapt",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "yarl",
"specs": [
[
"==",
"1.18.3"
]
]
}
],
"lcname": "metamist"
}
