# <div align="center"><i>datalab</i></div>
<div align="center" style="padding-bottom: 5px">
<a href="https://demo.datalab-org.io"><img src="https://img.shields.io/badge/try_it_out!-public_demo_server-orange?logo=firefox"></a>
</div>
<div align="center">
<a href="https://github.com/datalab-org/datalab/releases"><img src="https://badgen.net/github/release/datalab-org/datalab?icon=github&color=blue"></a>
<a href="https://github.com/datalab-org/datalab#MIT-1-ov-file"><img src="https://badgen.net/github/license/datalab-org/datalab?icon=license&color=purple"></a>
</div>
<div align="center">
<a href="https://github.com/datalab-org/datalab/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/datalab-org/datalab/ci.yml?logo=github"></a>
<a href="https://cloud.cypress.io/projects/4kqx5i/runs"><img src="https://img.shields.io/endpoint?url=https://cloud.cypress.io/badge/simple/4kqx5i/main&style=flat&logo=cypress"></a>
<a href="https://the-datalab.readthedocs.io/en/latest/?badge=latest"><img src="https://img.shields.io/readthedocs/the-datalab?logo=readthedocs"></a>
</div>
<div align="center">
<a href="https://github.com/datalab-org/datalab-ansible-terraform">
<img alt="Static Badge" src="https://img.shields.io/badge/Ansible-playbook-white?logo=ansible">
</a>
<a href="https://pypi.org/project/datalab-api">
<img alt="PyPI - Version" src="https://img.shields.io/pypi/v/datalab-api?logo=pypi&label=Python%20API">
</a>
</div>
<div align="center">
<a href="https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g"><img src="https://img.shields.io/badge/Slack-chat_with_us-yellow?logo=slack"></a>
</div>
_datalab_ is a user-friendly, open-source platform that can capture all the experimental data and metadata produced in a scientific lab, targeted (broadly) at materials chemistry but with customisability and extensability in mind.
_datalab_ records data and metadata securely and makes it accessible and reusable by both humans and machines _via_ the web UI and API, respectively.
_datalab_ can be self-hosted and managed deployments are also available.
You can try the demo deployment at [demo.datalab-org.io](https://demo.datalab-org.io/) and read the online documentation at [docs.datalab-org.io](https://docs.datalab-org.io) with release notes and changelog available on [GitHub](https://github.com/datalab-org/datalab/releases/) and [online](https://docs.datalab-org.io/en/latest/CHANGELOG).
Features:
* Capture and store sample and device metadata
* Connect and sync raw data directly and from laboratory instruments
* Built-in support for multiple characterisation techniques (XRD, NMR, echem, TEM, TGA, Mass Spec, Raman and more).
* Capture scientific context: store the graph of relationships between research objects.
* [Python API](https://github.com/datalab-org/datalab-api) for programmatic access to your lab's data enabling custom analysis and automation.
* Join the [_datalab_ federation](https://github.com/datalab-org/datalab-federation): you can add your _datalab_ to the federation for additional shared features.
* [Plugin ecosystem](https://docs.datalab-org.io/en/latest/plugins) allowing for custom data blocks, [AI integration](https://github.com/datalab-org/yellowhammer) and other instance-specific code.
* [Deployment and infrastructure automation](https://github.com/datalab-industries/datalab-ansible-terraform) via Ansible playbooks.
<div align="center">
<video width="400" controls src="https://github.com/datalab-org/datalab/assets/7916000/0065cdd6-a5f0-4391-b192-0137fe208acc">
</video>
</div>
> [!NOTE]
> You may be looking for the identically named project [DataLab](https://datalab-platform.com) for signal processing, which also has plugins, clients and other similar concepts!
## Getting started
To set up your own _datalab_ instance or to get started with development, you can follow the installation and deployment instructions in the [online documentation](https://docs.datalab-org.io/en/latest/INSTALL).
We can also provide paid managed deployments via [_datalab industries ltd._](https://datalab.industries): contact us at [hello@datalab.industries](mailto:hello@datalab.industries).
## Design philosophy and architecture
The _datalab_ architecture is shown below:
<center>
```mermaid
graph TD
classDef actor fill:#0066CC,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;
classDef clientInterface fill:#00AA44,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;
classDef coreComponent fill:#FF6600,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;
classDef umbrellaLabel fill:#666666,fill-opacity:0.3,stroke:#666,stroke-width:1px,color:#000,rx:5,ry:5,text-align:center;
classDef subgraphStyle fill:#f9f9f9,fill-opacity:0.1,stroke:#ccc,stroke-width:1px;
subgraph ExternalActors [External actors]
direction TB
User[User]
Machine[Machine]
end
class User,Machine actor;
class ExternalActors subgraphStyle;
UmbrellaDesc["Raw instrument data,<br>annotations, connections"]
class UmbrellaDesc umbrellaLabel;
subgraph ClientInterfaces [Client interfaces]
direction TB
BrowserApp[_datalab_<br>Browser app]
PythonAPI[_datalab_<br>Python API]
end
class BrowserApp,PythonAPI clientInterface;
class ClientInterfaces subgraphStyle;
subgraph Backend
direction TB
RESTAPI[_datalab_<br>REST API]
MongoDB[MongoDB Database]
DataLake[Data Lake]
end
class RESTAPI,MongoDB,DataLake coreComponent;
class Backend subgraphStyle;
User <-- "User data I/O" --> UmbrellaDesc;
Machine <-- "Machine data I/O" --> UmbrellaDesc;
UmbrellaDesc <-- "_via_ GUI" --> BrowserApp;
UmbrellaDesc <-- "_via_ scripts" --> PythonAPI;
BrowserApp <-- "HTTP (Data exchange)" --> RESTAPI;
PythonAPI <-- "API calls (Data exchange)" --> RESTAPI;
RESTAPI <-- "Annotations, connections" --> MongoDB;
RESTAPI <-- "Raw and structured characterisation data" --> DataLake;
linkStyle 0 stroke:#666,stroke-width:3px
linkStyle 1 stroke:#666,stroke-width:3px
linkStyle 2 stroke:#666,stroke-width:3px
linkStyle 3 stroke:#666,stroke-width:3px
linkStyle 4 stroke:#666,stroke-width:3px
linkStyle 5 stroke:#666,stroke-width:3px
linkStyle 6 stroke:#666,stroke-width:3px
linkStyle 7 stroke:#666,stroke-width:3px
click PythonAPI "https://github.com/datalab-org/datalab-api" "datalab Python API on GitHub" _blank
click BrowserApp "https://github.com/datalab-org/datalab/tree/main/webapp" "datalab Browser App on GitHub" _blank
click RESTAPI "https://github.com/datalab-org/datalab/tree/main/pydatalab" "pydatalab REST API on GitHub" _blank
```
</center>
The main aim of *datalab* is to provide a platform for capturing the significant amounts of long-tail experimental data and metadata produced in a typical lab, and enable storage, filtering and future data re-use by humans and machines. *datalab* is targeted (broadly) at materials chemistry labs but with customisability and extensability in mind.
The platform provides researchers with a way to record sample- and cell-specific metadata, attach and sync raw data from instruments, and perform analysis and visualisation of many characterisation techniques in the browser (XRD, NMR, electrochemical cycling, TEM, TGA, Mass Spec, Raman).
Importantly, *datalab* stores a network of interconnected research objects in the lab, such that individual pieces of data are stored with the context needed to make them scientifically useful.
## License
This software is released under the conditions of the MIT license.
Please see [LICENSE](./LICENSE) for the full text of the license.
## Contact
We are available for consultations on setting up and managing *datalab* deployments, as well as collaborating on or sponsoring additions of new features and techniques. Please contact Josh or Matthew on their academic emails, or join the [public *datalab* Slack workspace](https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g).
## Contributions
This software was conceived and developed by:
- [Prof Joshua Bocarsly](https://jdbocarsly.github.io) ([Department of Chemistry, University of Houston](https://www.uh.edu/nsm/chemistry), previously [Department of Chemistry, University of Cambridge](https://www.ch.cam.ac.uk/))
- [Dr Matthew Evans](https://ml-evs.science) ([MODL-IMCN,
UCLouvain](https://uclouvain.be/en/research-institutes/imcn/modl) & [Matgenix](https://matgenix.com))
with support from the group of [Professor Clare Grey](https://grey.group.ch.cam.ac.uk/group) (University of Cambridge), and major contributions from:
- [Benjamin Charmes](https://github.com/BenjaminCharmes)
- [Dr Ben Smith](https://github.com/be-smith/)
- [Dr Yue Wu](https://github.com/yue-here)
plus many contributions, feedback and testing performed by other members of the community, in particular, the groups of [Prof Matt Cliffe](https://cliffegroup.co.uk) (University of Nottingham) and [Dr Peter Kraus](https://www.tu.berlin/en/concat) (TUBerlin).
A full list of code contributions can be found on [GitHub](https://github.com/datalab-org/datalab/graphs/contributors).
## Funding
Contributions to _datalab_ have been supported by a mixture of academic funding and consultancy work through [_datalab industries ltd_](https://datalab.industries).
In particular, the developers thank:
- Initial proof-of-concept funding from the European Union's Horizon 2020 research and innovation programme under grant agreement 957189 (DOI: [10.3030/957189](https://doi.org/10.3030/957189)), the [Battery Interface Genome - Materials Acceleration Platform (BIG-MAP)](https://www.big-map.eu), as an external stakeholder project.
- The BEWARE fellowship scheme of the [Wallonie recherche SPW](https://recherche.wallonie.be/en/home.html) for funding under the European Commission's Marie Curie-Skłodowska Action (COFUND 847587).
Raw data
{
"_id": null,
"home_page": null,
"name": "datalab-server",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.10",
"maintainer_email": null,
"keywords": "research data management, materials, chemistry",
"author": null,
"author_email": "datalab development team <dev@datalab-org.io>",
"download_url": "https://files.pythonhosted.org/packages/18/b7/70d6a03f2287fcea2dea7cec0108cfaecac936a94e4fa884254c7e3d7b18/datalab_server-0.6.2.tar.gz",
"platform": null,
"description": "# <div align=\"center\"><i>datalab</i></div>\n\n<div align=\"center\" style=\"padding-bottom: 5px\">\n<a href=\"https://demo.datalab-org.io\"><img src=\"https://img.shields.io/badge/try_it_out!-public_demo_server-orange?logo=firefox\"></a>\n</div>\n\n<div align=\"center\">\n<a href=\"https://github.com/datalab-org/datalab/releases\"><img src=\"https://badgen.net/github/release/datalab-org/datalab?icon=github&color=blue\"></a>\n<a href=\"https://github.com/datalab-org/datalab#MIT-1-ov-file\"><img src=\"https://badgen.net/github/license/datalab-org/datalab?icon=license&color=purple\"></a>\n</div>\n\n<div align=\"center\">\n<a href=\"https://github.com/datalab-org/datalab/actions/workflows/ci.yml\"><img src=\"https://img.shields.io/github/actions/workflow/status/datalab-org/datalab/ci.yml?logo=github\"></a>\n<a href=\"https://cloud.cypress.io/projects/4kqx5i/runs\"><img src=\"https://img.shields.io/endpoint?url=https://cloud.cypress.io/badge/simple/4kqx5i/main&style=flat&logo=cypress\"></a>\n<a href=\"https://the-datalab.readthedocs.io/en/latest/?badge=latest\"><img src=\"https://img.shields.io/readthedocs/the-datalab?logo=readthedocs\"></a>\n</div>\n\n<div align=\"center\">\n<a href=\"https://github.com/datalab-org/datalab-ansible-terraform\">\n <img alt=\"Static Badge\" src=\"https://img.shields.io/badge/Ansible-playbook-white?logo=ansible\">\n</a>\n<a href=\"https://pypi.org/project/datalab-api\">\n <img alt=\"PyPI - Version\" src=\"https://img.shields.io/pypi/v/datalab-api?logo=pypi&label=Python%20API\">\n</a>\n</div>\n\n<div align=\"center\">\n<a href=\"https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g\"><img src=\"https://img.shields.io/badge/Slack-chat_with_us-yellow?logo=slack\"></a>\n</div>\n\n_datalab_ is a user-friendly, open-source platform that can capture all the experimental data and metadata produced in a scientific lab, targeted (broadly) at materials chemistry but with customisability and extensability in mind.\n_datalab_ records data and metadata securely and makes it accessible and reusable by both humans and machines _via_ the web UI and API, respectively.\n_datalab_ can be self-hosted and managed deployments are also available.\n\nYou can try the demo deployment at [demo.datalab-org.io](https://demo.datalab-org.io/) and read the online documentation at [docs.datalab-org.io](https://docs.datalab-org.io) with release notes and changelog available on [GitHub](https://github.com/datalab-org/datalab/releases/) and [online](https://docs.datalab-org.io/en/latest/CHANGELOG).\n\nFeatures:\n\n* Capture and store sample and device metadata\n* Connect and sync raw data directly and from laboratory instruments\n* Built-in support for multiple characterisation techniques (XRD, NMR, echem, TEM, TGA, Mass Spec, Raman and more).\n* Capture scientific context: store the graph of relationships between research objects.\n* [Python API](https://github.com/datalab-org/datalab-api) for programmatic access to your lab's data enabling custom analysis and automation.\n* Join the [_datalab_ federation](https://github.com/datalab-org/datalab-federation): you can add your _datalab_ to the federation for additional shared features.\n* [Plugin ecosystem](https://docs.datalab-org.io/en/latest/plugins) allowing for custom data blocks, [AI integration](https://github.com/datalab-org/yellowhammer) and other instance-specific code.\n* [Deployment and infrastructure automation](https://github.com/datalab-industries/datalab-ansible-terraform) via Ansible playbooks.\n\n<div align=\"center\">\n<video width=\"400\" controls src=\"https://github.com/datalab-org/datalab/assets/7916000/0065cdd6-a5f0-4391-b192-0137fe208acc\">\n</video>\n</div>\n\n> [!NOTE]\n> You may be looking for the identically named project [DataLab](https://datalab-platform.com) for signal processing, which also has plugins, clients and other similar concepts!\n\n## Getting started\n\nTo set up your own _datalab_ instance or to get started with development, you can follow the installation and deployment instructions in the [online documentation](https://docs.datalab-org.io/en/latest/INSTALL).\n\nWe can also provide paid managed deployments via [_datalab industries ltd._](https://datalab.industries): contact us at [hello@datalab.industries](mailto:hello@datalab.industries).\n\n## Design philosophy and architecture\n\nThe _datalab_ architecture is shown below:\n\n<center>\n\n```mermaid\ngraph TD\nclassDef actor fill:#0066CC,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;\nclassDef clientInterface fill:#00AA44,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;\nclassDef coreComponent fill:#FF6600,fill-opacity:0.3,stroke:#333,stroke-width:2px,color:#000;\nclassDef umbrellaLabel fill:#666666,fill-opacity:0.3,stroke:#666,stroke-width:1px,color:#000,rx:5,ry:5,text-align:center;\nclassDef subgraphStyle fill:#f9f9f9,fill-opacity:0.1,stroke:#ccc,stroke-width:1px;\n\n subgraph ExternalActors [External actors]\n direction TB\n User[User]\n Machine[Machine]\n end\n class User,Machine actor;\n class ExternalActors subgraphStyle;\n\n UmbrellaDesc[\"Raw instrument data,<br>annotations, connections\"]\n class UmbrellaDesc umbrellaLabel;\n\n subgraph ClientInterfaces [Client interfaces]\n direction TB\n BrowserApp[_datalab_<br>Browser app]\n PythonAPI[_datalab_<br>Python API]\n end\n class BrowserApp,PythonAPI clientInterface;\n class ClientInterfaces subgraphStyle;\n\n subgraph Backend\n direction TB\n RESTAPI[_datalab_<br>REST API]\n MongoDB[MongoDB Database]\n DataLake[Data Lake]\n end\n class RESTAPI,MongoDB,DataLake coreComponent;\n class Backend subgraphStyle;\n\n User <-- \"User data I/O\" --> UmbrellaDesc;\n Machine <-- \"Machine data I/O\" --> UmbrellaDesc;\n\n UmbrellaDesc <-- \"_via_ GUI\" --> BrowserApp;\n UmbrellaDesc <-- \"_via_ scripts\" --> PythonAPI;\n\n BrowserApp <-- \"HTTP (Data exchange)\" --> RESTAPI;\n PythonAPI <-- \"API calls (Data exchange)\" --> RESTAPI;\n\n RESTAPI <-- \"Annotations, connections\" --> MongoDB;\n RESTAPI <-- \"Raw and structured characterisation data\" --> DataLake;\n\n linkStyle 0 stroke:#666,stroke-width:3px\n linkStyle 1 stroke:#666,stroke-width:3px\n linkStyle 2 stroke:#666,stroke-width:3px\n linkStyle 3 stroke:#666,stroke-width:3px\n linkStyle 4 stroke:#666,stroke-width:3px\n linkStyle 5 stroke:#666,stroke-width:3px\n linkStyle 6 stroke:#666,stroke-width:3px\n linkStyle 7 stroke:#666,stroke-width:3px\n\n click PythonAPI \"https://github.com/datalab-org/datalab-api\" \"datalab Python API on GitHub\" _blank\n click BrowserApp \"https://github.com/datalab-org/datalab/tree/main/webapp\" \"datalab Browser App on GitHub\" _blank\n click RESTAPI \"https://github.com/datalab-org/datalab/tree/main/pydatalab\" \"pydatalab REST API on GitHub\" _blank\n```\n\n</center>\n\nThe main aim of *datalab* is to provide a platform for capturing the significant amounts of long-tail experimental data and metadata produced in a typical lab, and enable storage, filtering and future data re-use by humans and machines. *datalab* is targeted (broadly) at materials chemistry labs but with customisability and extensability in mind.\n\nThe platform provides researchers with a way to record sample- and cell-specific metadata, attach and sync raw data from instruments, and perform analysis and visualisation of many characterisation techniques in the browser (XRD, NMR, electrochemical cycling, TEM, TGA, Mass Spec, Raman).\n\nImportantly, *datalab* stores a network of interconnected research objects in the lab, such that individual pieces of data are stored with the context needed to make them scientifically useful.\n\n## License\n\nThis software is released under the conditions of the MIT license.\nPlease see [LICENSE](./LICENSE) for the full text of the license.\n\n## Contact\n\nWe are available for consultations on setting up and managing *datalab* deployments, as well as collaborating on or sponsoring additions of new features and techniques. Please contact Josh or Matthew on their academic emails, or join the [public *datalab* Slack workspace](https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g).\n\n## Contributions\n\nThis software was conceived and developed by:\n\n- [Prof Joshua Bocarsly](https://jdbocarsly.github.io) ([Department of Chemistry, University of Houston](https://www.uh.edu/nsm/chemistry), previously [Department of Chemistry, University of Cambridge](https://www.ch.cam.ac.uk/))\n- [Dr Matthew Evans](https://ml-evs.science) ([MODL-IMCN,\n UCLouvain](https://uclouvain.be/en/research-institutes/imcn/modl) & [Matgenix](https://matgenix.com))\n\nwith support from the group of [Professor Clare Grey](https://grey.group.ch.cam.ac.uk/group) (University of Cambridge), and major contributions from:\n\n- [Benjamin Charmes](https://github.com/BenjaminCharmes)\n- [Dr Ben Smith](https://github.com/be-smith/)\n- [Dr Yue Wu](https://github.com/yue-here)\n\nplus many contributions, feedback and testing performed by other members of the community, in particular, the groups of [Prof Matt Cliffe](https://cliffegroup.co.uk) (University of Nottingham) and [Dr Peter Kraus](https://www.tu.berlin/en/concat) (TUBerlin).\n\nA full list of code contributions can be found on [GitHub](https://github.com/datalab-org/datalab/graphs/contributors).\n\n## Funding\n\nContributions to _datalab_ have been supported by a mixture of academic funding and consultancy work through [_datalab industries ltd_](https://datalab.industries).\n\nIn particular, the developers thank:\n\n- Initial proof-of-concept funding from the European Union's Horizon 2020 research and innovation programme under grant agreement 957189 (DOI: [10.3030/957189](https://doi.org/10.3030/957189)), the [Battery Interface Genome - Materials Acceleration Platform (BIG-MAP)](https://www.big-map.eu), as an external stakeholder project.\n- The BEWARE fellowship scheme of the [Wallonie recherche SPW](https://recherche.wallonie.be/en/home.html) for funding under the European Commission's Marie Curie-Sk\u0142odowska Action (COFUND 847587).\n",
"bugtrack_url": null,
"license": null,
"summary": "datalab is a research data management platform for materials science and chemistry.",
"version": "0.6.2",
"project_urls": {
"changelog": "https://github.com/datalab-org/datalab/releases",
"documentation": "https://docs.datalab-org.io",
"homepage": "https://github.com/datalab-org/datalab",
"repository": "https://github.com/datalab-org/datalab"
},
"split_keywords": [
"research data management",
" materials",
" chemistry"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "06211a64fb90507839fab27ac34f59ae2425688ff677d50ac4a7c41ad15775cf",
"md5": "1c9704b9bf18158a551405b9c98ba092",
"sha256": "340aac22690a1a9980ececbb3aa62d7d4c68e4654cea271d0b12d4a91b168c20"
},
"downloads": -1,
"filename": "datalab_server-0.6.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1c9704b9bf18158a551405b9c98ba092",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.10",
"size": 134752,
"upload_time": "2025-08-20T19:34:36",
"upload_time_iso_8601": "2025-08-20T19:34:36.016124Z",
"url": "https://files.pythonhosted.org/packages/06/21/1a64fb90507839fab27ac34f59ae2425688ff677d50ac4a7c41ad15775cf/datalab_server-0.6.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "18b770d6a03f2287fcea2dea7cec0108cfaecac936a94e4fa884254c7e3d7b18",
"md5": "2425a8f5a4de1496cb54d1f3e48b88f9",
"sha256": "c4268adb7740e9fbfa9682725cffee109273765c9d47bf709bc95961a2267322"
},
"downloads": -1,
"filename": "datalab_server-0.6.2.tar.gz",
"has_sig": false,
"md5_digest": "2425a8f5a4de1496cb54d1f3e48b88f9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.10",
"size": 4582546,
"upload_time": "2025-08-20T19:34:37",
"upload_time_iso_8601": "2025-08-20T19:34:37.344556Z",
"url": "https://files.pythonhosted.org/packages/18/b7/70d6a03f2287fcea2dea7cec0108cfaecac936a94e4fa884254c7e3d7b18/datalab_server-0.6.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-20 19:34:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "datalab-org",
"github_project": "datalab",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "datalab-server"
}
JSON
