**********
Databroker
**********
|build_status| |coverage| |pypi_version| |license|
Databroker is a data **access** tool built around the `Bluesky Data Model`_.
The data it manages may be from ingested files, captured results of a
Python-based data analysis, or experimental data acquired using the Bluesky Run
Engine.
* Provide a consistent programmatic interface to data, **regardless of storage
details** like file format or storage medium.
* Provide **metadata and data** in a coherent bundle, using standard widely-used
Python and SciPy data structures.
* Support fast, flexible **search** over metadata.
* Enable software tools to operate seamlessly on a
mixture of **live-streaming** data from the Bluesky Run Engine and **saved**
data from Databroker.
Databroker is developed in concert with `Suitcase`_. Suitcase does data
*writing*, and databroker does the reading. Databroker builds on `Intake`_, a
generic data access tool (outside of the Bluesky Project).
============== ==============================================================
PyPI ``pip install databroker``
Conda ``pip install -c nsls2forge databroker``
Source code https://github.com/bluesky/databroker
Documentation https://blueskyproject.io/databroker
============== ==============================================================
The bundle of metadata and data looks like this, for example.
.. code:: python
>>> run
BlueskyRun
uid='4a794c63-8223-4893-895e-d16e763188a8'
exit_status='success'
2020-03-07 09:17:40.436 -- 2020-03-07 09:28:53.173
Streams:
* primary
* baseline
Additional user metadata beyond what is shown is stored in ``run.metadata``.
The bundle contains some number of logical tables of data ("streams"). They can
be accessed by name and read into a standard data structure from `xarray`_.
.. code:: python
>>> run.primary.read()
<xarray.Dataset>
Dimensions: (time: 411)
Coordinates:
* time (time) float64 1.584e+09 1.584e+09 ... 1.584e+09
Data variables:
I0 (time) float64 13.07 13.01 12.95 ... 9.862 9.845
It (time) float64 11.52 11.47 11.44 ... 4.971 4.968
Ir (time) float64 10.96 10.92 10.88 ... 4.761 4.763
dwti_dwell_time (time) float64 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0
dwti_dwell_time_setpoint (time) float64 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0
dcm_energy (time) float64 1.697e+04 1.698e+04 ... 1.791e+04
dcm_energy_setpoint (time) float64 1.697e+04 1.698e+04 ... 1.791e+04
Common search queries can be done with a high-level Python interface.
.. code:: python
>>> from databroker.queries import TimeRange
>>> catalog.search(TimeRange(since="2020"))
Custom queries can be done with the `MongoDB query language`_.
.. code:: python
>>> query = {
... "motors": {"$in": ["x", "y"]}, # scanning either x or y
... "temperature" {"$lt": 300}, # temperature less than 300
... "sample.element": "Ni",
... }
>>> catalog.search(query)
See the tutorials for more.
.. |build_status| image:: https://github.com/bluesky/databroker/workflows/Unit%20Tests/badge.svg?branch=master
:target: https://github.com/bluesky/databroker/actions?query=workflow%3A%22Unit+Tests%22
:alt: Build Status
.. |coverage| image:: https://codecov.io/gh/bluesky/databroker/branch/master/graph/badge.svg
:target: https://codecov.io/gh/bluesky/databroker
:alt: Test Coverage
.. |pypi_version| image:: https://img.shields.io/pypi/v/databroker.svg
:target: https://pypi.org/project/databroker
:alt: Latest PyPI version
.. |license| image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg
:target: https://opensource.org/licenses/BSD-3-Clause
:alt: BSD 3-Clause License
.. _xarray: https://xarray.pydata.org/
.. _MongoDB query language: https://docs.mongodb.com/manual/reference/operator/query/
.. _Bluesky Data Model: https://blueskyproject.io/event-model/data-model.html
.. _Suitcase: https://blueskyproject.io/suitcase/
.. _Intake: https://intake.readthedocs.io/en/latest/
Raw data
{
"_id": null,
"home_page": "https://github.com/NSLS-II/databroker",
"name": "databroker",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Brookhaven National Laboratory",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/1b/6e/d2eab495fe5c9c38af245e28283dbb40208723971b28e06f53cbe9f0d7ac/databroker-1.2.5.tar.gz",
"platform": "",
"description": "**********\nDatabroker\n**********\n\n|build_status| |coverage| |pypi_version| |license|\n\nDatabroker is a data **access** tool built around the `Bluesky Data Model`_.\nThe data it manages may be from ingested files, captured results of a\nPython-based data analysis, or experimental data acquired using the Bluesky Run\nEngine.\n\n* Provide a consistent programmatic interface to data, **regardless of storage\n details** like file format or storage medium.\n* Provide **metadata and data** in a coherent bundle, using standard widely-used\n Python and SciPy data structures.\n* Support fast, flexible **search** over metadata.\n* Enable software tools to operate seamlessly on a\n mixture of **live-streaming** data from the Bluesky Run Engine and **saved**\n data from Databroker.\n\nDatabroker is developed in concert with `Suitcase`_. Suitcase does data\n*writing*, and databroker does the reading. Databroker builds on `Intake`_, a\ngeneric data access tool (outside of the Bluesky Project).\n\n============== ==============================================================\nPyPI ``pip install databroker``\nConda ``pip install -c nsls2forge databroker``\nSource code https://github.com/bluesky/databroker\nDocumentation https://blueskyproject.io/databroker\n============== ==============================================================\n\nThe bundle of metadata and data looks like this, for example.\n\n.. code:: python\n\n >>> run\n BlueskyRun\n uid='4a794c63-8223-4893-895e-d16e763188a8'\n exit_status='success'\n 2020-03-07 09:17:40.436 -- 2020-03-07 09:28:53.173\n Streams:\n * primary\n * baseline\n\nAdditional user metadata beyond what is shown is stored in ``run.metadata``.\nThe bundle contains some number of logical tables of data (\"streams\"). They can\nbe accessed by name and read into a standard data structure from `xarray`_.\n \n.. code:: python\n\n >>> run.primary.read()\n <xarray.Dataset>\n Dimensions: (time: 411)\n Coordinates:\n * time (time) float64 1.584e+09 1.584e+09 ... 1.584e+09\n Data variables:\n I0 (time) float64 13.07 13.01 12.95 ... 9.862 9.845\n It (time) float64 11.52 11.47 11.44 ... 4.971 4.968\n Ir (time) float64 10.96 10.92 10.88 ... 4.761 4.763\n dwti_dwell_time (time) float64 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0\n dwti_dwell_time_setpoint (time) float64 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0\n dcm_energy (time) float64 1.697e+04 1.698e+04 ... 1.791e+04\n dcm_energy_setpoint (time) float64 1.697e+04 1.698e+04 ... 1.791e+04\n\nCommon search queries can be done with a high-level Python interface.\n\n.. code:: python\n\n >>> from databroker.queries import TimeRange\n >>> catalog.search(TimeRange(since=\"2020\"))\n\nCustom queries can be done with the `MongoDB query language`_.\n\n.. code:: python\n\n >>> query = {\n ... \"motors\": {\"$in\": [\"x\", \"y\"]}, # scanning either x or y\n ... \"temperature\" {\"$lt\": 300}, # temperature less than 300\n ... \"sample.element\": \"Ni\",\n ... }\n >>> catalog.search(query)\n\nSee the tutorials for more.\n\n.. |build_status| image:: https://github.com/bluesky/databroker/workflows/Unit%20Tests/badge.svg?branch=master\n :target: https://github.com/bluesky/databroker/actions?query=workflow%3A%22Unit+Tests%22\n :alt: Build Status\n\n.. |coverage| image:: https://codecov.io/gh/bluesky/databroker/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/bluesky/databroker\n :alt: Test Coverage\n\n.. |pypi_version| image:: https://img.shields.io/pypi/v/databroker.svg\n :target: https://pypi.org/project/databroker\n :alt: Latest PyPI version\n\n.. |license| image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg\n :target: https://opensource.org/licenses/BSD-3-Clause\n :alt: BSD 3-Clause License\n\n.. _xarray: https://xarray.pydata.org/\n\n.. _MongoDB query language: https://docs.mongodb.com/manual/reference/operator/query/\n\n.. _Bluesky Data Model: https://blueskyproject.io/event-model/data-model.html\n\n.. _Suitcase: https://blueskyproject.io/suitcase/\n\n.. _Intake: https://intake.readthedocs.io/en/latest/\n\n\n",
"bugtrack_url": null,
"license": "BSD (3-clause)",
"summary": "Unification of NSLS-II data sources",
"version": "1.2.5",
"project_urls": {
"Homepage": "https://github.com/NSLS-II/databroker"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "19f7a9f95700cf6d59e77d0b8f3191556df0baee8b4a2a76e444e4137a0b801e",
"md5": "5dd7eb4bfdd26377897168b0307dff50",
"sha256": "236f33cade38a98de59baac8a6eaa44e626574b56e2c3c8ee7a0fbbd1ffbed48"
},
"downloads": -1,
"filename": "databroker-1.2.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "5dd7eb4bfdd26377897168b0307dff50",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6",
"size": 232694,
"upload_time": "2022-01-21T19:59:29",
"upload_time_iso_8601": "2022-01-21T19:59:29.182984Z",
"url": "https://files.pythonhosted.org/packages/19/f7/a9f95700cf6d59e77d0b8f3191556df0baee8b4a2a76e444e4137a0b801e/databroker-1.2.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1b6ed2eab495fe5c9c38af245e28283dbb40208723971b28e06f53cbe9f0d7ac",
"md5": "8b3d614f60f4f62690954590aa07206e",
"sha256": "9950db880c3bf7350b7f0db03c02a83eec1afb8529247ffbcf098dfa078412e0"
},
"downloads": -1,
"filename": "databroker-1.2.5.tar.gz",
"has_sig": false,
"md5_digest": "8b3d614f60f4f62690954590aa07206e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 200090,
"upload_time": "2022-01-21T19:59:30",
"upload_time_iso_8601": "2022-01-21T19:59:30.478389Z",
"url": "https://files.pythonhosted.org/packages/1b/6e/d2eab495fe5c9c38af245e28283dbb40208723971b28e06f53cbe9f0d7ac/databroker-1.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-01-21 19:59:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NSLS-II",
"github_project": "databroker",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "databroker"
}
JSON
