KM3Pipe
=======
.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/pipeline.svg
:target: https://git.km3net.de/km3py/km3pipe/pipelines
.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/coverage.svg
:target: https://km3py.pages.km3net.de/km3pipe/coverage
.. image:: https://api.codacy.com/project/badge/Grade/9df4849cb9f840289bf883de0dc8e28f
:alt: Codacy Badge
:target: https://app.codacy.com/app/tamasgal/km3pipe?utm_source=github.com&utm_medium=referral&utm_content=tamasgal/km3pipe&utm_campaign=Badge_Grade_Settings
.. image:: https://git.km3net.de/examples/km3badges/-/raw/master/docs-latest-brightgreen.svg
:target: https://km3py.pages.km3net.de/km3pipe
.. image:: https://zenodo.org/badge/24634697.svg
:target: https://doi.org/10.5281/zenodo.808829
KM3Pipe is a framework for KM3NeT related stuff including MC, data files, live
access to detectors and databases, parsers for different file formats and an
easy to use framework for batch processing.
The main Git repository, where issues and merge requests are managed can be
found at https://git.km3net.de/km3py/km3pipe.git
The framework tries to standardise the way the data is processed by providing
a Pipeline-class, which can be used to put together different built-in or user
made Pumps, Sinks and Modules. Pumps act as data readers/parsers (from files,
memory or even socket connections), Sinks are responsible for writing data to
disk and Modules take care of data processing, output and user interaction.
Such a Pipeline setup can then be used to iteratively process data in a file or
from a stream. In our case for example, we store several thousands of neutrino
interaction events in a bunch of files and KM3Pipe is used to stitch together
an analysis chain which processes each event one-by-one by passing them through
a pipeline of modules.
Although it is mainly designed for the KM3NeT neutrino detectors, it can easily
be extended to support any kind of data formats. The core functionality is
written in a general way and is applicable to all kinds of data processing
workflows.
To start off, run::
pip install km3pipe
If you have Docker (https://www.docker.com) installed, you can start using
KM3Pipe immediately by typing::
docker run -it docker.km3net.de/km3pipe
Feel free to get in touch if you’re looking for a small, versatile framework
which provides a quite straightforward module system to make code exchange
between your project members as easily as possible. KM3Pipe already comes with
several types of Pumps, so it should be easy to find an example to implement
your owns. As of version 8.0.0 you find Pumps and Sinks based on popular
formats like HDF5 (https://www.hdfgroup.org), ROOT (https://root.cern.ch) but
also some very specialised project internal binary data formats, which on the
other hand can act as templates for your own ones. Just have a look at the io
subpackage and of course the documentation if you’re interested!
Read the latest docs at https://km3py.pages.km3net.de/km3pipe.
KM3NeT public project homepage http://www.km3net.org
Acknowledgements
----------------
Thanks especially to the gracious help of all contributors:
Tamas Gal, Moritz Lotze, Johannes Schumann, Piotr Kalaczynski, Jonas Reubelt, Michael Moser, Thomas Heid, Alba Domi, Agustin Sanchez Losa, Zineb Aly, Jordan Seneca, Nicole Geisselbrecht, Javier Barrios, Valentin Pestel, Jannik Hofestaedt, Matthias Bissinger, Vladimir Kulikovskiy, Lukas Hennig, Godefroy Vannoye
Raw data
{
"_id": null,
"home_page": "https://git.km3net.de/km3py/km3pipe",
"name": "km3pipe",
"maintainer": "Tamas Gal",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "tgal@km3net.de",
"keywords": "neutrino, astroparticle, physics, HEP, pipeline",
"author": "Tamas Gal and Moritz Lotze",
"author_email": "tgal@km3net.de",
"download_url": "https://files.pythonhosted.org/packages/3d/41/877e5327af52b827c43509dccad85281958532ce8a58eac8384239c3e336/km3pipe-10.0.1.tar.gz",
"platform": null,
"description": "KM3Pipe\n=======\n\n.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/pipeline.svg\n :target: https://git.km3net.de/km3py/km3pipe/pipelines\n\n.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/coverage.svg\n :target: https://km3py.pages.km3net.de/km3pipe/coverage\n\n.. image:: https://api.codacy.com/project/badge/Grade/9df4849cb9f840289bf883de0dc8e28f\n :alt: Codacy Badge\n :target: https://app.codacy.com/app/tamasgal/km3pipe?utm_source=github.com&utm_medium=referral&utm_content=tamasgal/km3pipe&utm_campaign=Badge_Grade_Settings\n\n.. image:: https://git.km3net.de/examples/km3badges/-/raw/master/docs-latest-brightgreen.svg\n :target: https://km3py.pages.km3net.de/km3pipe\n\n.. image:: https://zenodo.org/badge/24634697.svg\n :target: https://doi.org/10.5281/zenodo.808829\n\n\nKM3Pipe is a framework for KM3NeT related stuff including MC, data files, live\naccess to detectors and databases, parsers for different file formats and an\neasy to use framework for batch processing.\n\nThe main Git repository, where issues and merge requests are managed can be\nfound at https://git.km3net.de/km3py/km3pipe.git\n\nThe framework tries to standardise the way the data is processed by providing\na Pipeline-class, which can be used to put together different built-in or user\nmade Pumps, Sinks and Modules. Pumps act as data readers/parsers (from files,\nmemory or even socket connections), Sinks are responsible for writing data to\ndisk and Modules take care of data processing, output and user interaction.\nSuch a Pipeline setup can then be used to iteratively process data in a file or\nfrom a stream. In our case for example, we store several thousands of neutrino\ninteraction events in a bunch of files and KM3Pipe is used to stitch together\nan analysis chain which processes each event one-by-one by passing them through\na pipeline of modules.\n\nAlthough it is mainly designed for the KM3NeT neutrino detectors, it can easily\nbe extended to support any kind of data formats. The core functionality is\nwritten in a general way and is applicable to all kinds of data processing\nworkflows.\n\nTo start off, run::\n\n pip install km3pipe\n\nIf you have Docker (https://www.docker.com) installed, you can start using\nKM3Pipe immediately by typing::\n\n docker run -it docker.km3net.de/km3pipe\n\nFeel free to get in touch if you\u2019re looking for a small, versatile framework\nwhich provides a quite straightforward module system to make code exchange\nbetween your project members as easily as possible. KM3Pipe already comes with\nseveral types of Pumps, so it should be easy to find an example to implement\nyour owns. As of version 8.0.0 you find Pumps and Sinks based on popular\nformats like HDF5 (https://www.hdfgroup.org), ROOT (https://root.cern.ch) but\nalso some very specialised project internal binary data formats, which on the\nother hand can act as templates for your own ones. Just have a look at the io\nsubpackage and of course the documentation if you\u2019re interested!\n\nRead the latest docs at https://km3py.pages.km3net.de/km3pipe.\n\nKM3NeT public project homepage http://www.km3net.org\n\nAcknowledgements\n----------------\n\nThanks especially to the gracious help of all contributors:\n\nTamas Gal, Moritz Lotze, Johannes Schumann, Piotr Kalaczynski, Jonas Reubelt, Michael Moser, Thomas Heid, Alba Domi, Agustin Sanchez Losa, Zineb Aly, Jordan Seneca, Nicole Geisselbrecht, Javier Barrios, Valentin Pestel, Jannik Hofestaedt, Matthias Bissinger, Vladimir Kulikovskiy, Lukas Hennig, Godefroy Vannoye\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "\"An analysis framework for KM3NeT\"",
"version": "10.0.1",
"project_urls": {
"Homepage": "https://git.km3net.de/km3py/km3pipe"
},
"split_keywords": [
"neutrino",
" astroparticle",
" physics",
" hep",
" pipeline"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a000997d4a55dc0bfa3e60c28a98f6272a15960ddf4edc1a0fc7fa2129fe59d9",
"md5": "2bdd7c7dc0f11df12052c6cbfe91e996",
"sha256": "9a19106d80d0f21052d496a65be133429ed797fdd0f7e0fc6f8a0aecdc43fbee"
},
"downloads": -1,
"filename": "km3pipe-10.0.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "2bdd7c7dc0f11df12052c6cbfe91e996",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.10",
"size": 203030,
"upload_time": "2024-10-18T10:57:01",
"upload_time_iso_8601": "2024-10-18T10:57:01.324554Z",
"url": "https://files.pythonhosted.org/packages/a0/00/997d4a55dc0bfa3e60c28a98f6272a15960ddf4edc1a0fc7fa2129fe59d9/km3pipe-10.0.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3d41877e5327af52b827c43509dccad85281958532ce8a58eac8384239c3e336",
"md5": "b21db9b24062ce2fe7360031546d0733",
"sha256": "5fcc1f7a8090dd7c6bb2e870d9ea2eb7c6ae3849fb6cdb9615ff384fb0ba4e68"
},
"downloads": -1,
"filename": "km3pipe-10.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b21db9b24062ce2fe7360031546d0733",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 364378,
"upload_time": "2024-10-18T10:57:03",
"upload_time_iso_8601": "2024-10-18T10:57:03.396556Z",
"url": "https://files.pythonhosted.org/packages/3d/41/877e5327af52b827c43509dccad85281958532ce8a58eac8384239c3e336/km3pipe-10.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-18 10:57:03",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "km3pipe"
}
JSON