<center>
<a href="https://github.com/columnflow/columnflow">
<img src="https://media.githubusercontent.com/media/columnflow/columnflow/master/assets/logo_dark.png" width="480" />
</a>
</center>
<!-- marker-after-logo -->
[](https://github.com/columnflow/columnflow/actions/workflows/lint_and_test.yaml)
[](https://pypi.python.org/pypi/columnflow)
[](http://columnflow.readthedocs.io)
[](https://codecov.io/gh/columnflow/columnflow)
[](https://github.com/columnflow/columnflow/blob/master/LICENSE)
Backend for columnar, fully orchestrated HEP analyses with pure Python, [law](https://github.com/riga/law) and [order](https://github.com/riga/order).
Original source hosted at [GitHub](https://github.com/columnflow/columnflow).
<!-- marker-after-header -->
## Note on current development
This project is currently in a beta phase.
The project setup, suggested workflows, definitions of particular tasks, and the signatures of various helper classes and functions are mostly frozen but could still be subject to changes in the near future.
At this point (December 2022), four large-scale analyses based upon columnflow are being developed, and in the process, help test and verify various aspects of its core.
The first released version is expected in early 2023.
However, if you would like to join early on, contribute or just give it a spin, feel free to get in touch!
## Quickstart
To create an analysis using columnflow, it is recommended to start from a predefined template (located in [analysis_templates](https://github.com/columnflow/columnflow/tree/master/analysis_templates)).
The following command (no previous git clone required) interactively asks for a handful of names and settings, and creates a minimal, yet fully functioning project structure for you!
```shell
bash -c "$(curl -Ls https://raw.githubusercontent.com/columnflow/columnflow/master/create_analysis.sh)"
```
At the end of the setup, you will see further instructions and suggestions to run your first analysis tasks (example below).
```
Setup successfull! The next steps are:
1. Setup the repository and install the environment.
> cd
> source setup.sh [optional_setup_name]
2. Run local tests & linting checks to verify that the analysis is setup correctly.
> ./tests/run_all
3. Create a GRID proxy if you intend to run tasks that need one
> voms-proxy-init -rfc -valid 196:00
4. Checkout the 'Getting started' guide to run your first tasks.
https://columnflow.readthedocs.io/en/stable/start.html
Suggestions for tasks to run:
a) Run the 'calibration -> selection -> reduction' pipeline for the first file of the
default dataset using the default calibrator and default selector
(enter the command below and 'tab-tab' to see all arguments or add --help for help)
> law run cf.ReduceEvents --version dev1 --branch 0
Verify what you just run by adding '--print-status -1' (-1 = fully recursive)
> law run cf.ReduceEvents --version dev1 --branch 0 --print-status -1
b) Create the jet1_pt distribution for the single top datasets
(if you have an image/pdf viewer installed, add it via '--view-cmd <binary>')
> law run cf.PlotVariables1D --version dev1 --datasets 'st*' --variables jet1_pt
Again, verify what you just ran, now with recursion depth 4
> law run cf.PlotVariables1D --version dev1 --datasets 'st*' --variables jet1_pt --print-status 4
c) Include the ttbar dataset and also plot jet1_eta
> law run cf.PlotVariables1D --version dev1 --datasets 'tt*,st*' --variables jet1_pt,jet1_eta
```
For a better overview of the tasks that are triggered by the commands below, checkout the current (yet stylized) [task graph](https://github.com/columnflow/columnflow/wiki#default-task-graph).
## Projects using columnflow
- [hh2bbtautau](https://github.com/uhh-cms/hh2bbtautau): HH → bb𝜏𝜏 analysis with CMS.
- [hh2bbww](https://github.com/uhh-cms/hh2bbww): HH → bbWW analysis with CMS.
- [topmass](https://github.com/uhh-cms/topmass): Top quark mass measurement with CMS.
- [mttbar](https://github.com/uhh-cms/mttbar): Search for heavy resonances in ttbar events with CMS.
- [analysis playground](https://github.com/uhh-cms/analysis_playground): A testing playground for HEP analyses.
## Contributors
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
<tbody>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/riga"><img src="https://avatars.githubusercontent.com/u/1908734?v=4?s=100" width="100px;" alt="Marcel Rieger"/><br /><sub><b>Marcel Rieger</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=riga" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Ariga" title="Reviewed Pull Requests">👀</a> <a href="https://github.com/columnflow/columnflow/commits?author=riga" title="Documentation">📖</a> <a href="https://github.com/columnflow/columnflow/commits?author=riga" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/mafrahm"><img src="https://avatars.githubusercontent.com/u/49306645?v=4?s=100" width="100px;" alt="Mathis Frahm"/><br /><sub><b>Mathis Frahm</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=mafrahm" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Amafrahm" title="Reviewed Pull Requests">👀</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/dsavoiu"><img src="https://avatars.githubusercontent.com/u/17005255?v=4?s=100" width="100px;" alt="Daniel Savoiu"/><br /><sub><b>Daniel Savoiu</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=dsavoiu" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Adsavoiu" title="Reviewed Pull Requests">👀</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/pkausw"><img src="https://avatars.githubusercontent.com/u/26219567?v=4?s=100" width="100px;" alt="pkausw"/><br /><sub><b>pkausw</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=pkausw" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Apkausw" title="Reviewed Pull Requests">👀</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/nprouvost"><img src="https://avatars.githubusercontent.com/u/49162277?v=4?s=100" width="100px;" alt="nprouvost"/><br /><sub><b>nprouvost</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=nprouvost" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/commits?author=nprouvost" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Bogdan-Wiederspan"><img src="https://avatars.githubusercontent.com/u/79155113?v=4?s=100" width="100px;" alt="Bogdan-Wiederspan"/><br /><sub><b>Bogdan-Wiederspan</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=Bogdan-Wiederspan" title="Code">💻</a> <a href="https://github.com/columnflow/columnflow/commits?author=Bogdan-Wiederspan" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/kramerto"><img src="https://avatars.githubusercontent.com/u/18616159?v=4?s=100" width="100px;" alt="Tobias Kramer"/><br /><sub><b>Tobias Kramer</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=kramerto" title="Code">💻</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/mschrode"><img src="https://avatars.githubusercontent.com/u/5065234?v=4?s=100" width="100px;" alt="Matthias Schroeder"/><br /><sub><b>Matthias Schroeder</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=mschrode" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/jolange"><img src="https://avatars.githubusercontent.com/u/6584443?v=4" width="100px;" alt="Johannes Lange"/><br /><sub><b>Johannes Lange</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=jolange" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/BalduinLetzer"><img src="https://avatars.githubusercontent.com/u/70058868?v=4?s=100" width="100px;" alt="BalduinLetzer"/><br /><sub><b>BalduinLetzer</b></sub></a><br /><a href="https://github.com/columnflow/columnflow/commits?author=BalduinLetzer" title="Code">💻</a></td>
</tr>
</tbody>
</table>
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification.
## Development
- Source hosted at [GitHub](https://github.com/columnflow/columnflow)
- Report issues, questions, feature requests on [GitHub Issues](https://github.com/columnflow/columnflow/issues)
<!-- marker-after-body -->
Raw data
{
"_id": null,
"home_page": "https://github.com/columnflow/columnflow",
"name": "columnflow",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7, <4",
"maintainer_email": "",
"keywords": "physics,analysis,experiment,columnar,vectoized,law,order,luigi,lhc,cms",
"author": "The columnflow team",
"author_email": "github.riga@icloud.com",
"download_url": "https://files.pythonhosted.org/packages/23/85/a328c5ca0b898e980f0c9637b40c64842ba61468e2f172bbce046dcf61ca/columnflow-0.2.2.tar.gz",
"platform": null,
"description": "<center>\n <a href=\"https://github.com/columnflow/columnflow\">\n <img src=\"https://media.githubusercontent.com/media/columnflow/columnflow/master/assets/logo_dark.png\" width=\"480\" />\n </a>\n</center>\n\n\n<!-- marker-after-logo -->\n\n\n[](https://github.com/columnflow/columnflow/actions/workflows/lint_and_test.yaml)\n[](https://pypi.python.org/pypi/columnflow)\n[](http://columnflow.readthedocs.io)\n[](https://codecov.io/gh/columnflow/columnflow)\n[](https://github.com/columnflow/columnflow/blob/master/LICENSE)\n\nBackend for columnar, fully orchestrated HEP analyses with pure Python, [law](https://github.com/riga/law) and [order](https://github.com/riga/order).\n\nOriginal source hosted at [GitHub](https://github.com/columnflow/columnflow).\n\n\n<!-- marker-after-header -->\n\n\n## Note on current development\n\nThis project is currently in a beta phase.\nThe project setup, suggested workflows, definitions of particular tasks, and the signatures of various helper classes and functions are mostly frozen but could still be subject to changes in the near future.\nAt this point (December 2022), four large-scale analyses based upon columnflow are being developed, and in the process, help test and verify various aspects of its core.\nThe first released version is expected in early 2023.\nHowever, if you would like to join early on, contribute or just give it a spin, feel free to get in touch!\n\n![Columnflow analytics](https://repobeats.axiom.co/api/embed/b6ebc5ba41019de55eb48e195eecb438890442c8.svg \"Columnflow analytics\")\n\n\n## Quickstart\n\nTo create an analysis using columnflow, it is recommended to start from a predefined template (located in [analysis_templates](https://github.com/columnflow/columnflow/tree/master/analysis_templates)).\nThe following command (no previous git clone required) interactively asks for a handful of names and settings, and creates a minimal, yet fully functioning project structure for you!\n\n```shell\nbash -c \"$(curl -Ls https://raw.githubusercontent.com/columnflow/columnflow/master/create_analysis.sh)\"\n```\n\nAt the end of the setup, you will see further instructions and suggestions to run your first analysis tasks (example below).\n\n```\nSetup successfull! The next steps are:\n\n1. Setup the repository and install the environment.\n > cd\n > source setup.sh [optional_setup_name]\n\n2. Run local tests & linting checks to verify that the analysis is setup correctly.\n > ./tests/run_all\n\n3. Create a GRID proxy if you intend to run tasks that need one\n > voms-proxy-init -rfc -valid 196:00\n\n4. Checkout the 'Getting started' guide to run your first tasks.\n https://columnflow.readthedocs.io/en/stable/start.html\n\n Suggestions for tasks to run:\n\n a) Run the 'calibration -> selection -> reduction' pipeline for the first file of the\n default dataset using the default calibrator and default selector\n (enter the command below and 'tab-tab' to see all arguments or add --help for help)\n > law run cf.ReduceEvents --version dev1 --branch 0\n\n Verify what you just run by adding '--print-status -1' (-1 = fully recursive)\n > law run cf.ReduceEvents --version dev1 --branch 0 --print-status -1\n\n b) Create the jet1_pt distribution for the single top datasets\n (if you have an image/pdf viewer installed, add it via '--view-cmd <binary>')\n > law run cf.PlotVariables1D --version dev1 --datasets 'st*' --variables jet1_pt\n\n Again, verify what you just ran, now with recursion depth 4\n > law run cf.PlotVariables1D --version dev1 --datasets 'st*' --variables jet1_pt --print-status 4\n\n c) Include the ttbar dataset and also plot jet1_eta\n > law run cf.PlotVariables1D --version dev1 --datasets 'tt*,st*' --variables jet1_pt,jet1_eta\n```\n\nFor a better overview of the tasks that are triggered by the commands below, checkout the current (yet stylized) [task graph](https://github.com/columnflow/columnflow/wiki#default-task-graph).\n\n\n## Projects using columnflow\n\n- [hh2bbtautau](https://github.com/uhh-cms/hh2bbtautau): HH \u2192 bb\ud835\udf0f\ud835\udf0f analysis with CMS.\n- [hh2bbww](https://github.com/uhh-cms/hh2bbww): HH \u2192 bbWW analysis with CMS.\n- [topmass](https://github.com/uhh-cms/topmass): Top quark mass measurement with CMS.\n- [mttbar](https://github.com/uhh-cms/mttbar): Search for heavy resonances in ttbar events with CMS.\n- [analysis playground](https://github.com/uhh-cms/analysis_playground): A testing playground for HEP analyses.\n\n\n## Contributors\n\n<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->\n<!-- prettier-ignore-start -->\n<!-- markdownlint-disable -->\n<table>\n <tbody>\n <tr>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/riga\"><img src=\"https://avatars.githubusercontent.com/u/1908734?v=4?s=100\" width=\"100px;\" alt=\"Marcel Rieger\"/><br /><sub><b>Marcel Rieger</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=riga\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Ariga\" title=\"Reviewed Pull Requests\">\ud83d\udc40</a> <a href=\"https://github.com/columnflow/columnflow/commits?author=riga\" title=\"Documentation\">\ud83d\udcd6</a> <a href=\"https://github.com/columnflow/columnflow/commits?author=riga\" title=\"Tests\">\u26a0\ufe0f</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/mafrahm\"><img src=\"https://avatars.githubusercontent.com/u/49306645?v=4?s=100\" width=\"100px;\" alt=\"Mathis Frahm\"/><br /><sub><b>Mathis Frahm</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=mafrahm\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Amafrahm\" title=\"Reviewed Pull Requests\">\ud83d\udc40</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/dsavoiu\"><img src=\"https://avatars.githubusercontent.com/u/17005255?v=4?s=100\" width=\"100px;\" alt=\"Daniel Savoiu\"/><br /><sub><b>Daniel Savoiu</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=dsavoiu\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Adsavoiu\" title=\"Reviewed Pull Requests\">\ud83d\udc40</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/pkausw\"><img src=\"https://avatars.githubusercontent.com/u/26219567?v=4?s=100\" width=\"100px;\" alt=\"pkausw\"/><br /><sub><b>pkausw</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=pkausw\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/pulls?q=is%3Apr+reviewed-by%3Apkausw\" title=\"Reviewed Pull Requests\">\ud83d\udc40</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/nprouvost\"><img src=\"https://avatars.githubusercontent.com/u/49162277?v=4?s=100\" width=\"100px;\" alt=\"nprouvost\"/><br /><sub><b>nprouvost</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=nprouvost\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/commits?author=nprouvost\" title=\"Tests\">\u26a0\ufe0f</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/Bogdan-Wiederspan\"><img src=\"https://avatars.githubusercontent.com/u/79155113?v=4?s=100\" width=\"100px;\" alt=\"Bogdan-Wiederspan\"/><br /><sub><b>Bogdan-Wiederspan</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=Bogdan-Wiederspan\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/columnflow/columnflow/commits?author=Bogdan-Wiederspan\" title=\"Tests\">\u26a0\ufe0f</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/kramerto\"><img src=\"https://avatars.githubusercontent.com/u/18616159?v=4?s=100\" width=\"100px;\" alt=\"Tobias Kramer\"/><br /><sub><b>Tobias Kramer</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=kramerto\" title=\"Code\">\ud83d\udcbb</a></td>\n </tr>\n <tr>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/mschrode\"><img src=\"https://avatars.githubusercontent.com/u/5065234?v=4?s=100\" width=\"100px;\" alt=\"Matthias Schroeder\"/><br /><sub><b>Matthias Schroeder</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=mschrode\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/jolange\"><img src=\"https://avatars.githubusercontent.com/u/6584443?v=4\" width=\"100px;\" alt=\"Johannes Lange\"/><br /><sub><b>Johannes Lange</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=jolange\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/BalduinLetzer\"><img src=\"https://avatars.githubusercontent.com/u/70058868?v=4?s=100\" width=\"100px;\" alt=\"BalduinLetzer\"/><br /><sub><b>BalduinLetzer</b></sub></a><br /><a href=\"https://github.com/columnflow/columnflow/commits?author=BalduinLetzer\" title=\"Code\">\ud83d\udcbb</a></td>\n </tr>\n </tbody>\n</table>\n<!-- markdownlint-restore -->\n<!-- prettier-ignore-end -->\n<!-- ALL-CONTRIBUTORS-LIST:END -->\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification.\n\n\n## Development\n\n- Source hosted at [GitHub](https://github.com/columnflow/columnflow)\n- Report issues, questions, feature requests on [GitHub Issues](https://github.com/columnflow/columnflow/issues)\n\n\n<!-- marker-after-body -->\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "columnflow",
"version": "0.2.2",
"project_urls": {
"Homepage": "https://github.com/columnflow/columnflow"
},
"split_keywords": [
"physics",
"analysis",
"experiment",
"columnar",
"vectoized",
"law",
"order",
"luigi",
"lhc",
"cms"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2385a328c5ca0b898e980f0c9637b40c64842ba61468e2f172bbce046dcf61ca",
"md5": "2d0b7607d90e29158cd17009cb35f287",
"sha256": "c8d66d659c87887523de6ba6507861a664e87833e01d323ada4b16fb8752d03c"
},
"downloads": -1,
"filename": "columnflow-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "2d0b7607d90e29158cd17009cb35f287",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7, <4",
"size": 186842,
"upload_time": "2023-07-21T17:35:56",
"upload_time_iso_8601": "2023-07-21T17:35:56.462649Z",
"url": "https://files.pythonhosted.org/packages/23/85/a328c5ca0b898e980f0c9637b40c64842ba61468e2f172bbce046dcf61ca/columnflow-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-21 17:35:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "columnflow",
"github_project": "columnflow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "columnflow"
}
JSON
