| Name | law JSON |
| Version |
0.1.18
JSON |
| download |
| home_page | https://github.com/riga/law |
| Summary | Build large-scale task workflows using luigi, remote job submission, remote targets, and environment |
| upload_time | 2024-02-11 10:31:57 |
| maintainer | |
| docs_url | None |
| author | Marcel Rieger |
| requires_python | >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4 |
| license | BSD-3-Clause |
| keywords |
luigi
workflow
pipeline
remote
gfal
submission
cluster
grid
condor
lsf
glite
arc
sandboxing
docker
singularity
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<!-- marker-before-logo -->
<p align="center">
<a href="https://github.com/riga/law">
<img src="https://media.githubusercontent.com/media/riga/law/master/assets/logo.png" />
</a>
</p>
<!-- marker-after-logo -->
<!-- marker-before-badges -->
<p align="center">
<a href="https://github.com/riga/law/actions?query=workflow%3A%22Lint+and+test%22">
<img alt="Build status" src="https://github.com/riga/law/workflows/Lint%20and%20test/badge.svg" />
</a>
<a href="https://github.com/riga/law/actions?query=workflow%3A%22Deploy+images%22">
<img alt="Docker images" src="https://github.com/riga/law/workflows/Deploy%20images/badge.svg" />
</a>
<a href="http://law.readthedocs.io/en/latest">
<img alt="Documentation status" src="https://readthedocs.org/projects/law/badge/?version=latest" />
</a>
<a href="https://pypi.python.org/pypi/law">
<img alt="Package version" src="https://img.shields.io/pypi/v/law.svg?style=flat" />
</a>
<a href="https://github.com/riga/law/blob/master/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/riga/law.svg" />
</a>
<a href="https://zenodo.org/badge/latestdoi/75482295">
<img alt="DOI" src="https://zenodo.org/badge/75482295.svg" />
</a>
</p>
<!-- marker-after-badges -->
<!-- marker-before-header -->
> [!NOTE]
> This project is currently under development.
> - Version 1.0.0 will support Python ≥3.7 and is developed in the [release_prep branch](https://github.com/riga/law/tree/release_prep).
> - There will be a legacy branch with Python 2.7 and ≤3.6 support that, however, won't receive any new features.
> - The release is targetted for spring 2024.
Use law to build complex and large-scale task workflows.
It is build on top of [luigi](https://github.com/spotify/luigi) and adds abstractions for **run locations**, **storage locations** and **software environments**.
Law strictly disentangles these building blocks and ensures they remain interchangeable and resource-opportunistic.
Key features:
- CLI with auto-completion and interactive status and dependency inspection.
- Remote targets with automatic retries and local caching
- WebDAV, HTTP, Dropbox, SFTP, all WLCG protocols (srm, xrootd, dcap, gsiftp, webdav, ...)
- Automatic submission to batch systems from within tasks
- HTCondor, LSF, gLite, ARC, Slurm, CMS-CRAB
- Environment sandboxing, configurable on task level
- Docker, Singularity, Sub-Shells, Virutal envs
<!-- marker-after-header -->
## Contents
<!-- marker-before-contents -->
- [First steps](#first-steps)
- [Installation and dependencies](#installation-and-dependencies)
- [Usage at CERN](#usage-at-cern)
- [Overcomplete example config](#overcomplete-example-config)
- [Projects using law](#projects-using-law)
- [Examples](#examples)
- [Further topics](#further-topics)
- [Auto completion on the command-line](#auto-completion-on-the-command-line)
- [Development](#development)
- [Tests](#tests)
- [Contributors](#contributors)
<!-- marker-after-contents -->
<!-- marker-before-body -->
## First steps
### Installation and dependencies
Install via [pip](https://pypi.python.org/pypi/law)
```shell
pip install law
```
or [conda / (micro)mamba](https://anaconda.org/conda-forge/law)
```shel
conda install -c conda-forge law
```
If you plan to use remote targets, the (default) implementation also requires [gfal2](https://dmc-docs.web.cern.ch/dmc-docs/gfal2/gfal2.html) and [gfal2-python](https://pypi.python.org/pypi/gfal2-python) (optional) to be installed, either via pip or conda / (micro)mamba.
```shell
conda install -c conda-forge gfal2 gfal2-util
```
### Usage at CERN
See the [wiki](https://github.com/riga/law/wiki/Usage-at-CERN).
### Overcomplete example config
See [law.cfg.example](https://github.com/riga/law/tree/master/law.cfg.example).
## Projects using law
- CMS Di-Higgs Inference Tools:
- Basis for statistical analysis for all Di-Higgs searches in CMS, starting at datacard-level
- [repo](https://gitlab.cern.ch/hh/tools/inference), [docs](https://cms-hh.web.cern.ch/cms-hh/tools/inference/index.html)
- columnflow ([+ all analyses using it](https://github.com/columnflow/columnflow#projects-using-columnflow)):
- Python based, fully automated, columnar framework, including job submission, resolution of systematics and ML pipelines, starting at NanoAOD-level with an optimized multi-threaded column reader
- [repo](https://github.com/columnflow/columnflow), [docs](http://columnflow.readthedocs.io), [task structure](https://github.com/columnflow/columnflow/wiki#default-task-graph)
- CMS B-Tag SF Measurement:
- Automated workflow for deriving shape-calibrating b-tag scale factors, starting at MiniAOD-level
- [repo](https://github.com/cms-btv-pog/jet-tagging-sf)
- CMS Tau POG ML Tools:
- Preprocessing pipeline for ML trainings in the TAU group
- [repo](https://github.com/cms-tau-pog/TauMLTools)
- CMS HLT Config Parser:
- Collects information from various databases (HLT, bril, etc.) and shows menus, triggers paths, filter names for configurable MC datasets or data runs
- [repo](https://github.com/riga/cms-hlt-parser)
- RWTH-CMS Analysis Framework:
- Basis for multiple CMS analyses ranging from Di-Higgs, to single Higgs and b-tag SF measurements, starting at NanoAOD-level and based on coffea processors
- [repo](https://git.rwth-aachen.de/3pia/cms_analyses/common/-/tree/master/)
- CIEMAT-CMS Analysis Framework:
- Python and RDataFrame based framework starting from NanoAOD and targetting multiple CMS analyses
- [repo](https://gitlab.cern.ch/cms-phys-ciemat/nanoaod_base_analysis/)
- CMS 3D Z+jet 13TeV analysis
- Analysis workflow management from NTuple production to final plots and fits
- [repo](https://gitlab.etp.kit.edu/cverstege/zjet-analysis)
- NP-correction derivation tool
- MC generation with Herwig and analysis of generated events with Rivet
- [repo](https://github.com/HerrHorizontal/herwig-run)
- CMS SUSY Searches at DESY
- Analysis framework for CMS SUSY searches going from custom NanoAODs -> NTuple production -> DNN-based inference -> final plots and fits
- [repo](https://github.com/frengelk/Susy1LeptonAnalysis)
- Kingmaker (CMS Ntuple Production with CROWN)
- Ntuple conversion from CMS nanoAOD to analysis Ntuples using the CROWN framework. Also includes the training of an event classifier on those ntuples.
- [repo](https://github.com/KIT-CMS/KingMaker), [CROWN](https://github.com/KIT-CMS/CROWN)
If your project uses law but is not yet listed here, feel free to open a pull request or mention your project details in a new [issue](https://github.com/riga/law/issues/new?assignees=riga&labels=docs&template=register-project.md) and it will be added.
## Examples
All examples can be run either in a Jupyter notebook or a dedicated docker container.
For the latter, do
```shell
docker run -ti riga/law:example <example_name>
```
- [loremipsum](https://github.com/riga/law/tree/master/examples/loremipsum): The *hello world* example of law.
- [workflows](https://github.com/riga/law/tree/master/examples/workflows): Law workflows.
- [workflow_parameters](https://github.com/riga/law/tree/master/examples/workflows): Alternative way of parametrizing workflows with explicit branch parameters.
- [notebooks](https://github.com/riga/law/tree/master/examples/notebooks): Examples showing how to use and work with law in notebooks.
- [dropbox_targets](https://github.com/riga/law/tree/master/examples/dropbox_targets): Working with targets that are stored on Dropbox.
- [wlcg_targets](https://github.com/riga/law/tree/master/examples/wlcg_targets): Working with targets that are stored on WLCG storage elements (dCache, EOS, ...). TODO.
- [htcondor_at_vispa](https://github.com/riga/law/tree/master/examples/htcondor_at_vispa): HTCondor workflows at the [VISPA service](https://vispa.physik.rwth-aachen.de).
- [htcondor_at_cern](https://github.com/riga/law/tree/master/examples/htcondor_at_cern): HTCondor workflows at the CERN batch infrastructure.
- [CMS Crab at CERN](https://github.com/riga/law_example_CMSCrabWorkflows): CMS Crab workflows executed from lxplus at CERN.
- [sequential_htcondor_at_cern](https://github.com/riga/law/tree/master/examples/sequential_htcondor_at_cern): Continuation of the [htcondor_at_cern](https://github.com/riga/law/tree/master/examples/htcondor_at_cern) example, showing sequential jobs that eagerly start once jobs running previous requirements succeeded.
- [htcondor_at_naf](https://github.com/riga/law/tree/master/examples/htcondor_at_naf): HTCondor workflows at German [National Analysis Facility (NAF)](https://confluence.desy.de/display/IS/NAF+-+National+Analysis+Facility).
- [slurm_at_maxwell](https://github.com/riga/law/tree/master/examples/slurm_at_maxwell): Slurm workflows at the [Desy Maxwell cluster](https://confluence.desy.de/display/MXW/Maxwell+Cluster).
- [grid_at_cern](https://github.com/riga/law_example_WLCG): Workflows that run jobs and store data on the WLCG.
- [lsf_at_cern](https://github.com/riga/law/tree/master/examples/lsf_at_cern): LSF workflows at the CERN batch infrastructure.
- [docker_sandboxes](https://github.com/riga/law/tree/master/examples/docker_sandboxes): Environment sandboxing using Docker. TODO.
- [singularity_sandboxes](https://github.com/riga/law/tree/master/examples/singularity_sandboxes): Environment sandboxing using Singularity. TODO.
- [subshell_sandboxes](https://github.com/riga/law/tree/master/examples/subshell_sandboxes): Environment sandboxing using Subshells. TODO.
- [parallel_optimization](https://github.com/riga/law/tree/master/examples/parallel_optimization): Parallel optimization using [scikit optimize](https://scikit-optimize.github.io).
- [notifications](https://github.com/riga/law/tree/master/examples/notifications): Demonstration of slack and telegram task status notifications..
- [CMS Single Top Analysis](https://github.com/riga/law_example_CMSSingleTopAnalysis): Simple physics analysis using law.
## Further topics
### Auto completion on the command-line
**bash**
```shell
source "$( law completion )"
```
**zsh**
zsh is able to load and evaluate bash completion scripts via `bashcompinit`.
In order for `bashcompinit` to work, you should run `compinstall` to enable completion scripts:
```shell
autoload -Uz compinstall && compinstall
```
After following the instructions, these lines should be present in your `~/.zshrc`:
```shell
# The following lines were added by compinstall
zstyle :compinstall filename '~/.zshrc'
autoload -Uz +X compinit && compinit
autoload -Uz +X bashcompinit && bashcompinit
# End of lines added by compinstall
```
If this is the case, just source the law completion script (which internally enables `bashcompinit`) and you're good to go:
```shell
source "$( law completion )"
```
## Development
- Source hosted at [GitHub](https://github.com/riga/law)
- Report issues, questions, feature requests on [GitHub Issues](https://github.com/riga/law/issues)
### Tests
To run and test law, there are various docker `riga/law` images available on the [DockerHub](https://cloud.docker.com/u/riga/repository/docker/riga/law), corresponding to different OS and Python versions (based on [micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)).
Start them via
```shell
docker run -ti riga/law:<the_tag>
```
| OS | Python | Tags |
| ----------- | ------ | ----------------------------------------- |
| AlmaLinux 9 | 3.11 | a9-py311, a9-py3, a9, py311, py3, latest |
| AlmaLinux 9 | 3.10 | a9-py310, py310 |
| AlmaLinux 9 | 3.9 | a9-py39, py39 |
| AlmaLinux 9 | 3.8 | a9-py38, py38 |
| AlmaLinux 9 | 3.7 | a9-py37, py37 |
| CentOS 8 | 3.11 | c8-py311, c8-py3, c8 |
| CentOS 8 | 3.10 | c8-py310 |
| CentOS 8 | 3.9 | c8-py39 |
| CentOS 8 | 3.8 | c8-py38 |
| CentOS 8 | 3.7 | c8-py37 |
| CentOS 7 | 3.10 | c7-py310, c7-py3, c7 |
| CentOS 7 | 3.9 | c7-py39 |
| CentOS 7 | 3.8 | c7-py38 |
| CentOS 7 | 3.7 | c7-py37 |
| CentOS 7 | 3.6 | c7-py36, py36 (removed soon) |
| CentOS 7 | 2.7 | c7-py27, c7-py2, py27, py2 (removed soon) |
### Contributors
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
<tbody>
<tr>
<td align="center" valign="top" width="14.28%"><a href="http://github.com/riga"><img src="https://avatars.githubusercontent.com/u/1908734?v=4?s=100" width="100px;" alt="Marcel Rieger"/><br /><sub><b>Marcel Rieger</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=riga" title="Code">💻</a> <a href="https://github.com/riga/law/pulls?q=is%3Apr+reviewed-by%3Ariga" title="Reviewed Pull Requests">👀</a> <a href="#maintenance-riga" title="Maintenance">🚧</a> <a href="https://github.com/riga/law/commits?author=riga" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/pfackeldey"><img src="https://avatars.githubusercontent.com/u/18463582?v=4?s=100" width="100px;" alt="Peter Fackeldey"/><br /><sub><b>Peter Fackeldey</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=pfackeldey" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yrath"><img src="https://avatars.githubusercontent.com/u/20044510?v=4?s=100" width="100px;" alt="Yannik Rath"/><br /><sub><b>Yannik Rath</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=yrath" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/jaimeleonh"><img src="https://avatars.githubusercontent.com/u/47629805?v=4?s=100" width="100px;" alt="Jaime Leon Holgado"/><br /><sub><b>Jaime Leon Holgado</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=jaimeleonh" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lmoureaux"><img src="https://avatars.githubusercontent.com/u/22327575?v=4?s=100" width="100px;" alt="Louis Moureaux"/><br /><sub><b>Louis Moureaux</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=lmoureaux" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="http://www.linkedin.com/in/lgeiger"><img src="https://avatars.githubusercontent.com/u/13285808?v=4?s=100" width="100px;" alt="Lukas Geiger"/><br /><sub><b>Lukas Geiger</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=lgeiger" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://too.gy"><img src="https://avatars.githubusercontent.com/u/694873?v=4?s=100" width="100px;" alt="Valentin Iovene"/><br /><sub><b>Valentin Iovene</b></sub></a><br /><a href="https://github.com/riga/law/commits?author=tgy" title="Code">💻</a></td>
</tr>
</tbody>
</table>
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification.
<!-- marker-after-body -->
Raw data
{
"_id": null,
"home_page": "https://github.com/riga/law",
"name": "law",
"maintainer": "",
"docs_url": null,
"requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
"maintainer_email": "",
"keywords": "luigi workflow pipeline remote gfal submission cluster grid condor lsf glite arc sandboxing docker singularity",
"author": "Marcel Rieger",
"author_email": "github.riga@icloud.com",
"download_url": "https://files.pythonhosted.org/packages/2d/9a/28b395a403afe033891dab41d74df54d8103b593e6a0996224332dba2f6e/law-0.1.18.tar.gz",
"platform": null,
"description": "<!-- marker-before-logo -->\n\n<p align=\"center\">\n <a href=\"https://github.com/riga/law\">\n <img src=\"https://media.githubusercontent.com/media/riga/law/master/assets/logo.png\" />\n </a>\n</p>\n\n<!-- marker-after-logo -->\n\n<!-- marker-before-badges -->\n\n<p align=\"center\">\n <a href=\"https://github.com/riga/law/actions?query=workflow%3A%22Lint+and+test%22\">\n <img alt=\"Build status\" src=\"https://github.com/riga/law/workflows/Lint%20and%20test/badge.svg\" />\n </a>\n <a href=\"https://github.com/riga/law/actions?query=workflow%3A%22Deploy+images%22\">\n <img alt=\"Docker images\" src=\"https://github.com/riga/law/workflows/Deploy%20images/badge.svg\" />\n </a>\n <a href=\"http://law.readthedocs.io/en/latest\">\n <img alt=\"Documentation status\" src=\"https://readthedocs.org/projects/law/badge/?version=latest\" />\n </a>\n <a href=\"https://pypi.python.org/pypi/law\">\n <img alt=\"Package version\" src=\"https://img.shields.io/pypi/v/law.svg?style=flat\" />\n </a>\n <a href=\"https://github.com/riga/law/blob/master/LICENSE\">\n <img alt=\"License\" src=\"https://img.shields.io/github/license/riga/law.svg\" />\n </a>\n <a href=\"https://zenodo.org/badge/latestdoi/75482295\">\n <img alt=\"DOI\" src=\"https://zenodo.org/badge/75482295.svg\" />\n </a>\n</p>\n\n<!-- marker-after-badges -->\n\n<!-- marker-before-header -->\n\n> [!NOTE] \n> This project is currently under development.\n> - Version 1.0.0 will support Python \u22653.7 and is developed in the [release_prep branch](https://github.com/riga/law/tree/release_prep).\n> - There will be a legacy branch with Python 2.7 and \u22643.6 support that, however, won't receive any new features.\n> - The release is targetted for spring 2024.\n\nUse law to build complex and large-scale task workflows.\nIt is build on top of [luigi](https://github.com/spotify/luigi) and adds abstractions for **run locations**, **storage locations** and **software environments**.\nLaw strictly disentangles these building blocks and ensures they remain interchangeable and resource-opportunistic.\n\nKey features:\n\n- CLI with auto-completion and interactive status and dependency inspection.\n- Remote targets with automatic retries and local caching\n - WebDAV, HTTP, Dropbox, SFTP, all WLCG protocols (srm, xrootd, dcap, gsiftp, webdav, ...)\n- Automatic submission to batch systems from within tasks\n - HTCondor, LSF, gLite, ARC, Slurm, CMS-CRAB\n- Environment sandboxing, configurable on task level\n - Docker, Singularity, Sub-Shells, Virutal envs\n\n<!-- marker-after-header -->\n\n\n## Contents\n\n<!-- marker-before-contents -->\n\n- [First steps](#first-steps)\n - [Installation and dependencies](#installation-and-dependencies)\n - [Usage at CERN](#usage-at-cern)\n - [Overcomplete example config](#overcomplete-example-config)\n- [Projects using law](#projects-using-law)\n- [Examples](#examples)\n- [Further topics](#further-topics)\n - [Auto completion on the command-line](#auto-completion-on-the-command-line)\n- [Development](#development)\n - [Tests](#tests)\n - [Contributors](#contributors)\n\n<!-- marker-after-contents -->\n\n\n<!-- marker-before-body -->\n\n## First steps\n\n### Installation and dependencies\n\nInstall via [pip](https://pypi.python.org/pypi/law)\n\n```shell\npip install law\n```\n\nor [conda / (micro)mamba](https://anaconda.org/conda-forge/law)\n\n```shel\nconda install -c conda-forge law\n```\n\nIf you plan to use remote targets, the (default) implementation also requires [gfal2](https://dmc-docs.web.cern.ch/dmc-docs/gfal2/gfal2.html) and [gfal2-python](https://pypi.python.org/pypi/gfal2-python) (optional) to be installed, either via pip or conda / (micro)mamba.\n\n```shell\nconda install -c conda-forge gfal2 gfal2-util\n```\n\n\n### Usage at CERN\n\nSee the [wiki](https://github.com/riga/law/wiki/Usage-at-CERN).\n\n\n### Overcomplete example config\n\nSee [law.cfg.example](https://github.com/riga/law/tree/master/law.cfg.example).\n\n\n## Projects using law\n\n- CMS Di-Higgs Inference Tools:\n - Basis for statistical analysis for all Di-Higgs searches in CMS, starting at datacard-level\n - [repo](https://gitlab.cern.ch/hh/tools/inference), [docs](https://cms-hh.web.cern.ch/cms-hh/tools/inference/index.html)\n- columnflow ([+ all analyses using it](https://github.com/columnflow/columnflow#projects-using-columnflow)):\n - Python based, fully automated, columnar framework, including job submission, resolution of systematics and ML pipelines, starting at NanoAOD-level with an optimized multi-threaded column reader\n - [repo](https://github.com/columnflow/columnflow), [docs](http://columnflow.readthedocs.io), [task structure](https://github.com/columnflow/columnflow/wiki#default-task-graph)\n- CMS B-Tag SF Measurement:\n - Automated workflow for deriving shape-calibrating b-tag scale factors, starting at MiniAOD-level\n - [repo](https://github.com/cms-btv-pog/jet-tagging-sf)\n- CMS Tau POG ML Tools:\n - Preprocessing pipeline for ML trainings in the TAU group\n - [repo](https://github.com/cms-tau-pog/TauMLTools)\n- CMS HLT Config Parser:\n - Collects information from various databases (HLT, bril, etc.) and shows menus, triggers paths, filter names for configurable MC datasets or data runs\n - [repo](https://github.com/riga/cms-hlt-parser)\n- RWTH-CMS Analysis Framework:\n - Basis for multiple CMS analyses ranging from Di-Higgs, to single Higgs and b-tag SF measurements, starting at NanoAOD-level and based on coffea processors\n - [repo](https://git.rwth-aachen.de/3pia/cms_analyses/common/-/tree/master/)\n- CIEMAT-CMS Analysis Framework:\n - Python and RDataFrame based framework starting from NanoAOD and targetting multiple CMS analyses\n - [repo](https://gitlab.cern.ch/cms-phys-ciemat/nanoaod_base_analysis/)\n- CMS 3D Z+jet 13TeV analysis\n - Analysis workflow management from NTuple production to final plots and fits\n - [repo](https://gitlab.etp.kit.edu/cverstege/zjet-analysis)\n- NP-correction derivation tool\n - MC generation with Herwig and analysis of generated events with Rivet\n - [repo](https://github.com/HerrHorizontal/herwig-run)\n- CMS SUSY Searches at DESY\n - Analysis framework for CMS SUSY searches going from custom NanoAODs -> NTuple production -> DNN-based inference -> final plots and fits\n - [repo](https://github.com/frengelk/Susy1LeptonAnalysis)\n- Kingmaker (CMS Ntuple Production with CROWN)\n - Ntuple conversion from CMS nanoAOD to analysis Ntuples using the CROWN framework. Also includes the training of an event classifier on those ntuples.\n - [repo](https://github.com/KIT-CMS/KingMaker), [CROWN](https://github.com/KIT-CMS/CROWN)\n\nIf your project uses law but is not yet listed here, feel free to open a pull request or mention your project details in a new [issue](https://github.com/riga/law/issues/new?assignees=riga&labels=docs&template=register-project.md) and it will be added.\n\n\n## Examples\n\nAll examples can be run either in a Jupyter notebook or a dedicated docker container.\nFor the latter, do\n\n```shell\ndocker run -ti riga/law:example <example_name>\n```\n\n- [loremipsum](https://github.com/riga/law/tree/master/examples/loremipsum): The *hello world* example of law.\n- [workflows](https://github.com/riga/law/tree/master/examples/workflows): Law workflows.\n- [workflow_parameters](https://github.com/riga/law/tree/master/examples/workflows): Alternative way of parametrizing workflows with explicit branch parameters.\n- [notebooks](https://github.com/riga/law/tree/master/examples/notebooks): Examples showing how to use and work with law in notebooks.\n- [dropbox_targets](https://github.com/riga/law/tree/master/examples/dropbox_targets): Working with targets that are stored on Dropbox.\n- [wlcg_targets](https://github.com/riga/law/tree/master/examples/wlcg_targets): Working with targets that are stored on WLCG storage elements (dCache, EOS, ...). TODO.\n- [htcondor_at_vispa](https://github.com/riga/law/tree/master/examples/htcondor_at_vispa): HTCondor workflows at the [VISPA service](https://vispa.physik.rwth-aachen.de).\n- [htcondor_at_cern](https://github.com/riga/law/tree/master/examples/htcondor_at_cern): HTCondor workflows at the CERN batch infrastructure.\n- [CMS Crab at CERN](https://github.com/riga/law_example_CMSCrabWorkflows): CMS Crab workflows executed from lxplus at CERN.\n- [sequential_htcondor_at_cern](https://github.com/riga/law/tree/master/examples/sequential_htcondor_at_cern): Continuation of the [htcondor_at_cern](https://github.com/riga/law/tree/master/examples/htcondor_at_cern) example, showing sequential jobs that eagerly start once jobs running previous requirements succeeded.\n- [htcondor_at_naf](https://github.com/riga/law/tree/master/examples/htcondor_at_naf): HTCondor workflows at German [National Analysis Facility (NAF)](https://confluence.desy.de/display/IS/NAF+-+National+Analysis+Facility).\n- [slurm_at_maxwell](https://github.com/riga/law/tree/master/examples/slurm_at_maxwell): Slurm workflows at the [Desy Maxwell cluster](https://confluence.desy.de/display/MXW/Maxwell+Cluster).\n- [grid_at_cern](https://github.com/riga/law_example_WLCG): Workflows that run jobs and store data on the WLCG.\n- [lsf_at_cern](https://github.com/riga/law/tree/master/examples/lsf_at_cern): LSF workflows at the CERN batch infrastructure.\n- [docker_sandboxes](https://github.com/riga/law/tree/master/examples/docker_sandboxes): Environment sandboxing using Docker. TODO.\n- [singularity_sandboxes](https://github.com/riga/law/tree/master/examples/singularity_sandboxes): Environment sandboxing using Singularity. TODO.\n- [subshell_sandboxes](https://github.com/riga/law/tree/master/examples/subshell_sandboxes): Environment sandboxing using Subshells. TODO.\n- [parallel_optimization](https://github.com/riga/law/tree/master/examples/parallel_optimization): Parallel optimization using [scikit optimize](https://scikit-optimize.github.io).\n- [notifications](https://github.com/riga/law/tree/master/examples/notifications): Demonstration of slack and telegram task status notifications..\n- [CMS Single Top Analysis](https://github.com/riga/law_example_CMSSingleTopAnalysis): Simple physics analysis using law.\n\n\n## Further topics\n\n### Auto completion on the command-line\n\n**bash**\n\n```shell\nsource \"$( law completion )\"\n```\n\n**zsh**\n\nzsh is able to load and evaluate bash completion scripts via `bashcompinit`.\nIn order for `bashcompinit` to work, you should run `compinstall` to enable completion scripts:\n\n```shell\nautoload -Uz compinstall && compinstall\n```\n\nAfter following the instructions, these lines should be present in your `~/.zshrc`:\n\n```shell\n# The following lines were added by compinstall\nzstyle :compinstall filename '~/.zshrc'\n\nautoload -Uz +X compinit && compinit\nautoload -Uz +X bashcompinit && bashcompinit\n# End of lines added by compinstall\n```\n\nIf this is the case, just source the law completion script (which internally enables `bashcompinit`) and you're good to go:\n\n```shell\nsource \"$( law completion )\"\n```\n\n\n## Development\n\n- Source hosted at [GitHub](https://github.com/riga/law)\n- Report issues, questions, feature requests on [GitHub Issues](https://github.com/riga/law/issues)\n\n\n### Tests\n\nTo run and test law, there are various docker `riga/law` images available on the [DockerHub](https://cloud.docker.com/u/riga/repository/docker/riga/law), corresponding to different OS and Python versions (based on [micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)).\nStart them via\n\n```shell\ndocker run -ti riga/law:<the_tag>\n```\n\n| OS | Python | Tags |\n| ----------- | ------ | ----------------------------------------- |\n| AlmaLinux 9 | 3.11 | a9-py311, a9-py3, a9, py311, py3, latest |\n| AlmaLinux 9 | 3.10 | a9-py310, py310 |\n| AlmaLinux 9 | 3.9 | a9-py39, py39 |\n| AlmaLinux 9 | 3.8 | a9-py38, py38 |\n| AlmaLinux 9 | 3.7 | a9-py37, py37 |\n| CentOS 8 | 3.11 | c8-py311, c8-py3, c8 |\n| CentOS 8 | 3.10 | c8-py310 |\n| CentOS 8 | 3.9 | c8-py39 |\n| CentOS 8 | 3.8 | c8-py38 |\n| CentOS 8 | 3.7 | c8-py37 |\n| CentOS 7 | 3.10 | c7-py310, c7-py3, c7 |\n| CentOS 7 | 3.9 | c7-py39 |\n| CentOS 7 | 3.8 | c7-py38 |\n| CentOS 7 | 3.7 | c7-py37 |\n| CentOS 7 | 3.6 | c7-py36, py36 (removed soon) |\n| CentOS 7 | 2.7 | c7-py27, c7-py2, py27, py2 (removed soon) |\n\n\n### Contributors\n\n<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->\n<!-- prettier-ignore-start -->\n<!-- markdownlint-disable -->\n<table>\n <tbody>\n <tr>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"http://github.com/riga\"><img src=\"https://avatars.githubusercontent.com/u/1908734?v=4?s=100\" width=\"100px;\" alt=\"Marcel Rieger\"/><br /><sub><b>Marcel Rieger</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=riga\" title=\"Code\">\ud83d\udcbb</a> <a href=\"https://github.com/riga/law/pulls?q=is%3Apr+reviewed-by%3Ariga\" title=\"Reviewed Pull Requests\">\ud83d\udc40</a> <a href=\"#maintenance-riga\" title=\"Maintenance\">\ud83d\udea7</a> <a href=\"https://github.com/riga/law/commits?author=riga\" title=\"Documentation\">\ud83d\udcd6</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/pfackeldey\"><img src=\"https://avatars.githubusercontent.com/u/18463582?v=4?s=100\" width=\"100px;\" alt=\"Peter Fackeldey\"/><br /><sub><b>Peter Fackeldey</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=pfackeldey\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/yrath\"><img src=\"https://avatars.githubusercontent.com/u/20044510?v=4?s=100\" width=\"100px;\" alt=\"Yannik Rath\"/><br /><sub><b>Yannik Rath</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=yrath\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/jaimeleonh\"><img src=\"https://avatars.githubusercontent.com/u/47629805?v=4?s=100\" width=\"100px;\" alt=\"Jaime Leon Holgado\"/><br /><sub><b>Jaime Leon Holgado</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=jaimeleonh\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/lmoureaux\"><img src=\"https://avatars.githubusercontent.com/u/22327575?v=4?s=100\" width=\"100px;\" alt=\"Louis Moureaux\"/><br /><sub><b>Louis Moureaux</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=lmoureaux\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"http://www.linkedin.com/in/lgeiger\"><img src=\"https://avatars.githubusercontent.com/u/13285808?v=4?s=100\" width=\"100px;\" alt=\"Lukas Geiger\"/><br /><sub><b>Lukas Geiger</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=lgeiger\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://too.gy\"><img src=\"https://avatars.githubusercontent.com/u/694873?v=4?s=100\" width=\"100px;\" alt=\"Valentin Iovene\"/><br /><sub><b>Valentin Iovene</b></sub></a><br /><a href=\"https://github.com/riga/law/commits?author=tgy\" title=\"Code\">\ud83d\udcbb</a></td>\n </tr>\n </tbody>\n</table>\n\n<!-- markdownlint-restore -->\n<!-- prettier-ignore-end -->\n\n<!-- ALL-CONTRIBUTORS-LIST:END -->\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification.\n\n<!-- marker-after-body -->\n\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Build large-scale task workflows using luigi, remote job submission, remote targets, and environment",
"version": "0.1.18",
"project_urls": {
"Homepage": "https://github.com/riga/law"
},
"split_keywords": [
"luigi",
"workflow",
"pipeline",
"remote",
"gfal",
"submission",
"cluster",
"grid",
"condor",
"lsf",
"glite",
"arc",
"sandboxing",
"docker",
"singularity"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2d9a28b395a403afe033891dab41d74df54d8103b593e6a0996224332dba2f6e",
"md5": "6c511b64b05d75800a198647e3b053c1",
"sha256": "b9b655c8b91ea0c56e9a8aa7243bd03b135f17b9ef700ee0c1de591729f3fd05"
},
"downloads": -1,
"filename": "law-0.1.18.tar.gz",
"has_sig": false,
"md5_digest": "6c511b64b05d75800a198647e3b053c1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4",
"size": 287857,
"upload_time": "2024-02-11T10:31:57",
"upload_time_iso_8601": "2024-02-11T10:31:57.245777Z",
"url": "https://files.pythonhosted.org/packages/2d/9a/28b395a403afe033891dab41d74df54d8103b593e6a0996224332dba2f6e/law-0.1.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-11 10:31:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "riga",
"github_project": "law",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "law"
}
