| Name | LbCondaWrappers JSON |
| Version |
0.4.3
JSON |
| download |
| home_page | |
| Summary | Wrappers for using LHCb CVMFS conda installations |
| upload_time | 2024-02-25 10:26:31 |
| maintainer | |
| docs_url | None |
| author | LHCb |
| requires_python | >=3.9 |
| license | |
| keywords |
lhcb
core
task
runner
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LHCb Conda Wrappers
Wrapper scripts for providing access to conda environments which are installed on CVMFS.
## Standard Usage
The main environment provided is named `default` and aims to provide most software tools that will be needed for analaysis that is performed outside of the standard LHCb software stack.
This includes a recent version of Python 3, ROOT, Snakemake, jupyterlab, matplotlib, scikit-learn, tensorflow and many more.
### Basic usage
Environments can be activated using the `lb-conda` command which works similarly to `lb-run`.
To launch a `bash` shell inside the default environment run:
```bash
$ lb-conda default bash
[bash-5.0]$ python --version
Python 3.7.6
[bash-5.0]$ root --version
ROOT Version: 6.20/04
Built for linuxx8664gcc on Apr 20 2020, 15:03:00
From @
```
Alternative commands can be ran directly using:
```bash
$ lb-conda default python -c 'import math; print(math.sqrt(2))'
1.4142135623730951
```
### Including texlive
A fully featured texlive installation is not included in any of the environments due to it's large size however one can be added by passing the `--texlive` argument.
```bash
$ lb-conda --texlive default latex --version
pdfTeX 3.14159265-2.6-1.40.21 (TeX Live 2020)
kpathsea version 6.3.2
Copyright 2020 Han The Thanh (pdfTeX) et al.
There is NO warranty. Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Han The Thanh (pdfTeX) et al.
Compiled with libpng 1.6.37; using libpng 1.6.37
Compiled with zlib 1.2.11; using zlib 1.2.11
Compiled with xpdf version 4.02
```
### Versioning
Occasionally new versions of the `default` environment will be made to add new packages and generally keep it up-to-date.
If you wish to use an older version you can list the available versions using:
```bash
$ lb-conda --list default
2019-12-26
2020-05-14
```
The `2020-05-14` version can explicitly chosen using:
```bash
$ lb-conda default/2020-05-14 bash
```
### Customising
As the conda environment used is installed on the read-only CVMFS filesystem the usual `pip install` and `conda install` commands will not work.
Two options are provided to allow environments to be customised.
### Installing additional python packages
This is the preferred option as most data will be kept on CVMFS and works by creating a virtual environment on top of the CVMFS installed environment.
It is however limited to only allowing `pip` installed Python packages and locally built software.
To create the virtual environment in a local directory ("my-local-directory") run
```bash
$ lb-conda-dev virtual-env default my-local-directory
```
The environment can now be used similarly to `lb-conda default` using the `run` script:
```bash
$ my-local-directory/run bash
```
To upgrade the `uproot` Python package run:
```bash
$ my-local-directory/run bahs
[bash-5.0]$ python -c 'import uproot; print(uproot.__version__)'
3.11.6 # <- Original version
[bash-5.0]$ pip install --upgrade uproot
Collecting uproot
# Truncated output
Successfully installed uproot-3.11.7
[bash-5.0]$ python -c 'import uproot; print(uproot.__version__)'
3.11.7 # <- New version
```
To install a locally compiled application in the virtual environment you should set the install prefix to be the absolute path to your local directory.
How this is done will depend on the build system used by the specific application.
### Fully customising an environment
This option is discouraged, especially when running on AFS/EOS as it will result in a large amount of data and many small files being copied to your local directory.
It does however allow you to completely clone the environment and then make any modifications.
TODO: This is not currently documented...
## Advanced usage
### Using non-default environments
To see the full list of available environments run:
```bash
$ lb-conda --list
B2OC/b2dstdspi-gpu
B2OC/b2dstdspi
Charm/D02KsHH
Semilep/rdst
default
DPA/analysis-productions-certification
DPA/analysis-productions
```
A custom environment can then be chosen using it's name:
```bash
$ lb-conda Charm/D02KsHH python --version
Python 3.6.5 :: Anaconda, Inc.
```
### Suggesting new or upgraded packages in the default environment
Please open an issue [here](https://gitlab.cern.ch/lhcb-core/conda-environments/-/issues) explaining what should be added/upgraded and why.
### Adding new environments to CVMFS
This is not yet widely available available.
If you have a strong reason to add an environment please open an issue [here](https://gitlab.cern.ch/lhcb-core/conda-environments/-/issues) to explaing what your requirements are.
### Acces conda environments from a CI job.
Add the the tag `cvmfs` for the job to be executed in a shared runner where CVMFS is mounted, and source the `LbEnv` environment.
```yaml
testjob:
tags:
- cvmfs
before_script:
- source /cvmfs/lhcb.cern.ch/lib/LbEnv
script:
- lb-conda default snakemake -j
```
Raw data
{
"_id": null,
"home_page": "",
"name": "LbCondaWrappers",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "LHCb,Core,task,runner",
"author": "LHCb",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/f1/25/576c1ce1aac84b4d4a3f6ef3afb7d9311a6a246114d9bd847654ca7a8a5a/LbCondaWrappers-0.4.3.tar.gz",
"platform": null,
"description": "# LHCb Conda Wrappers\n\nWrapper scripts for providing access to conda environments which are installed on CVMFS.\n\n## Standard Usage\n\nThe main environment provided is named `default` and aims to provide most software tools that will be needed for analaysis that is performed outside of the standard LHCb software stack.\nThis includes a recent version of Python 3, ROOT, Snakemake, jupyterlab, matplotlib, scikit-learn, tensorflow and many more.\n\n### Basic usage\n\nEnvironments can be activated using the `lb-conda` command which works similarly to `lb-run`.\nTo launch a `bash` shell inside the default environment run:\n\n```bash\n$ lb-conda default bash\n[bash-5.0]$ python --version\nPython 3.7.6\n[bash-5.0]$ root --version\nROOT Version: 6.20/04\nBuilt for linuxx8664gcc on Apr 20 2020, 15:03:00\nFrom @\n```\n\nAlternative commands can be ran directly using:\n\n```bash\n$ lb-conda default python -c 'import math; print(math.sqrt(2))'\n1.4142135623730951\n```\n\n### Including texlive\n\nA fully featured texlive installation is not included in any of the environments due to it's large size however one can be added by passing the `--texlive` argument.\n\n```bash\n$ lb-conda --texlive default latex --version\npdfTeX 3.14159265-2.6-1.40.21 (TeX Live 2020)\nkpathsea version 6.3.2\nCopyright 2020 Han The Thanh (pdfTeX) et al.\nThere is NO warranty. Redistribution of this software is\ncovered by the terms of both the pdfTeX copyright and\nthe Lesser GNU General Public License.\nFor more information about these matters, see the file\nnamed COPYING and the pdfTeX source.\nPrimary author of pdfTeX: Han The Thanh (pdfTeX) et al.\nCompiled with libpng 1.6.37; using libpng 1.6.37\nCompiled with zlib 1.2.11; using zlib 1.2.11\nCompiled with xpdf version 4.02\n```\n\n### Versioning\n\nOccasionally new versions of the `default` environment will be made to add new packages and generally keep it up-to-date.\nIf you wish to use an older version you can list the available versions using:\n\n```bash\n$ lb-conda --list default\n2019-12-26\n2020-05-14\n```\n\nThe `2020-05-14` version can explicitly chosen using:\n\n```bash\n$ lb-conda default/2020-05-14 bash\n```\n\n### Customising\n\nAs the conda environment used is installed on the read-only CVMFS filesystem the usual `pip install` and `conda install` commands will not work.\nTwo options are provided to allow environments to be customised.\n\n### Installing additional python packages\n\nThis is the preferred option as most data will be kept on CVMFS and works by creating a virtual environment on top of the CVMFS installed environment.\nIt is however limited to only allowing `pip` installed Python packages and locally built software.\n\nTo create the virtual environment in a local directory (\"my-local-directory\") run\n\n```bash\n$ lb-conda-dev virtual-env default my-local-directory\n```\n\nThe environment can now be used similarly to `lb-conda default` using the `run` script:\n\n```bash\n$ my-local-directory/run bash\n```\n\nTo upgrade the `uproot` Python package run:\n\n```bash\n$ my-local-directory/run bahs\n[bash-5.0]$ python -c 'import uproot; print(uproot.__version__)'\n3.11.6 # <- Original version\n[bash-5.0]$ pip install --upgrade uproot\nCollecting uproot\n# Truncated output\nSuccessfully installed uproot-3.11.7\n[bash-5.0]$ python -c 'import uproot; print(uproot.__version__)'\n3.11.7 # <- New version\n```\n\nTo install a locally compiled application in the virtual environment you should set the install prefix to be the absolute path to your local directory.\nHow this is done will depend on the build system used by the specific application.\n\n### Fully customising an environment\n\nThis option is discouraged, especially when running on AFS/EOS as it will result in a large amount of data and many small files being copied to your local directory.\nIt does however allow you to completely clone the environment and then make any modifications.\n\nTODO: This is not currently documented...\n\n## Advanced usage\n\n### Using non-default environments\n\nTo see the full list of available environments run:\n\n```bash\n$ lb-conda --list\nB2OC/b2dstdspi-gpu\nB2OC/b2dstdspi\nCharm/D02KsHH\nSemilep/rdst\ndefault\nDPA/analysis-productions-certification\nDPA/analysis-productions\n```\n\nA custom environment can then be chosen using it's name:\n\n```bash\n$ lb-conda Charm/D02KsHH python --version\nPython 3.6.5 :: Anaconda, Inc.\n```\n\n### Suggesting new or upgraded packages in the default environment\n\nPlease open an issue [here](https://gitlab.cern.ch/lhcb-core/conda-environments/-/issues) explaining what should be added/upgraded and why.\n\n### Adding new environments to CVMFS\n\nThis is not yet widely available available.\nIf you have a strong reason to add an environment please open an issue [here](https://gitlab.cern.ch/lhcb-core/conda-environments/-/issues) to explaing what your requirements are.\n\n### Acces conda environments from a CI job.\n\nAdd the the tag `cvmfs` for the job to be executed in a shared runner where CVMFS is mounted, and source the `LbEnv` environment.\n\n```yaml\ntestjob:\n tags:\n - cvmfs\n before_script:\n - source /cvmfs/lhcb.cern.ch/lib/LbEnv\n script:\n - lb-conda default snakemake -j\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "Wrappers for using LHCb CVMFS conda installations",
"version": "0.4.3",
"project_urls": {
"Bug Reports": "https://gitlab.cern.ch/lhcb-core/LbCondaWrappers/issues",
"Homepage": "https://gitlab.cern.ch/lhcb-core/LbCondaWrappers",
"Source": "https://gitlab.cern.ch/lhcb-core/LbCondaWrappers"
},
"split_keywords": [
"lhcb",
"core",
"task",
"runner"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7738617ceccd9a15796edd9d13e746ab8f7ab19c55d786f2581a2f863286d792",
"md5": "40d4eb6fe43c4c7e0b3cec0670303be8",
"sha256": "f2f86cc0982d4532a55acc1748f34f6261b8fd934cb80fd74daafbf8aaebc496"
},
"downloads": -1,
"filename": "LbCondaWrappers-0.4.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "40d4eb6fe43c4c7e0b3cec0670303be8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 20890,
"upload_time": "2024-02-25T10:26:29",
"upload_time_iso_8601": "2024-02-25T10:26:29.750092Z",
"url": "https://files.pythonhosted.org/packages/77/38/617ceccd9a15796edd9d13e746ab8f7ab19c55d786f2581a2f863286d792/LbCondaWrappers-0.4.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f125576c1ce1aac84b4d4a3f6ef3afb7d9311a6a246114d9bd847654ca7a8a5a",
"md5": "13f5519db8d7788aff7a866622c1e5b7",
"sha256": "20937e503df420b6e248c9aab82cfa2545b65a7d5f93f0abcfa14dcea4e7eca4"
},
"downloads": -1,
"filename": "LbCondaWrappers-0.4.3.tar.gz",
"has_sig": false,
"md5_digest": "13f5519db8d7788aff7a866622c1e5b7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 26766,
"upload_time": "2024-02-25T10:26:31",
"upload_time_iso_8601": "2024-02-25T10:26:31.570369Z",
"url": "https://files.pythonhosted.org/packages/f1/25/576c1ce1aac84b4d4a3f6ef3afb7d9311a6a246114d9bd847654ca7a8a5a/LbCondaWrappers-0.4.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-25 10:26:31",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "lbcondawrappers"
}
