CP3SlurmUtils
=============
CP3SlurmUtils was written with the aim to be an utility for physicists of the
[CP3 institute](https://uclouvain.be/en/research-institutes/irmp/cp3) that would
facilitate the submission (and resubmission) of jobs to their computing cluster,
which is managed by [Slurm](https://slurm.schedmd.com/documentation.html). Most
of the physicists at CP3 that would use this utility are collaborating with the
CMS experiment at CERN.
In CP3SlurmUtils, jobs are defined via a submit configuration file written in
Python similar to the type of submit configuration file used in [CRABClient](https://github.com/dmwm/CRABClient),
the tool used by CMS users to submit jobs to the Worldwide LHC Computing Grid.
The utility creates then the batch scripts to be submitted with the Slurm __sbatch__ command.
The submission can be done on-the-go by the utility itself or later-on by the user.
CP3SlurmUtils provides the following features:
- handling of job stdout/stderr
- handling of a scratch directory
- handling of job exit codes
- handling of job input and output files
- setup of the CMS environment (optional)
For the copying of files to/from the worker nodes, the __cp__ command is used,
assuming there is a filesystem shared with the submission node.
## INSTALLATION
CP3SlurmUtils can be installed via the [Python Package Installer](https://pip.pypa.io/en/stable/),
or it can simply be downloaded from the git repository:
https://gitlab.cern.ch/cp3-cms/CP3SlurmUtils.
## CONFIGURATION
An example configuration file for CP3SlurmUtils named _defaults.cfg.example_
is included in the package. In the git repository, it can be found in the
__etc__ folder. In the distribution, it can be found in a sub-directory named
__etc/CP3SlurmUtils__.
CP3SlurmUtils will try to read the _defaults.cfg_ configuration file from the
following locations, in this order:
- /etc/CP3SlurmUtils/
- $xdg_config_dir/CP3SlurmUtils/ where $xdg_config_dir is each of the paths
defined by the environment variable $XDG_CONFIG_DIRS; if $XDG_CONFIG_DIRS is
not defined, $xdg_config_dir defaults to /etc/xdg/
- site.USER_BASE/etc/CP3SlurmUtils/ where site.USER_BASE is the python variable
that defines the location for a user installation
- $xdg_config_home/CP3SlurmUtils/ where $xdg_config_home is the environment
variable $XDG_CONFIG_HOME when defined and not empty; otherwise it defaults to
$HOME/.config
- $VIRTUAL_ENV/etc/CP3SlurmUtils/ when $VIRTUAL_ENV is defined and not empty
There is no installation or configuration script provided with this utility;
it is left to the user to put the _defaults.cfg_ configuration file in a visible
location or appropriately point the environment variable $XDG_CONFIG_DIRS or
$XDG_CONFIG_HOME to it.
## REPORTING BUGS
To report an issue/bug or to provide suggestions for enhancement/improvement,
open an issue in the git repository: https://gitlab.cern.ch/cp3-cms/CP3SlurmUtils.
Pull requests are also welcome.
Questions can be sent to cp3-support@uclouvain.be.
## LEGAL
CP3SlurmUtils is provided "as is" and with no warranty. This software is
distributed under the GNU General Public License; please see the file LICENSE
for details.
Raw data
{
"_id": null,
"home_page": "https://gitlab.cern.ch/cp3-cms/CP3SlurmUtils",
"name": "CP3SlurmUtils",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "slurm",
"author": "Andres Tanasijczuk (Universite\u0301 catholique de Louvain)",
"author_email": "cp3-support@listes.uclouvain.be",
"download_url": "https://files.pythonhosted.org/packages/98/22/33c2829f0b0130671de568366d878fbeceacd85c97359abfcc681a19737e/CP3SlurmUtils-0.3.0.tar.gz",
"platform": null,
"description": "CP3SlurmUtils\n=============\n\nCP3SlurmUtils was written with the aim to be an utility for physicists of the\n[CP3 institute](https://uclouvain.be/en/research-institutes/irmp/cp3) that would\nfacilitate the submission (and resubmission) of jobs to their computing cluster,\nwhich is managed by [Slurm](https://slurm.schedmd.com/documentation.html). Most\nof the physicists at CP3 that would use this utility are collaborating with the\nCMS experiment at CERN.\n\nIn CP3SlurmUtils, jobs are defined via a submit configuration file written in\nPython similar to the type of submit configuration file used in [CRABClient](https://github.com/dmwm/CRABClient),\nthe tool used by CMS users to submit jobs to the Worldwide LHC Computing Grid.\nThe utility creates then the batch scripts to be submitted with the Slurm __sbatch__ command.\nThe submission can be done on-the-go by the utility itself or later-on by the user.\n\nCP3SlurmUtils provides the following features:\n\n- handling of job stdout/stderr\n- handling of a scratch directory\n- handling of job exit codes\n- handling of job input and output files\n- setup of the CMS environment (optional)\n\nFor the copying of files to/from the worker nodes, the __cp__ command is used,\nassuming there is a filesystem shared with the submission node.\n\n## INSTALLATION\n\nCP3SlurmUtils can be installed via the [Python Package Installer](https://pip.pypa.io/en/stable/),\nor it can simply be downloaded from the git repository:\nhttps://gitlab.cern.ch/cp3-cms/CP3SlurmUtils.\n\n## CONFIGURATION\n\nAn example configuration file for CP3SlurmUtils named _defaults.cfg.example_\nis included in the package. In the git repository, it can be found in the\n__etc__ folder. In the distribution, it can be found in a sub-directory named\n__etc/CP3SlurmUtils__.\n\nCP3SlurmUtils will try to read the _defaults.cfg_ configuration file from the\nfollowing locations, in this order:\n\n- /etc/CP3SlurmUtils/\n- $xdg_config_dir/CP3SlurmUtils/ where $xdg_config_dir is each of the paths\n defined by the environment variable $XDG_CONFIG_DIRS; if $XDG_CONFIG_DIRS is\n not defined, $xdg_config_dir defaults to /etc/xdg/\n- site.USER_BASE/etc/CP3SlurmUtils/ where site.USER_BASE is the python variable\n that defines the location for a user installation\n- $xdg_config_home/CP3SlurmUtils/ where $xdg_config_home is the environment\n variable $XDG_CONFIG_HOME when defined and not empty; otherwise it defaults to\n $HOME/.config\n- $VIRTUAL_ENV/etc/CP3SlurmUtils/ when $VIRTUAL_ENV is defined and not empty \n\nThere is no installation or configuration script provided with this utility;\nit is left to the user to put the _defaults.cfg_ configuration file in a visible\nlocation or appropriately point the environment variable $XDG_CONFIG_DIRS or\n$XDG_CONFIG_HOME to it.\n\n## REPORTING BUGS\n\nTo report an issue/bug or to provide suggestions for enhancement/improvement,\nopen an issue in the git repository: https://gitlab.cern.ch/cp3-cms/CP3SlurmUtils.\nPull requests are also welcome.\nQuestions can be sent to cp3-support@uclouvain.be.\n\n## LEGAL\n\nCP3SlurmUtils is provided \"as is\" and with no warranty. This software is\ndistributed under the GNU General Public License; please see the file LICENSE\nfor details.\n\n\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Utilities package to submit jobs to SLURM using as input a python sbatch configuration file.",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://gitlab.cern.ch/cp3-cms/CP3SlurmUtils"
},
"split_keywords": [
"slurm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1f6af7ce4a1dd8c361fa99b3f0cec8d3561cdc89176175061442bad0b75b09f7",
"md5": "969cd35c56c463b84a636fcbe5335a65",
"sha256": "ce53ee79b1c1c7b02471dcc2f78395a21ad3d823dc1ca954a62456d8a43d48d4"
},
"downloads": -1,
"filename": "CP3SlurmUtils-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "969cd35c56c463b84a636fcbe5335a65",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 54748,
"upload_time": "2023-09-07T13:34:56",
"upload_time_iso_8601": "2023-09-07T13:34:56.748392Z",
"url": "https://files.pythonhosted.org/packages/1f/6a/f7ce4a1dd8c361fa99b3f0cec8d3561cdc89176175061442bad0b75b09f7/CP3SlurmUtils-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "982233c2829f0b0130671de568366d878fbeceacd85c97359abfcc681a19737e",
"md5": "1ed0b1653dd77f0e67e3ee31c8eec388",
"sha256": "90d54de88c0ac9dfe291023fc23add90de0fcdd50986985c9eab40ae371200ce"
},
"downloads": -1,
"filename": "CP3SlurmUtils-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "1ed0b1653dd77f0e67e3ee31c8eec388",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 49340,
"upload_time": "2023-09-07T13:34:58",
"upload_time_iso_8601": "2023-09-07T13:34:58.292327Z",
"url": "https://files.pythonhosted.org/packages/98/22/33c2829f0b0130671de568366d878fbeceacd85c97359abfcc681a19737e/CP3SlurmUtils-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-07 13:34:58",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "cp3slurmutils"
}
JSON