<div align="center">
<br/>
<img src="slappt.png" style="position:relative;width:250px;" />
[Slurm](https://slurm.schedmd.com/overview.html) scripts for [Apptainer](http://apptainer.org) jobs
[](https://pypi.python.org/pypi/slappt)
[](https://pypi.python.org/pypi/slappt)
[](https://pypi.python.org/pypi/slappt)
[](https://github.com/Computational-Plant-Science/slappt/actions/workflows/ci.yml)
[](https://slappt.readthedocs.io/en/latest/?badge=latest)
</div>
## Contents
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
- [Overview](#overview)
- [Requirements](#requirements)
- [Installation](#installation)
- [Quickstart](#quickstart)
- [Caveats](#caveats)
- [Shell selection](#shell-selection)
- [Singularity support](#singularity-support)
- [Pre-commands](#pre-commands)
- [Documentation](#documentation)
- [Disclaimer](#disclaimer)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## Overview
`slappt` generates and submits Slurm job scripts for [Apptainer](https://apptainer.org/docs/user/main/) workflows. Jobs can be configured in YAML or via CLI.
## Requirements
`slappt` requires Python3.8+ and a few core dependencies, including `click`, `pyaml`, `paramiko`, and `requests`, among others.
To submit a job script, the host machine must either run `slurmctld` with standard commands available, or must be able to connect via key- or password-authenticated SSH to the target cluster.
## Installation
`slappt` is [available on the Python Package Index](https://pypi.org/project/slappt/) and can be installed with `pip`:
```shell
pip install slappt
```
## Quickstart
Say you have access to a Slurm cluster with `apptainer` installed, and you have permission to submit to the `batch` partition.
Copy the `hello.yaml` file from the `examples` directory to your current working directory, then run:
```shell
slappt hello.yaml > job.sh
```
Alternatively, without the configuration file:
```shell
slappt --image docker://alpine \
--shell sh \
--partition batch \
--entrypoint "echo 'hello world'" > hello.sh
```
Your `hello.sh` script should now contain:
```shell
#!/bin/bash
#SBATCH --job-name=0477f4b9-e119-4354-8384-f50d7a96adad
#SBATCH --output=slappt.0477f4b9-e119-4354-8384-f50d7a96adad.%j.out
#SBATCH --error=slappt.0477f4b9-e119-4354-8384-f50d7a96adad.%j.err
#SBATCH --partition=batch
#SBATCH -c 1
#SBATCH -N 1
#SBATCH --ntasks=1
#SBATCH --time=01:00:00
#SBATCH --mem=1GB
apptainer exec docker://alpine sh -c "echo 'hello world'"
```
If already on the cluster, use the `--submit` flag to submit the job directly. (Standard Slurm commands must be available for this to work.) In this case the job ID is shown if submission was successful.
You can provide authentication information to submit the script to remote clusters over SSH. For instance, assuming you have key authentication set up and your key is `~/.ssh/id_rsa`:
```shell
slappt ... --host <cluster IP or FQDN> --username <username>
```
### Caveats
There are a few things to note about the example above.
#### Shell selection
For most image definitions, specifying the `shell` is likely not necessary, as the default is `bash`. However, for images that don't have `bash` installed (e.g., `alphine` only has `sh`) a different shell must be selected.
#### Singularity support
If your cluster still uses `singularity`, pass the `--singularity` flag (or set the `singularity` key in the configuration file to `true`) to substitute `singularity` for `apptainer` in the command wrapping your workflow entrypoint.
#### Pre-commands
**Note:** if `apptainer` or `singularity` are not available by default on your cluster's compute nodes, you may need to add `--pre` commands (or a `pre` section to the configuration file), for instance `--pre "module load apptainer"`, or:
```yaml
...
pre:
- module load apptainer
...
```
## Documentation
Documentation is available at [slappt.readthedocs.io](https://slappt.readthedocs.io/en/latest/).
<!--
## Related
There is a companion repository [`slappt-action`](https://github.com/Computational-Plant-Science/slappt-action) for easy integration with GitHub Actions.
-->
## Disclaimer
This project is not affiliated with Slurm, Apptainer or Singularity and cannot guarantee compatibility with all cluster configurations.
Raw data
{
"_id": null,
"home_page": "https://github.com/Computational-Plant-Science/slappt",
"name": "slappt",
"maintainer": "Wes Bonelli",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "wbonelli@uga.edu",
"keywords": "hpc,containers,slurm,apptainer",
"author": "Computational Plant Science Lab",
"author_email": "wbonelli@uga.edu",
"download_url": "https://files.pythonhosted.org/packages/75/67/65ce8bde81b3ebb008df229647786553445a5903e9b831ac8a1788ca0b28/slappt-0.0.3.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n<br/>\n<img src=\"slappt.png\" style=\"position:relative;width:250px;\" />\n\n[Slurm](https://slurm.schedmd.com/overview.html) scripts for [Apptainer](http://apptainer.org) jobs\n\n[](https://pypi.python.org/pypi/slappt)\n[](https://pypi.python.org/pypi/slappt)\n[](https://pypi.python.org/pypi/slappt)\n\n[](https://github.com/Computational-Plant-Science/slappt/actions/workflows/ci.yml)\n[](https://slappt.readthedocs.io/en/latest/?badge=latest)\n\n</div>\n\n## Contents\n\n<!-- START doctoc generated TOC please keep comment here to allow auto update -->\n<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->\n\n- [Overview](#overview)\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [Quickstart](#quickstart)\n - [Caveats](#caveats)\n - [Shell selection](#shell-selection)\n - [Singularity support](#singularity-support)\n - [Pre-commands](#pre-commands)\n- [Documentation](#documentation)\n- [Disclaimer](#disclaimer)\n\n<!-- END doctoc generated TOC please keep comment here to allow auto update -->\n\n## Overview\n\n`slappt` generates and submits Slurm job scripts for [Apptainer](https://apptainer.org/docs/user/main/) workflows. Jobs can be configured in YAML or via CLI.\n\n\n## Requirements\n\n`slappt` requires Python3.8+ and a few core dependencies, including `click`, `pyaml`, `paramiko`, and `requests`, among others.\n\nTo submit a job script, the host machine must either run `slurmctld` with standard commands available, or must be able to connect via key- or password-authenticated SSH to the target cluster.\n\n## Installation\n\n`slappt` is [available on the Python Package Index](https://pypi.org/project/slappt/) and can be installed with `pip`:\n\n```shell\npip install slappt\n```\n\n## Quickstart\n\nSay you have access to a Slurm cluster with `apptainer` installed, and you have permission to submit to the `batch` partition.\n\nCopy the `hello.yaml` file from the `examples` directory to your current working directory, then run:\n\n```shell\nslappt hello.yaml > job.sh\n```\n\nAlternatively, without the configuration file:\n\n```shell\nslappt --image docker://alpine \\\n --shell sh \\\n --partition batch \\\n --entrypoint \"echo 'hello world'\" > hello.sh\n```\n\nYour `hello.sh` script should now contain:\n\n```shell\n#!/bin/bash\n#SBATCH --job-name=0477f4b9-e119-4354-8384-f50d7a96adad\n#SBATCH --output=slappt.0477f4b9-e119-4354-8384-f50d7a96adad.%j.out\n#SBATCH --error=slappt.0477f4b9-e119-4354-8384-f50d7a96adad.%j.err\n#SBATCH --partition=batch\n#SBATCH -c 1\n#SBATCH -N 1\n#SBATCH --ntasks=1\n#SBATCH --time=01:00:00\n#SBATCH --mem=1GB\napptainer exec docker://alpine sh -c \"echo 'hello world'\"\n```\n\nIf already on the cluster, use the `--submit` flag to submit the job directly. (Standard Slurm commands must be available for this to work.) In this case the job ID is shown if submission was successful.\n\nYou can provide authentication information to submit the script to remote clusters over SSH. For instance, assuming you have key authentication set up and your key is `~/.ssh/id_rsa`:\n\n```shell\nslappt ... --host <cluster IP or FQDN> --username <username>\n```\n\n### Caveats\n\nThere are a few things to note about the example above.\n\n#### Shell selection\n\nFor most image definitions, specifying the `shell` is likely not necessary, as the default is `bash`. However, for images that don't have `bash` installed (e.g., `alphine` only has `sh`) a different shell must be selected.\n\n#### Singularity support\n\nIf your cluster still uses `singularity`, pass the `--singularity` flag (or set the `singularity` key in the configuration file to `true`) to substitute `singularity` for `apptainer` in the command wrapping your workflow entrypoint.\n\n#### Pre-commands\n\n**Note:** if `apptainer` or `singularity` are not available by default on your cluster's compute nodes, you may need to add `--pre` commands (or a `pre` section to the configuration file), for instance `--pre \"module load apptainer\"`, or:\n\n```yaml\n...\npre:\n - module load apptainer\n...\n```\n\n## Documentation\n\nDocumentation is available at [slappt.readthedocs.io](https://slappt.readthedocs.io/en/latest/).\n\n<!--\n\n## Related\n\nThere is a companion repository [`slappt-action`](https://github.com/Computational-Plant-Science/slappt-action) for easy integration with GitHub Actions.\n\n-->\n\n## Disclaimer\n\nThis project is not affiliated with Slurm, Apptainer or Singularity and cannot guarantee compatibility with all cluster configurations.\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "generate slurm scripts for apptainer jobs",
"version": "0.0.3",
"split_keywords": [
"hpc",
"containers",
"slurm",
"apptainer"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "2b6094cf3fe4e069be2bd6b91a01c1c4",
"sha256": "be4fbf0140879fbebe609e9da4a5c553d4e2779a9a1a5b9d43cccb76841ae14d"
},
"downloads": -1,
"filename": "slappt-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2b6094cf3fe4e069be2bd6b91a01c1c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 19963,
"upload_time": "2022-12-31T22:41:32",
"upload_time_iso_8601": "2022-12-31T22:41:32.155044Z",
"url": "https://files.pythonhosted.org/packages/61/5b/eb8cdf32cb01b160a6b8d4a78f66052b65fc125d8ef3df28e150b5324f4a/slappt-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "718bce5517d8df53b6da4268ab1b843c",
"sha256": "ac4cca927c4c2344cb6d88b735c68edfc2aee1cc94df0cf929f737e06a12d872"
},
"downloads": -1,
"filename": "slappt-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "718bce5517d8df53b6da4268ab1b843c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 18631,
"upload_time": "2022-12-31T22:41:33",
"upload_time_iso_8601": "2022-12-31T22:41:33.771926Z",
"url": "https://files.pythonhosted.org/packages/75/67/65ce8bde81b3ebb008df229647786553445a5903e9b831ac8a1788ca0b28/slappt-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-31 22:41:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "Computational-Plant-Science",
"github_project": "slappt",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "slappt"
}
JSON
