<h1 align="center">Simple Slurm</h1>
<p align="center">A simple Python wrapper for Slurm with flexibility in mind<p>
<p align="center">
<a href="https://github.com/amq92/simple_slurm"><img src="https://img.shields.io/github/actions/workflow/status/amq92/simple_slurm/python-run-tests.yml" alt="Run Tests"/></a>
<a href="https://pypistats.org/packages/simple-slurm"><img src="https://img.shields.io/pypi/dm/simple_slurm" alt="PyPI Downloads"/></a>
<a href="https://pypi.org/project/simple-slurm"><img src="https://img.shields.io/pypi/v/simple_slurm" alt="PyPI Version"/></a>
<a href="https://anaconda.org/conda-forge/simple_slurm"><img src="https://img.shields.io/conda/vn/conda-forge/simple_slurm.svg" alt="Conda Version"/></a>
</p>
```python
import datetime
from simple_slurm import Slurm
slurm = Slurm(
array=range(3, 12),
cpus_per_task=15,
dependency=dict(after=65541, afterok=34987),
gres=["gpu:kepler:2", "gpu:tesla:2", "mps:400"],
ignore_pbs=True,
job_name="name",
output=f"{Slurm.JOB_ARRAY_MASTER_ID}_{Slurm.JOB_ARRAY_ID}.out",
time=datetime.timedelta(days=1, hours=2, minutes=3, seconds=4),
)
slurm.add_cmd("module load python")
slurm.sbatch("python demo.py", Slurm.SLURM_ARRAY_TASK_ID)
```
The above snippet is equivalent to running the following command:
```bash
sbatch << EOF
#!/bin/sh
#SBATCH --array 3-11
#SBATCH --cpus-per-task 15
#SBATCH --dependency after:65541,afterok:34987
#SBATCH --gres gpu:kepler:2,gpu:tesla:2,mps:400
#SBATCH --ignore-pbs
#SBATCH --job-name name
#SBATCH --output %A_%a.out
#SBATCH --time 1-02:03:04
module load python
python demo.py \$SLURM_ARRAY_TASK_ID
EOF
```
## Contents
+ [Installation](#installation)
+ [Introduction](#introduction)
+ [Core Features](#core-features)
- [Pythonic Slurm Syntax](#pythonic-slurm-syntax) *(was "Many syntaxes available")*
- [Adding Commands with `add_cmd`](#adding-commands-with-add_cmd)
- [Job Dependencies](#job-dependencies)
+ [Advanced Features](#advanced-features)
- [Command-Line Interface (CLI)](#command-line-interface-cli)
- [Using Configuration Files](#using-configuration-files)
- [Filename Patterns and Environment Variables](#filename-patterns-and-environment-variables)
+ [Job Management](#job-management)
- [Monitoring Jobs with `squeue`](#monitoring-jobs-with-squeue)
- [Canceling Jobs with `scancel`](#canceling-jobs-with-scancel)
+ [Error Handling](#error-handling)
+ [Project Growth](#project-growth)
## Installation
The source code is currently hosted : [https://github.com/amq92/simple_slurm](https://github.com/amq92/simple_slurm)
Install the latest `simple_slurm` version with:
```bash
pip install simple_slurm
```
or using `conda`
```bash
conda install -c conda-forge simple_slurm
```
## Introduction
The [`sbatch`](https://slurm.schedmd.com/sbatch.html) and [`srun`](https://slurm.schedmd.com/srun.html) commands in [Slurm](https://slurm.schedmd.com/overview.html) allow submitting parallel jobs into a Linux cluster in the form of batch scripts that follow a certain structure.
The goal of this library is to provide a simple wrapper for these core functions so that Python code can be used for constructing and launching the aforementioned batch script.
Indeed, the generated batch script can be shown by printing the `Slurm` object:
```python
from simple_slurm import Slurm
slurm = Slurm(array=range(3, 12), job_name="name")
print(slurm)
```
```bash
>> #!/bin/sh
>>
>> #SBATCH --array 3-11
>> #SBATCH --job-name name
```
Then, the job can be launched with either command:
```python
slurm.srun("echo hello!")
slurm.sbatch("echo hello!")
```
```bash
>> Submitted batch job 34987
```
While both commands are quite similar, [`srun`](https://slurm.schedmd.com/srun.html) will wait for the job completion, while [`sbatch`](https://slurm.schedmd.com/sbatch.html) will launch and disconnect from the jobs.
> More information can be found in [Slurm's Quick Start Guide](https://slurm.schedmd.com/quickstart.html) and in [here](https://stackoverflow.com/questions/43767866/slurm-srun-vs-sbatch-and-their-parameters).
## Core Features
### Pythonic Slurm Syntax
```python
slurm = Slurm("-a", "3-11")
slurm = Slurm("--array", "3-11")
slurm = Slurm("array", "3-11")
slurm = Slurm(array="3-11")
slurm = Slurm(array=range(3, 12))
slurm.add_arguments(array=range(3, 12))
slurm.set_array(range(3, 12))
```
All these arguments are equivalent!
It's up to you to choose the one(s) that best suits you needs.
> *"With great flexibility comes great responsability"*
You can either keep a command-line-like syntax or a more Python-like one.
```python
slurm = Slurm()
slurm.set_dependency("after:65541,afterok:34987")
slurm.set_dependency(["after:65541", "afterok:34987"])
slurm.set_dependency(dict(after=65541, afterok=34987))
```
All the possible arguments have their own setter methods
(ex. `set_array`, `set_dependency`, `set_job_name`).
Please note that hyphenated arguments, such as `--job-name`, need to be underscored
(so to comply with Python syntax and be coherent).
```python
slurm = Slurm("--job_name", "name")
slurm = Slurm(job_name="name")
# slurm = Slurm("--job-name", "name") # NOT VALID
# slurm = Slurm(job-name="name") # NOT VALID
```
Moreover, boolean arguments such as `--contiguous`, `--ignore_pbs` or `--overcommit`
can be activated with `True` or an empty string.
```python
slurm = Slurm("--contiguous", True)
slurm.add_arguments(ignore_pbs="")
slurm.set_wait(False)
print(slurm)
```
```bash
#!/bin/sh
#SBATCH --contiguous
#SBATCH --ignore-pbs
```
### Adding Commands with `add_cmd`
The `add_cmd` method allows you to add multiple commands to the Slurm job script. These commands will be executed in the order they are added before the main command specified in `sbatch` or `srun` directive.
```python
from simple_slurm import Slurm
slurm = Slurm(job_name="my_job", output="output.log")
# Add multiple commands
slurm.add_cmd("module load python")
slurm.add_cmd("export PYTHONPATH=/path/to/my/module")
slurm.add_cmd('echo "Environment setup complete"')
# Submit the job with the main command
slurm.sbatch("python my_script.py")
```
This will generate a Slurm job script like:
```bash
#!/bin/sh
#SBATCH --job-name my_job
#SBATCH --output output.log
module load python
export PYTHONPATH=/path/to/my/module
echo "Environment setup complete"
python my_script.py
```
You can reset the list of commands using the `reset_cmd` method:
```python
slurm.reset_cmd() # Clears all previously added commands
```
### Job dependencies
The `sbatch` call prints a message if successful and returns the corresponding `job_id`
```python
job_id = slurm.sbatch("python demo.py " + Slurm.SLURM_ARRAY_TAKSK_ID)
```
If the job submission was successful, it prints:
```
Submitted batch job 34987
```
And returns the variable `job_id = 34987`, which can be used for setting dependencies on subsequent jobs
```python
slurm_after = Slurm(dependency=dict(afterok=job_id)))
```
## Advanced Features
### Command-Line Interface (CLI)
For simpler dispatch jobs, a command line entry point is also made available.
```bash
simple_slurm [OPTIONS] "COMMAND_TO_RUN_WITH_SBATCH"
```
As such, both of these `python` and `bash` calls are equivalent.
```python
slurm = Slurm(partition="compute.p", output="slurm.log", ignore_pbs=True)
slurm.sbatch("echo \$HOSTNAME")
```
```bash
simple_slurm --partition=compute.p --output slurm.log --ignore_pbs "echo \$HOSTNAME"
```
### Using Configuration Files
Let's define the *static* components of a job definition in a YAML file `slurm_default.yml`
```yaml
cpus_per_task: 15
job_name: "name"
output: "%A_%a.out"
```
Including these options with the using the `yaml` package is very *simple*
```python
import yaml
from simple_slurm import Slurm
slurm = Slurm(**yaml.load(open("slurm_default.yml", "r")))
...
slurm.set_array(range(NUMBER_OF_SIMULATIONS))
```
The job can be updated according to the *dynamic* project needs (ex. `NUMBER_OF_SIMULATIONS`).
### Filename Patterns and Environment Variables
For convenience, Filename Patterns and Output Environment Variables are available as attributes of the Simple Slurm object.
See [https://slurm.schedmd.com/sbatch.html](https://slurm.schedmd.com/sbatch.html#lbAH) for details on the commands.
```python
from slurm import Slurm
slurm = Slurm(output=('{}_{}.out'.format(
Slurm.JOB_ARRAY_MASTER_ID,
Slurm.JOB_ARRAY_ID))
slurm.sbatch('python demo.py ' + slurm.SLURM_ARRAY_JOB_ID)
```
This example would result in output files of the form `65541_15.out`.
Here the job submission ID is `65541`, and this output file corresponds to the submission number `15` in the job array. Moreover, this index is passed to the Python code `demo.py` as an argument.
`sbatch` allows for a filename pattern to contain one or more replacement symbols. They can be accessed with `Slurm.<name>`
name | value | description
:-------------------|------:|:-----------
JOB_ARRAY_MASTER_ID | %A | job array's master job allocation number
JOB_ARRAY_ID | %a | job array id (index) number
JOB_ID_STEP_ID | %J | jobid.stepid of the running job. (e.g. "128.0")
JOB_ID | %j | jobid of the running job
HOSTNAME | %N | short hostname. this will create a separate io file per node
NODE_IDENTIFIER | %n | node identifier relative to current job (e.g. "0" is the first node of the running job) this will create a separate io file per node
STEP_ID | %s | stepid of the running job
TASK_IDENTIFIER | %t | task identifier (rank) relative to current job. this will create a separate io file per task
USER_NAME | %u | user name
JOB_NAME | %x | job name
PERCENTAGE | %% | the character "%"
DO_NOT_PROCESS | \\\\ | do not process any of the replacement symbols
The Slurm controller will set the following variables in the environment of the batch script. They can be accessed with `Slurm.<name>`.
name | description
:----------------------|:-----------
SLURM_ARRAY_TASK_COUNT | total number of tasks in a job array
SLURM_ARRAY_TASK_ID | job array id (index) number
SLURM_ARRAY_TASK_MAX | job array's maximum id (index) number
SLURM_ARRAY_TASK_MIN | job array's minimum id (index) number
SLURM_ARRAY_TASK_STEP | job array's index step size
SLURM_ARRAY_JOB_ID | job array's master job id number
... | ...
## Job Management
Simple Slurm provides a simple interface to Slurm's job management tools (`squeue` and `scance`l) to let you monitor and control running jobs.
### Monitoring Jobs with `squeue`
Retrieve and display job information for the current user:
```python
from simple_slurm import Slurm
slurm = Slurm()
slurm.squeue.update() # Fetch latest job data
# Get the jobs as a dictionary
jobs = slurm.squeue.jobs
for job_id, job in jobs.items():
print(job)
```
### Canceling Jobs with `scancel`
Cancel single jobs or entire job arrays:
```python
from simple_slurm import Slurm
slurm = Slurm()
# Cancel a specific job
slurm.scancel.cancel_job(34987)
# Cancel multiple jobs
for job_id in [34987, 34988, 34989]:
slurm.scancel.cancel_job(job_id)
# Send SIGTERM before canceling (graceful termination)
slurm.scancel.signal_job(34987)
slurm.scancel.cancel_job(34987)
```
## Error Handling
The library does not raise specific exceptions for invalid Slurm arguments or job submission failures. Instead, it relies on the underlying Slurm commands (`sbatch`, `srun`, etc.) to handle errors. If a job submission fails, the error message from Slurm will be printed to the console.
Additionally, if invalid arguments are passed to the Slurm object, the library uses `argparse` to validate them. If an argument is invalid, `argparse` will raise an error and print a helpful message.
For example:
```bash
simple_slurm --invalid_argument=value "echo \$HOSTNAME"
```
This will result in an error like:
```bash
usage: simple_slurm [OPTIONS] "COMMAND_TO_RUN_WITH_SBATCH"
simple_slurm: error: unrecognized arguments: --invalid_argument=value
```
## Project growth
[](https://star-history.com/#amq92/simple_slurm&Date)
Raw data
{
"_id": null,
"home_page": "https://github.com/amq92/simple_slurm",
"name": "simple-slurm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Arturo Mendoza",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/cc/57/dc898cd6ea39407cff09e89015a2bae1e42579da98a585cd3c6b796ec8d0/simple_slurm-0.3.5.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">Simple Slurm</h1>\n<p align=\"center\">A simple Python wrapper for Slurm with flexibility in mind<p>\n<p align=\"center\">\n<a href=\"https://github.com/amq92/simple_slurm\"><img src=\"https://img.shields.io/github/actions/workflow/status/amq92/simple_slurm/python-run-tests.yml\" alt=\"Run Tests\"/></a>\n<a href=\"https://pypistats.org/packages/simple-slurm\"><img src=\"https://img.shields.io/pypi/dm/simple_slurm\" alt=\"PyPI Downloads\"/></a>\n<a href=\"https://pypi.org/project/simple-slurm\"><img src=\"https://img.shields.io/pypi/v/simple_slurm\" alt=\"PyPI Version\"/></a>\n<a href=\"https://anaconda.org/conda-forge/simple_slurm\"><img src=\"https://img.shields.io/conda/vn/conda-forge/simple_slurm.svg\" alt=\"Conda Version\"/></a>\n</p>\n\n\n```python\nimport datetime\n\nfrom simple_slurm import Slurm\n\nslurm = Slurm(\n array=range(3, 12),\n cpus_per_task=15,\n dependency=dict(after=65541, afterok=34987),\n gres=[\"gpu:kepler:2\", \"gpu:tesla:2\", \"mps:400\"],\n ignore_pbs=True,\n job_name=\"name\",\n output=f\"{Slurm.JOB_ARRAY_MASTER_ID}_{Slurm.JOB_ARRAY_ID}.out\",\n time=datetime.timedelta(days=1, hours=2, minutes=3, seconds=4),\n)\nslurm.add_cmd(\"module load python\")\nslurm.sbatch(\"python demo.py\", Slurm.SLURM_ARRAY_TASK_ID)\n```\nThe above snippet is equivalent to running the following command:\n\n```bash\nsbatch << EOF\n#!/bin/sh\n\n#SBATCH --array 3-11\n#SBATCH --cpus-per-task 15\n#SBATCH --dependency after:65541,afterok:34987\n#SBATCH --gres gpu:kepler:2,gpu:tesla:2,mps:400\n#SBATCH --ignore-pbs\n#SBATCH --job-name name\n#SBATCH --output %A_%a.out\n#SBATCH --time 1-02:03:04\n\nmodule load python\npython demo.py \\$SLURM_ARRAY_TASK_ID\n\nEOF\n```\n\n## Contents\n\n+ [Installation](#installation)\n+ [Introduction](#introduction)\n+ [Core Features](#core-features)\n - [Pythonic Slurm Syntax](#pythonic-slurm-syntax) *(was \"Many syntaxes available\")*\n - [Adding Commands with `add_cmd`](#adding-commands-with-add_cmd)\n - [Job Dependencies](#job-dependencies)\n+ [Advanced Features](#advanced-features)\n - [Command-Line Interface (CLI)](#command-line-interface-cli)\n - [Using Configuration Files](#using-configuration-files)\n - [Filename Patterns and Environment Variables](#filename-patterns-and-environment-variables)\n+ [Job Management](#job-management)\n - [Monitoring Jobs with `squeue`](#monitoring-jobs-with-squeue)\n - [Canceling Jobs with `scancel`](#canceling-jobs-with-scancel)\n+ [Error Handling](#error-handling)\n+ [Project Growth](#project-growth)\n\n\n## Installation\n\nThe source code is currently hosted : [https://github.com/amq92/simple_slurm](https://github.com/amq92/simple_slurm)\n\nInstall the latest `simple_slurm` version with:\n\n```bash\npip install simple_slurm\n```\nor using `conda`\n```bash\nconda install -c conda-forge simple_slurm\n```\n\n## Introduction\n\nThe [`sbatch`](https://slurm.schedmd.com/sbatch.html) and [`srun`](https://slurm.schedmd.com/srun.html) commands in [Slurm](https://slurm.schedmd.com/overview.html) allow submitting parallel jobs into a Linux cluster in the form of batch scripts that follow a certain structure.\n\nThe goal of this library is to provide a simple wrapper for these core functions so that Python code can be used for constructing and launching the aforementioned batch script.\n\nIndeed, the generated batch script can be shown by printing the `Slurm` object:\n\n```python\nfrom simple_slurm import Slurm\n\nslurm = Slurm(array=range(3, 12), job_name=\"name\")\nprint(slurm)\n```\n```bash\n>> #!/bin/sh\n>> \n>> #SBATCH --array 3-11\n>> #SBATCH --job-name name\n```\n\nThen, the job can be launched with either command:\n```python\nslurm.srun(\"echo hello!\")\nslurm.sbatch(\"echo hello!\")\n```\n```bash\n>> Submitted batch job 34987\n```\n\nWhile both commands are quite similar, [`srun`](https://slurm.schedmd.com/srun.html) will wait for the job completion, while [`sbatch`](https://slurm.schedmd.com/sbatch.html) will launch and disconnect from the jobs.\n> More information can be found in [Slurm's Quick Start Guide](https://slurm.schedmd.com/quickstart.html) and in [here](https://stackoverflow.com/questions/43767866/slurm-srun-vs-sbatch-and-their-parameters).\n\n\n\n## Core Features\n\n### Pythonic Slurm Syntax\n\n```python\nslurm = Slurm(\"-a\", \"3-11\")\nslurm = Slurm(\"--array\", \"3-11\")\nslurm = Slurm(\"array\", \"3-11\")\nslurm = Slurm(array=\"3-11\")\nslurm = Slurm(array=range(3, 12))\nslurm.add_arguments(array=range(3, 12))\nslurm.set_array(range(3, 12))\n```\n\nAll these arguments are equivalent!\nIt's up to you to choose the one(s) that best suits you needs.\n\n> *\"With great flexibility comes great responsability\"*\n\nYou can either keep a command-line-like syntax or a more Python-like one.\n\n```python\nslurm = Slurm()\nslurm.set_dependency(\"after:65541,afterok:34987\")\nslurm.set_dependency([\"after:65541\", \"afterok:34987\"])\nslurm.set_dependency(dict(after=65541, afterok=34987))\n```\n\nAll the possible arguments have their own setter methods\n(ex. `set_array`, `set_dependency`, `set_job_name`).\n\nPlease note that hyphenated arguments, such as `--job-name`, need to be underscored\n(so to comply with Python syntax and be coherent).\n\n```python\nslurm = Slurm(\"--job_name\", \"name\")\nslurm = Slurm(job_name=\"name\")\n\n# slurm = Slurm(\"--job-name\", \"name\") # NOT VALID\n# slurm = Slurm(job-name=\"name\") # NOT VALID\n```\n\nMoreover, boolean arguments such as `--contiguous`, `--ignore_pbs` or `--overcommit` \ncan be activated with `True` or an empty string.\n\n```python\nslurm = Slurm(\"--contiguous\", True)\nslurm.add_arguments(ignore_pbs=\"\")\nslurm.set_wait(False)\nprint(slurm)\n```\n```bash\n#!/bin/sh\n\n#SBATCH --contiguous\n#SBATCH --ignore-pbs\n```\n\n\n### Adding Commands with `add_cmd`\n\nThe `add_cmd` method allows you to add multiple commands to the Slurm job script. These commands will be executed in the order they are added before the main command specified in `sbatch` or `srun` directive.\n\n```python\nfrom simple_slurm import Slurm\n\nslurm = Slurm(job_name=\"my_job\", output=\"output.log\")\n\n# Add multiple commands\nslurm.add_cmd(\"module load python\")\nslurm.add_cmd(\"export PYTHONPATH=/path/to/my/module\")\nslurm.add_cmd('echo \"Environment setup complete\"')\n\n# Submit the job with the main command\nslurm.sbatch(\"python my_script.py\")\n```\n\nThis will generate a Slurm job script like:\n\n```bash\n#!/bin/sh\n\n#SBATCH --job-name my_job\n#SBATCH --output output.log\n\nmodule load python\nexport PYTHONPATH=/path/to/my/module\necho \"Environment setup complete\"\npython my_script.py\n```\n\nYou can reset the list of commands using the `reset_cmd` method:\n\n```python\nslurm.reset_cmd() # Clears all previously added commands\n```\n\n\n### Job dependencies\n\nThe `sbatch` call prints a message if successful and returns the corresponding `job_id` \n\n```python\njob_id = slurm.sbatch(\"python demo.py \" + Slurm.SLURM_ARRAY_TAKSK_ID)\n```\n\nIf the job submission was successful, it prints:\n\n```\nSubmitted batch job 34987\n```\n\nAnd returns the variable `job_id = 34987`, which can be used for setting dependencies on subsequent jobs\n\n```python\nslurm_after = Slurm(dependency=dict(afterok=job_id)))\n```\n\n\n## Advanced Features\n\n### Command-Line Interface (CLI)\n\nFor simpler dispatch jobs, a command line entry point is also made available.\n\n```bash\nsimple_slurm [OPTIONS] \"COMMAND_TO_RUN_WITH_SBATCH\"\n```\n\nAs such, both of these `python` and `bash` calls are equivalent.\n\n```python\nslurm = Slurm(partition=\"compute.p\", output=\"slurm.log\", ignore_pbs=True)\nslurm.sbatch(\"echo \\$HOSTNAME\")\n```\n```bash\nsimple_slurm --partition=compute.p --output slurm.log --ignore_pbs \"echo \\$HOSTNAME\"\n```\n\n\n### Using Configuration Files\n\nLet's define the *static* components of a job definition in a YAML file `slurm_default.yml`\n\n```yaml\ncpus_per_task: 15\njob_name: \"name\"\noutput: \"%A_%a.out\"\n```\n\nIncluding these options with the using the `yaml` package is very *simple*\n\n```python\nimport yaml\n\nfrom simple_slurm import Slurm\n\nslurm = Slurm(**yaml.load(open(\"slurm_default.yml\", \"r\")))\n\n...\n\nslurm.set_array(range(NUMBER_OF_SIMULATIONS))\n```\n\nThe job can be updated according to the *dynamic* project needs (ex. `NUMBER_OF_SIMULATIONS`).\n\n\n\n\n### Filename Patterns and Environment Variables\n\nFor convenience, Filename Patterns and Output Environment Variables are available as attributes of the Simple Slurm object.\n\nSee [https://slurm.schedmd.com/sbatch.html](https://slurm.schedmd.com/sbatch.html#lbAH) for details on the commands.\n\n```python\nfrom slurm import Slurm\n\nslurm = Slurm(output=('{}_{}.out'.format(\n Slurm.JOB_ARRAY_MASTER_ID,\n Slurm.JOB_ARRAY_ID))\nslurm.sbatch('python demo.py ' + slurm.SLURM_ARRAY_JOB_ID)\n```\n\nThis example would result in output files of the form `65541_15.out`.\nHere the job submission ID is `65541`, and this output file corresponds to the submission number `15` in the job array. Moreover, this index is passed to the Python code `demo.py` as an argument.\n\n\n`sbatch` allows for a filename pattern to contain one or more replacement symbols. They can be accessed with `Slurm.<name>`\n\nname | value | description\n:-------------------|------:|:-----------\nJOB_ARRAY_MASTER_ID | %A | job array's master job allocation number\nJOB_ARRAY_ID | %a | job array id (index) number\nJOB_ID_STEP_ID | %J | jobid.stepid of the running job. (e.g. \"128.0\")\nJOB_ID | %j | jobid of the running job\nHOSTNAME | %N | short hostname. this will create a separate io file per node\nNODE_IDENTIFIER | %n | node identifier relative to current job (e.g. \"0\" is the first node of the running job) this will create a separate io file per node\nSTEP_ID | %s | stepid of the running job\nTASK_IDENTIFIER | %t | task identifier (rank) relative to current job. this will create a separate io file per task\nUSER_NAME | %u | user name\nJOB_NAME | %x | job name\nPERCENTAGE | %% | the character \"%\"\nDO_NOT_PROCESS | \\\\\\\\ | do not process any of the replacement symbols\n\n\n\nThe Slurm controller will set the following variables in the environment of the batch script. They can be accessed with `Slurm.<name>`.\n\nname | description\n:----------------------|:-----------\nSLURM_ARRAY_TASK_COUNT | total number of tasks in a job array\nSLURM_ARRAY_TASK_ID | job array id (index) number\nSLURM_ARRAY_TASK_MAX | job array's maximum id (index) number\nSLURM_ARRAY_TASK_MIN | job array's minimum id (index) number\nSLURM_ARRAY_TASK_STEP | job array's index step size\nSLURM_ARRAY_JOB_ID | job array's master job id number\n... | ...\n\n\n## Job Management\n\nSimple Slurm provides a simple interface to Slurm's job management tools (`squeue` and `scance`l) to let you monitor and control running jobs.\n\n### Monitoring Jobs with `squeue`\n\nRetrieve and display job information for the current user:\n\n```python\nfrom simple_slurm import Slurm\n\nslurm = Slurm()\nslurm.squeue.update() # Fetch latest job data\n\n# Get the jobs as a dictionary\njobs = slurm.squeue.jobs\n\nfor job_id, job in jobs.items():\n print(job)\n```\n\n\n### Canceling Jobs with `scancel`\n\nCancel single jobs or entire job arrays:\n\n```python\nfrom simple_slurm import Slurm\n\nslurm = Slurm()\n\n# Cancel a specific job\nslurm.scancel.cancel_job(34987)\n\n# Cancel multiple jobs\nfor job_id in [34987, 34988, 34989]:\n slurm.scancel.cancel_job(job_id)\n\n# Send SIGTERM before canceling (graceful termination)\nslurm.scancel.signal_job(34987)\nslurm.scancel.cancel_job(34987)\n```\n\n\n## Error Handling\nThe library does not raise specific exceptions for invalid Slurm arguments or job submission failures. Instead, it relies on the underlying Slurm commands (`sbatch`, `srun`, etc.) to handle errors. If a job submission fails, the error message from Slurm will be printed to the console.\n\nAdditionally, if invalid arguments are passed to the Slurm object, the library uses `argparse` to validate them. If an argument is invalid, `argparse` will raise an error and print a helpful message.\n\nFor example:\n\n```bash\nsimple_slurm --invalid_argument=value \"echo \\$HOSTNAME\"\n```\n\nThis will result in an error like:\n\n```bash\nusage: simple_slurm [OPTIONS] \"COMMAND_TO_RUN_WITH_SBATCH\"\nsimple_slurm: error: unrecognized arguments: --invalid_argument=value\n```\n\n\n## Project growth\n[](https://star-history.com/#amq92/simple_slurm&Date)\n",
"bugtrack_url": null,
"license": "GNU Affero General Public License v3",
"summary": "A simple Python wrapper for Slurm with flexibility in mind.",
"version": "0.3.5",
"project_urls": {
"Homepage": "https://github.com/amq92/simple_slurm"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e6f3f9e247faca74934168674430f6ecb871df3958f63b8dab23638056c4ac03",
"md5": "bfb1d046678f2c7023c2c1ac0f5237bf",
"sha256": "b6635407096de727bbc2d97b85037625305a80b09300fec24a2a3f233836671a"
},
"downloads": -1,
"filename": "simple_slurm-0.3.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bfb1d046678f2c7023c2c1ac0f5237bf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 29980,
"upload_time": "2025-02-01T02:36:15",
"upload_time_iso_8601": "2025-02-01T02:36:15.592021Z",
"url": "https://files.pythonhosted.org/packages/e6/f3/f9e247faca74934168674430f6ecb871df3958f63b8dab23638056c4ac03/simple_slurm-0.3.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "cc57dc898cd6ea39407cff09e89015a2bae1e42579da98a585cd3c6b796ec8d0",
"md5": "d27825160c967242364136e300401966",
"sha256": "a85c7f8cca25ece364b6ed19c0bfb9e194374cfb2ffc157f0d5811618e957458"
},
"downloads": -1,
"filename": "simple_slurm-0.3.5.tar.gz",
"has_sig": false,
"md5_digest": "d27825160c967242364136e300401966",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 30936,
"upload_time": "2025-02-01T02:36:17",
"upload_time_iso_8601": "2025-02-01T02:36:17.314644Z",
"url": "https://files.pythonhosted.org/packages/cc/57/dc898cd6ea39407cff09e89015a2bae1e42579da98a585cd3c6b796ec8d0/simple_slurm-0.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-01 02:36:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "amq92",
"github_project": "simple_slurm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "simple-slurm"
}
JSON
