==================
SLURM Helper Utils
==================
Collection of Python scripts for SLURM related tasks.
The following exported scripts are available:
- create-slurm-script
- validate-slurm-script
These scripts expect that the configuration files are present in the current working directory.
Recommended steps:
1. Create a directory `conf` e.g.: `mkdir conf`.
2. Copy the `config.yaml` from your virtual environment into that directory e.g.: `cp venv/lib/python3.10/site-packages/slurm_helper_utils/conf/config*.yaml conf/.`.
3. Edit the configuration files per your needs.
The following build configuration file content will provide the qualified options that the user will interactively select.
.. code-block:: shell
cat -n conf/config_builder.yaml
1 ---
2 required_directives:
3 job-name:
4 max_length: 20
5 min_length: 5
6 nodes:
7 qualified_options:
8 - 1
9 - 2
10 - 3
11 - 4
12 qos:
13 qualified_options:
14 - qos-A
15 - qos-B
16 gres:
17 qualified_options:
18 - gres-1
19 - gres-2
20 export:
21 qualified_options:
22 - NONE
23 propagate:
24 qualified_options:
25 - ALL
26 - NONE
27 - AS
28 - CORE
29 - CPU
30 - DATA
31 - FSIZE
32 - MEMLOCK
33 - NOFILE
34 - NPROC
35 - RSS
36 - STACK
37 mail-user:
38 qualified_options:
39 - jay@gmail.com
40 - sun@gmail.com
41 mail-type:
42 qualified_options:
43 - NONE
44 - BEGIN
45 - END
46 - FAIL
47 - REQUEUE
48 - ALL
49 - INVALID_DEPEND
50 - STAGE_OUT
51 - TIME_LIMIT
52 - TIME_LIMIT_90
53 - TIME_LIMIT_80
54 - TIME_LIMIT_50
55 - ARRAY_TASKS
56 output:
57 desc: The STDOUT from the SLURM sbatch script
58 error:
59 desc: The STDERR from the SLURM sbatch script
60 partition:
61 qualified_options:
62 - partition-1
63 - partition-22
64 ntasks-per-node:
65 desc: TBD
66 cpus-per-task:
67 desc: TBD
68 time:
69 desc: TBD
Invoke the exported console script that will allow user to interactively create a SLURM sbatch script:
.. code-block:: shell
create-slurm-script --config_file conf/config_builder.yaml
--outdir was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'
Created output directory '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'
--logfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log'
--outfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'
There are '14' required directives
Please enter the option for directive 'nodes':
options: ['1', '2', '3', '4']
Enter nodes: 2
Please enter the option for directive 'qos':
options: ['qos-A', 'qos-B']
Enter qos: qos-A
Please enter the option for directive 'gres':
options: ['gres-1', 'gres-2']
Enter gres: gres-1
Please enter the option for directive 'export':
options: ['NONE']
Enter export: NONE
Please enter the option for directive 'propagate':
options: ['ALL', 'NONE', 'AS', 'CORE', 'CPU', 'DATA', 'FSIZE', 'MEMLOCK', 'NOFILE', 'NPROC', 'RSS', 'STACK']
Enter propagate: ALL
Please enter the option for directive 'mail-user':
options: ['jay@gmail.com', 'sun@gmail.com']
Enter mail-user: jay@gmail.com
Please enter the option for directive 'mail-type':
options: ['NONE', 'BEGIN', 'END', 'FAIL', 'REQUEUE', 'ALL', 'INVALID_DEPEND', 'STAGE_OUT', 'TIME_LIMIT', 'TIME_LIMIT_90', 'TIME_LIMIT_80', 'TIME_LIMIT_50', 'ARRAY_TASKS']
Enter mail-type: ALL
Please enter the option for directive 'partition':
options: ['partition-1', 'partition-22']
Enter partition: partition-22
Wrote SLURM sbatch script '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'
Contents of the generated SLURM sbatch script:
.. code-block:: shell
cat -n /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh
1 #!/bin/bash
2
3 ## method-created: /tmp/test-slurm-helper-utils/venv/lib/python3.10/site-packages/slurm_helper_utils/create_slurm_script.py
4 ## date-created: 2025-02-15-152508
5 ## created-by: sundaram
6 ## config-file: conf/config_builder.yaml
7 ## logfile: /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log
8
9
10 #SBATCH --nodes=2
11 #SBATCH --qos=qos-A
12 #SBATCH --gres=gres-1
13 #SBATCH --export=NONE
14 #SBATCH --propagate=ALL
15 #SBATCH --mail-user=jay@gmail.com
16 #SBATCH --mail-type=ALL
17 #SBATCH --partition=partition-22
=======
History
=======
0.1.0 (2024-01-09)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/jai-python3/slurm-helper-utils",
"name": "slurm-helper-utils",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "slurm_helper_utils",
"author": "Jaideep Sundaram",
"author_email": "jai.python3@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c1/da/1ed80efbf2fd09dd8605e39d4bf64ae05f0525c348d1e09664b175bddb80/slurm_helper_utils-0.2.1.tar.gz",
"platform": null,
"description": "==================\nSLURM Helper Utils\n==================\n\nCollection of Python scripts for SLURM related tasks.\n\nThe following exported scripts are available:\n- create-slurm-script\n- validate-slurm-script\n\nThese scripts expect that the configuration files are present in the current working directory.\nRecommended steps:\n1. Create a directory `conf` e.g.: `mkdir conf`.\n2. Copy the `config.yaml` from your virtual environment into that directory e.g.: `cp venv/lib/python3.10/site-packages/slurm_helper_utils/conf/config*.yaml conf/.`.\n3. Edit the configuration files per your needs.\n\nThe following build configuration file content will provide the qualified options that the user will interactively select.\n\n.. code-block:: shell\n\n cat -n conf/config_builder.yaml\n 1 ---\n 2 required_directives:\n 3 job-name:\n 4 max_length: 20\n 5 min_length: 5\n 6 nodes:\n 7 qualified_options:\n 8 - 1\n 9 - 2\n 10 - 3\n 11 - 4\n 12 qos:\n 13 qualified_options:\n 14 - qos-A\n 15 - qos-B\n 16 gres:\n 17 qualified_options:\n 18 - gres-1\n 19 - gres-2\n 20 export:\n 21 qualified_options:\n 22 - NONE\n 23 propagate:\n 24 qualified_options:\n 25 - ALL\n 26 - NONE\n 27 - AS\n 28 - CORE\n 29 - CPU\n 30 - DATA\n 31 - FSIZE\n 32 - MEMLOCK\n 33 - NOFILE\n 34 - NPROC\n 35 - RSS\n 36 - STACK\n 37 mail-user:\n 38 qualified_options:\n 39 - jay@gmail.com\n 40 - sun@gmail.com\n 41 mail-type:\n 42 qualified_options:\n 43 - NONE\n 44 - BEGIN\n 45 - END\n 46 - FAIL\n 47 - REQUEUE\n 48 - ALL\n 49 - INVALID_DEPEND\n 50 - STAGE_OUT\n 51 - TIME_LIMIT\n 52 - TIME_LIMIT_90\n 53 - TIME_LIMIT_80\n 54 - TIME_LIMIT_50\n 55 - ARRAY_TASKS\n 56 output:\n 57 desc: The STDOUT from the SLURM sbatch script\n 58 error:\n 59 desc: The STDERR from the SLURM sbatch script\n 60 partition:\n 61 qualified_options:\n 62 - partition-1\n 63 - partition-22\n 64 ntasks-per-node:\n 65 desc: TBD\n 66 cpus-per-task:\n 67 desc: TBD\n 68 time:\n 69 desc: TBD\n\nInvoke the exported console script that will allow user to interactively create a SLURM sbatch script:\n\n.. code-block:: shell\n\n create-slurm-script --config_file conf/config_builder.yaml \n --outdir was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'\n Created output directory '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'\n --logfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log'\n --outfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'\n There are '14' required directives\n Please enter the option for directive 'nodes':\n options: ['1', '2', '3', '4']\n Enter nodes: 2\n Please enter the option for directive 'qos':\n options: ['qos-A', 'qos-B']\n Enter qos: qos-A\n Please enter the option for directive 'gres':\n options: ['gres-1', 'gres-2']\n Enter gres: gres-1\n Please enter the option for directive 'export':\n options: ['NONE']\n Enter export: NONE\n Please enter the option for directive 'propagate':\n options: ['ALL', 'NONE', 'AS', 'CORE', 'CPU', 'DATA', 'FSIZE', 'MEMLOCK', 'NOFILE', 'NPROC', 'RSS', 'STACK']\n Enter propagate: ALL\n Please enter the option for directive 'mail-user':\n options: ['jay@gmail.com', 'sun@gmail.com']\n Enter mail-user: jay@gmail.com\n Please enter the option for directive 'mail-type':\n options: ['NONE', 'BEGIN', 'END', 'FAIL', 'REQUEUE', 'ALL', 'INVALID_DEPEND', 'STAGE_OUT', 'TIME_LIMIT', 'TIME_LIMIT_90', 'TIME_LIMIT_80', 'TIME_LIMIT_50', 'ARRAY_TASKS']\n Enter mail-type: ALL\n Please enter the option for directive 'partition':\n options: ['partition-1', 'partition-22']\n Enter partition: partition-22\n Wrote SLURM sbatch script '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'\n\nContents of the generated SLURM sbatch script:\n\n.. code-block:: shell\n\n cat -n /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh \n 1 #!/bin/bash\n 2\n 3 ## method-created: /tmp/test-slurm-helper-utils/venv/lib/python3.10/site-packages/slurm_helper_utils/create_slurm_script.py\n 4 ## date-created: 2025-02-15-152508\n 5 ## created-by: sundaram\n 6 ## config-file: conf/config_builder.yaml\n 7 ## logfile: /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log\n 8\n 9\n 10 #SBATCH --nodes=2\n 11 #SBATCH --qos=qos-A\n 12 #SBATCH --gres=gres-1\n 13 #SBATCH --export=NONE\n 14 #SBATCH --propagate=ALL\n 15 #SBATCH --mail-user=jay@gmail.com\n 16 #SBATCH --mail-type=ALL\n 17 #SBATCH --partition=partition-22\n\n\n=======\nHistory\n=======\n\n0.1.0 (2024-01-09)\n------------------\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": null,
"summary": "Collection of Python scripts for SLURM related tasks.",
"version": "0.2.1",
"project_urls": {
"Homepage": "https://github.com/jai-python3/slurm-helper-utils"
},
"split_keywords": [
"slurm_helper_utils"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "80fe341c4a02f2e0cd37c35f1ce11ab8ca2c634280b16adb9dcfc634a82d79c9",
"md5": "822d297426c0d32c51b9fe2981ccafda",
"sha256": "b0acbc714d7976300c3d25c59c3cc29218247872d219abe411cc1bf50b157249"
},
"downloads": -1,
"filename": "slurm_helper_utils-0.2.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "822d297426c0d32c51b9fe2981ccafda",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.10",
"size": 29044,
"upload_time": "2025-02-15T21:20:42",
"upload_time_iso_8601": "2025-02-15T21:20:42.159266Z",
"url": "https://files.pythonhosted.org/packages/80/fe/341c4a02f2e0cd37c35f1ce11ab8ca2c634280b16adb9dcfc634a82d79c9/slurm_helper_utils-0.2.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c1da1ed80efbf2fd09dd8605e39d4bf64ae05f0525c348d1e09664b175bddb80",
"md5": "801b8eaf64da4747f7f945c5ca2543fb",
"sha256": "8c410afb0e64ef0b2135f38902fdd49d858900ea2f860b903d9a86fd87837aaf"
},
"downloads": -1,
"filename": "slurm_helper_utils-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "801b8eaf64da4747f7f945c5ca2543fb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 33607,
"upload_time": "2025-02-15T21:20:44",
"upload_time_iso_8601": "2025-02-15T21:20:44.034016Z",
"url": "https://files.pythonhosted.org/packages/c1/da/1ed80efbf2fd09dd8605e39d4bf64ae05f0525c348d1e09664b175bddb80/slurm_helper_utils-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-15 21:20:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jai-python3",
"github_project": "slurm-helper-utils",
"github_not_found": true,
"lcname": "slurm-helper-utils"
}
JSON
