# slurmutils
[](https://matrix.to/#/#hpc:ubuntu.com)
Utilities and APIs for interfacing with the Slurm workload manager.
slurmutils is a collection of various utilities that make it easier
for you and your friends to interface with the Slurm workload manager, especially if you
are orchestrating deployments of new and current Slurm clusters. Gone are the days of
seething over incomplete Jinja2 templates. Current utilities shipped in the
slurmutils package include:
#### `from slurmutils.editors import ...`
* `acctgatherconfig`: An editor for _acct_gather.conf_ configuration files.
* `gresconfig`: An editor for _gres.conf_ configuration files.
* `cgroupconfig`: An editor for _cgroup.conf_ configuration files.
* `slurmconfig`: An editor for _slurm.conf_ configuration files.
* `slurmdbdconfig`: An editor for _slurmdbd.conf_ configuration files.
For more information on how to use or contribute to slurmutils,
check out the [Getting Started](#-getting-started) and [Development](#-development)
sections below 👇
## ✨ Getting Started
### Installation
#### Option 1: Install from PyPI
```shell
$ python3 -m pip install slurmutils
```
#### Option 2: Install from source
We use the [Poetry](https://python-poetry.org) packaging and dependency manager to
manage this project. It must be installed on your system if installing slurmutils
from source.
```shell
$ git clone https://github.com/canonical/slurmutils.git
$ cd slurmutils
$ poetry install
```
### Usage
#### `slurmutils.editors`
This module provides an API for editing files, and creating new files if they do not
exist. Here's some operations you can perform on files using the editors in this module:
##### `acctgatherconfig`
###### Edit a pre-existing _acct_gather.conf_ configuration file
```python
from slurmutils.editors import acctgatherconfig
with acctgatherconfig.edit("/etc/slurm/acct_gather.conf") as config:
config.profile_influx_db_database = "test_acct_gather_db"
config.profile_influx_db_default = ["NONE"]
config.profile_influx_db_host = "testhostname1"
config.profile_influx_db_pass = "testpassword1"
config.profile_influx_dbrt_policy = "testpolicy1"
config.profile_influx_db_user = "testuser1"
config.profile_influx_db_timeout = "20"
```
##### `cgroupconfig`
###### Edit a pre-existing _cgroup.conf_ configuration file
```python
from slurmutils.editors import cgroupconfig
with cgroupconfig.edit("/etc/slurm/cgroup.conf") as config:
config.constrain_cores = "yes"
config.constrain_devices = "yes"
config.constrain_ram_space = "yes"
config.constrain_swap_space = "yes"
```
##### `gresconfig`
###### Edit a pre-existing _gres.conf_ configuration file
```python
from slurmutils.editors import gresconfig
from slurmutils.models import GRESName, GRESNode
with gresconfig.edit("/etc/slurm/gres.conf") as config:
name = GRESName(
Name="gpu",
Type="epyc",
File="/dev/amd4",
Cores=["0", "1"],
)
node = GRESNode(
NodeName="juju-abc654-[1-20]",
Name="gpu",
Type="epyc",
File="/dev/amd[0-3]",
Count="12G",
)
config.auto_detect = "rsmi"
config.names.append(name.dict())
config.nodes.updaten(node.dict())
```
##### `slurmconfig`
###### Edit a pre-existing _slurm.conf_ configuration file
```python
from slurmutils.editors import slurmconfig
# Open, edit, and save the slurm.conf file located at _/etc/slurm/slurm.conf_.
with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
del config.inactive_limit
config.max_job_count = 20000
config.proctrack_type = "proctrack/linuxproc"
```
###### Add a new node to the _slurm.conf_ file
```python
from slurmutils.editors import slurmconfig
from slurmutils.models import Node
with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
node = Node(
NodeName="batch-[0-25]",
NodeAddr="12.34.56.78",
CPUs=1,
RealMemory=1000,
TmpDisk=10000,
)
config.nodes.update(node.dict())
```
##### `slurmdbdconfig`
###### Edit a pre-existing _slurmdbd.conf_ configuration file
```python
from slurmutils.editors import slurmdbdconfig
with slurmdbdconfig.edit("/etc/slurm/slurmdbd.conf") as config:
config.archive_usage = "yes"
config.log_file = "/var/spool/slurmdbd.log"
config.debug_flags = ["DB_EVENT", "DB_JOB", "DB_USAGE"]
del config.auth_alt_types
del config.auth_alt_parameters
```
## 🤔 What's next?
If you want to learn more about all the things you can do with slurmutils,
here are some further resources for you to explore:
* [Open an issue](https://github.com/charmed-hpc/slurmutils/issues/new?title=ISSUE+TITLE&body=*Please+describe+your+issue*)
* [Ask a question on Github](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)
## 🛠️ Development
This project uses [tox](https://tox.wiki) as its command runner, which provides
some useful commands that will help you while hacking on slurmutils:
```shell
tox run -e fmt # Apply formatting standards to code.
tox run -e lint # Check code against coding style standards.
tox run -e unit # Run unit tests.
```
If you're interested in contributing your work to slurmutils,
take a look at our [contributing guidelines](./CONTRIBUTING.md) for further details.
## 🤝 Project and community
slurmutils is a project of the [Ubuntu High-Performance Computing community](https://ubuntu.com/community/governance/teams/hpc).
Interested in contributing bug fixes, new editors, documentation, or feedback? Want to join the Ubuntu HPC community? You’ve come to the right place 🤩
Here’s some links to help you get started with joining the community:
* [Ubuntu Code of Conduct](https://ubuntu.com/community/ethos/code-of-conduct)
* [Contributing guidelines](./CONTRIBUTING.md)
* [Join the conversation on Matrix](https://matrix.to/#/#hpc:ubuntu.com)
* [Get the latest news on Discourse](https://discourse.ubuntu.com/c/hpc/151)
* [Ask and answer questions on GitHub](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)
## 📋 License
slurmutils is free software, distributed under the GNU Lesser General Public License, v3.0.
See the [LGPL-3.0 LICENSE](./LICENSE) file for further details.
Raw data
{
"_id": null,
"home_page": "https://github.com/charmed-hpc/slurmutils",
"name": "slurmutils",
"maintainer": "Jason C. Nucciarone",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "nuccitheboss@ubuntu.com",
"keywords": "HPC, administration, orchestration, utility",
"author": "Jason C. Nucciarone",
"author_email": "nuccitheboss@ubuntu.com",
"download_url": "https://files.pythonhosted.org/packages/7d/52/9c551686fc6523340f62f2bbfe6be8b0252bb31591fdcfd3278b04ee7586/slurmutils-0.10.0.tar.gz",
"platform": null,
"description": "# slurmutils\n\n\n\n\n[](https://matrix.to/#/#hpc:ubuntu.com)\n\nUtilities and APIs for interfacing with the Slurm workload manager.\n\nslurmutils is a collection of various utilities that make it easier \nfor you and your friends to interface with the Slurm workload manager, especially if you \nare orchestrating deployments of new and current Slurm clusters. Gone are the days of\nseething over incomplete Jinja2 templates. Current utilities shipped in the \nslurmutils package include:\n\n#### `from slurmutils.editors import ...`\n\n* `acctgatherconfig`: An editor for _acct_gather.conf_ configuration files.\n* `gresconfig`: An editor for _gres.conf_ configuration files.\n* `cgroupconfig`: An editor for _cgroup.conf_ configuration files.\n* `slurmconfig`: An editor for _slurm.conf_ configuration files.\n* `slurmdbdconfig`: An editor for _slurmdbd.conf_ configuration files.\n\nFor more information on how to use or contribute to slurmutils, \ncheck out the [Getting Started](#-getting-started) and [Development](#-development) \nsections below \ud83d\udc47\n\n## \u2728 Getting Started\n\n### Installation\n\n#### Option 1: Install from PyPI\n\n```shell\n$ python3 -m pip install slurmutils\n```\n\n#### Option 2: Install from source\n\nWe use the [Poetry](https://python-poetry.org) packaging and dependency manager to\nmanage this project. It must be installed on your system if installing slurmutils\nfrom source.\n\n```shell\n$ git clone https://github.com/canonical/slurmutils.git\n$ cd slurmutils\n$ poetry install\n```\n\n### Usage\n\n#### `slurmutils.editors`\n\nThis module provides an API for editing files, and creating new files if they do not\nexist. Here's some operations you can perform on files using the editors in this module:\n\n##### `acctgatherconfig`\n\n###### Edit a pre-existing _acct_gather.conf_ configuration file\n\n```python\nfrom slurmutils.editors import acctgatherconfig\n\nwith acctgatherconfig.edit(\"/etc/slurm/acct_gather.conf\") as config:\n config.profile_influx_db_database = \"test_acct_gather_db\"\n config.profile_influx_db_default = [\"NONE\"]\n config.profile_influx_db_host = \"testhostname1\"\n config.profile_influx_db_pass = \"testpassword1\"\n config.profile_influx_dbrt_policy = \"testpolicy1\"\n config.profile_influx_db_user = \"testuser1\"\n config.profile_influx_db_timeout = \"20\"\n```\n\n##### `cgroupconfig`\n\n###### Edit a pre-existing _cgroup.conf_ configuration file\n\n```python\nfrom slurmutils.editors import cgroupconfig\n\nwith cgroupconfig.edit(\"/etc/slurm/cgroup.conf\") as config:\n config.constrain_cores = \"yes\"\n config.constrain_devices = \"yes\"\n config.constrain_ram_space = \"yes\"\n config.constrain_swap_space = \"yes\"\n```\n\n##### `gresconfig`\n\n###### Edit a pre-existing _gres.conf_ configuration file\n\n```python\nfrom slurmutils.editors import gresconfig\nfrom slurmutils.models import GRESName, GRESNode\n\nwith gresconfig.edit(\"/etc/slurm/gres.conf\") as config:\n name = GRESName(\n Name=\"gpu\",\n Type=\"epyc\",\n File=\"/dev/amd4\",\n Cores=[\"0\", \"1\"],\n )\n node = GRESNode(\n NodeName=\"juju-abc654-[1-20]\",\n Name=\"gpu\",\n Type=\"epyc\",\n File=\"/dev/amd[0-3]\",\n Count=\"12G\",\n )\n config.auto_detect = \"rsmi\"\n config.names.append(name.dict())\n config.nodes.updaten(node.dict())\n```\n\n##### `slurmconfig`\n\n###### Edit a pre-existing _slurm.conf_ configuration file\n\n```python\nfrom slurmutils.editors import slurmconfig\n\n# Open, edit, and save the slurm.conf file located at _/etc/slurm/slurm.conf_.\nwith slurmconfig.edit(\"/etc/slurm/slurm.conf\") as config:\n del config.inactive_limit\n config.max_job_count = 20000\n config.proctrack_type = \"proctrack/linuxproc\"\n```\n\n###### Add a new node to the _slurm.conf_ file\n\n```python\nfrom slurmutils.editors import slurmconfig\nfrom slurmutils.models import Node\n\nwith slurmconfig.edit(\"/etc/slurm/slurm.conf\") as config:\n node = Node(\n NodeName=\"batch-[0-25]\", \n NodeAddr=\"12.34.56.78\", \n CPUs=1, \n RealMemory=1000, \n TmpDisk=10000,\n )\n config.nodes.update(node.dict())\n```\n\n##### `slurmdbdconfig`\n\n###### Edit a pre-existing _slurmdbd.conf_ configuration file\n\n```python\nfrom slurmutils.editors import slurmdbdconfig\n\nwith slurmdbdconfig.edit(\"/etc/slurm/slurmdbd.conf\") as config:\n config.archive_usage = \"yes\"\n config.log_file = \"/var/spool/slurmdbd.log\"\n config.debug_flags = [\"DB_EVENT\", \"DB_JOB\", \"DB_USAGE\"]\n del config.auth_alt_types\n del config.auth_alt_parameters\n```\n\n## \ud83e\udd14 What's next?\n\nIf you want to learn more about all the things you can do with slurmutils, \nhere are some further resources for you to explore:\n\n* [Open an issue](https://github.com/charmed-hpc/slurmutils/issues/new?title=ISSUE+TITLE&body=*Please+describe+your+issue*)\n* [Ask a question on Github](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)\n\n## \ud83d\udee0\ufe0f Development\n\nThis project uses [tox](https://tox.wiki) as its command runner, which provides \nsome useful commands that will help you while hacking on slurmutils:\n\n```shell\ntox run -e fmt # Apply formatting standards to code.\ntox run -e lint # Check code against coding style standards.\ntox run -e unit # Run unit tests.\n```\n\nIf you're interested in contributing your work to slurmutils, \ntake a look at our [contributing guidelines](./CONTRIBUTING.md) for further details.\n\n## \ud83e\udd1d Project and community\n\nslurmutils is a project of the [Ubuntu High-Performance Computing community](https://ubuntu.com/community/governance/teams/hpc).\nInterested in contributing bug fixes, new editors, documentation, or feedback? Want to join the Ubuntu HPC community? You\u2019ve come to the right place \ud83e\udd29\n\nHere\u2019s some links to help you get started with joining the community:\n\n* [Ubuntu Code of Conduct](https://ubuntu.com/community/ethos/code-of-conduct)\n* [Contributing guidelines](./CONTRIBUTING.md)\n* [Join the conversation on Matrix](https://matrix.to/#/#hpc:ubuntu.com)\n* [Get the latest news on Discourse](https://discourse.ubuntu.com/c/hpc/151)\n* [Ask and answer questions on GitHub](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)\n\n## \ud83d\udccb License\n\nslurmutils is free software, distributed under the GNU Lesser General Public License, v3.0.\nSee the [LGPL-3.0 LICENSE](./LICENSE) file for further details.\n",
"bugtrack_url": null,
"license": "LGPL-3.0-only",
"summary": "Utilities and APIs for interfacing with the Slurm workload manager.",
"version": "0.10.0",
"project_urls": {
"Bug Tracker": "https://github.com/charmed-hpc/slurmutils/issues",
"Homepage": "https://github.com/charmed-hpc/slurmutils",
"Repository": "https://github.com/charmed-hpc/slurmutils"
},
"split_keywords": [
"hpc",
" administration",
" orchestration",
" utility"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6d8d554dbceab3b9f6ad03eb410948cc804ce0a5aad21db76242ebe1cb7fcc3a",
"md5": "4d1267f6722d149385158ba56e7a2e1f",
"sha256": "212a426ad6e3dee9e7fb4d8241129b3a36002e5ff8463978b4998b01f73399e0"
},
"downloads": -1,
"filename": "slurmutils-0.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4d1267f6722d149385158ba56e7a2e1f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 32224,
"upload_time": "2024-12-11T14:26:24",
"upload_time_iso_8601": "2024-12-11T14:26:24.377313Z",
"url": "https://files.pythonhosted.org/packages/6d/8d/554dbceab3b9f6ad03eb410948cc804ce0a5aad21db76242ebe1cb7fcc3a/slurmutils-0.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7d529c551686fc6523340f62f2bbfe6be8b0252bb31591fdcfd3278b04ee7586",
"md5": "ccd802f798b6dfe96fb94e7378cde751",
"sha256": "338c1e02637629444bb86ca039bf71e27255229905a5ab79c588cef6c3ebdeff"
},
"downloads": -1,
"filename": "slurmutils-0.10.0.tar.gz",
"has_sig": false,
"md5_digest": "ccd802f798b6dfe96fb94e7378cde751",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 21081,
"upload_time": "2024-12-11T14:26:26",
"upload_time_iso_8601": "2024-12-11T14:26:26.572482Z",
"url": "https://files.pythonhosted.org/packages/7d/52/9c551686fc6523340f62f2bbfe6be8b0252bb31591fdcfd3278b04ee7586/slurmutils-0.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-11 14:26:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "charmed-hpc",
"github_project": "slurmutils",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "slurmutils"
}
JSON
