# Hydra Slurm Rich Launcher
<p align="center">
<img src="https://github.com/creinders/hydra-slurm-rich-launcher/assets/8905380/b34924df-7b94-4e20-bf54-7592e432ab74">
</p>
> A rich, visual interface for easily starting and monitoring your [Hydra](https://hydra.cc) applications on [SLURM](https://slurm.schedmd.com/documentation.html) clusters.
- **Ease of Use**: Streamline your workflow with a simplified process for submitting jobs to SLURM
- **Rich Visualization**: A clear and beautiful visual overview of your jobs
- **Integration**: Seamlessly integrates with Hydra-powered CLIs
- **Real-Time Updates**: Monitor the status of your jobs in real-time
## Installation
The Hydra Slurm Rich Launcher can be installed via pip:
```
pip install hydra-slurm-rich-launcher --upgrade
```
<details>
<summary>Alternative installation methods</summary>
### Locally
```
git clone git@github.com:creinders/hydra-slurm-rich-launcher.git
cd hydra-slurm-rich-launcher
poetry install
```
</details>
## Quick Start
Define your configuration in `config.yaml`:
```yaml
defaults:
- override hydra/launcher: slurm_rich
hydra:
launcher:
partition: <SLURM_PARTITION>
task: 1
```
Implement your Hydra app in `my_app.py`:
```python
import hydra
@hydra.main(config_path=".", config_name="config", version_base="1.3")
def my_app(cfg) -> None:
print(f"Task: {cfg.task}")
if __name__ == "__main__":
my_app()
```
Starting the app with `task=1,2,4` will launch three jobs with different configurations:
```bash
python my_app.py task=1,2,4 --multirun
```
Please see the [Hydra documentation](https://hydra.cc/docs/intro/) for details regarding the configuration and multi-run.
## Scalability
Lots of run? No problem! Hydra Slurm Rich Launcher smartly organizes all of your runs.
## Restarts
Easily monitor the status of your jobs and swiftly restart any failed runs.
## Parameters
The Hydra Slurm Rich Launcher has the following parameters.
```yaml
slurm_query_interval_s: 15 # Query update interval from SLURM controller
filter_job_ids: null # Filter specific jobs from the job array, separated by comma (e.g., "1,4"), that should not be executed
retry_strategy: 'prompt' # Defines job retry strategy. 'prompt': will ask the user, 'never': never restarts, and 'always': restarts the runs automatically
max_retries: 3 # Maximum retry attempts
le_mode: 'auto' # Low energy mode settings. The low energy mode disables all animations and can be turned on if the cpu-usage must be minimized. Values are: 'on', 'off', and 'auto'. 'auto' will turn on the low energy mode if the environment variable HYDRA_SLURM_PROGRESS_LE_MODE is set.
submitit_folder: ${hydra.sweep.dir}/.submitit/%j
timeout_min: 60
cpus_per_task: null
gpus_per_node: null
tasks_per_node: 1
mem_gb: null
nodes: 1
name: ${hydra.job.name}
partition: null
qos: null
comment: null
constraint: null
exclude: null
gres: null
cpus_per_gpu: null
gpus_per_task: null
mem_per_gpu: null
mem_per_cpu: null
account: null
signal_delay_s: 120
max_num_timeout: 0
additional_parameters: {}
array_parallelism: 256
setup: null
```
## License
Hydra Slurm Rich Launcher is licensed under [MIT License](./LICENSE).
## Credits
This package was inspired by and extends the capabilities of the `hydra-submitit-launcher`. We gratefully acknowledge the developers of [hydra-submitit-launcher](https://hydra.cc/docs/plugins/submitit_launcher/) and [Hydra](https://hydra.cc) for their contributions to the open-source community.
Raw data
{
"_id": null,
"home_page": "https://github.com/creinders/hydra-slurm-rich-launcher",
"name": "hydra-slurm-rich-launcher",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "hydra, slurm, rich, launcher, command-line, cli, scheduler, monitoring, visualization",
"author": "Christoph Reinders",
"author_email": "reinders@tnt.uni-hannover.de",
"download_url": "https://files.pythonhosted.org/packages/8f/82/28bd1962b8841d6d86adbff07dfef673d545c534ee5207ad0cce52e3d2ee/hydra_slurm_rich_launcher-1.1.0.tar.gz",
"platform": null,
"description": "# Hydra Slurm Rich Launcher\n\n<p align=\"center\">\n <img src=\"https://github.com/creinders/hydra-slurm-rich-launcher/assets/8905380/b34924df-7b94-4e20-bf54-7592e432ab74\">\n</p>\n\n> A rich, visual interface for easily starting and monitoring your [Hydra](https://hydra.cc) applications on [SLURM](https://slurm.schedmd.com/documentation.html) clusters.\n\n- **Ease of Use**: Streamline your workflow with a simplified process for submitting jobs to SLURM\n- **Rich Visualization**: A clear and beautiful visual overview of your jobs\n- **Integration**: Seamlessly integrates with Hydra-powered CLIs\n- **Real-Time Updates**: Monitor the status of your jobs in real-time\n\n## Installation\n\nThe Hydra Slurm Rich Launcher can be installed via pip:\n```\npip install hydra-slurm-rich-launcher --upgrade\n```\n<details>\n <summary>Alternative installation methods</summary>\n \n ### Locally\n ```\n git clone git@github.com:creinders/hydra-slurm-rich-launcher.git\n cd hydra-slurm-rich-launcher\n poetry install\n ```\n</details>\n\n## Quick Start\n\nDefine your configuration in `config.yaml`:\n\n```yaml\ndefaults:\n - override hydra/launcher: slurm_rich\nhydra:\n launcher:\n partition: <SLURM_PARTITION>\n\ntask: 1\n```\n\nImplement your Hydra app in `my_app.py`:\n```python\nimport hydra\n\n@hydra.main(config_path=\".\", config_name=\"config\", version_base=\"1.3\")\ndef my_app(cfg) -> None:\n print(f\"Task: {cfg.task}\")\n\nif __name__ == \"__main__\":\n my_app()\n```\n\nStarting the app with `task=1,2,4` will launch three jobs with different configurations:\n```bash\npython my_app.py task=1,2,4 --multirun\n```\n\n\nPlease see the [Hydra documentation](https://hydra.cc/docs/intro/) for details regarding the configuration and multi-run.\n\n## Scalability\n\nLots of run? No problem! Hydra Slurm Rich Launcher smartly organizes all of your runs.\n\n\n\n## Restarts\n\nEasily monitor the status of your jobs and swiftly restart any failed runs.\n\n\n\n## Parameters\n\nThe Hydra Slurm Rich Launcher has the following parameters.\n```yaml\nslurm_query_interval_s: 15 # Query update interval from SLURM controller\nfilter_job_ids: null # Filter specific jobs from the job array, separated by comma (e.g., \"1,4\"), that should not be executed\nretry_strategy: 'prompt' # Defines job retry strategy. 'prompt': will ask the user, 'never': never restarts, and 'always': restarts the runs automatically\nmax_retries: 3 # Maximum retry attempts\nle_mode: 'auto' # Low energy mode settings. The low energy mode disables all animations and can be turned on if the cpu-usage must be minimized. Values are: 'on', 'off', and 'auto'. 'auto' will turn on the low energy mode if the environment variable HYDRA_SLURM_PROGRESS_LE_MODE is set.\n\nsubmitit_folder: ${hydra.sweep.dir}/.submitit/%j\ntimeout_min: 60\ncpus_per_task: null\ngpus_per_node: null\ntasks_per_node: 1\nmem_gb: null\nnodes: 1\nname: ${hydra.job.name}\npartition: null\nqos: null\ncomment: null\nconstraint: null\nexclude: null\ngres: null\ncpus_per_gpu: null\ngpus_per_task: null\nmem_per_gpu: null\nmem_per_cpu: null\naccount: null\nsignal_delay_s: 120\nmax_num_timeout: 0\nadditional_parameters: {}\narray_parallelism: 256\nsetup: null\n```\n\n## License\n\nHydra Slurm Rich Launcher is licensed under [MIT License](./LICENSE).\n\n## Credits\n\nThis package was inspired by and extends the capabilities of the `hydra-submitit-launcher`. We gratefully acknowledge the developers of [hydra-submitit-launcher](https://hydra.cc/docs/plugins/submitit_launcher/) and [Hydra](https://hydra.cc) for their contributions to the open-source community.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A rich, visual interface for easily starting and monitoring your Hydra applications on SLURM clusters",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/creinders/hydra-slurm-rich-launcher"
},
"split_keywords": [
"hydra",
" slurm",
" rich",
" launcher",
" command-line",
" cli",
" scheduler",
" monitoring",
" visualization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "270a6f2b0d7c6dd68bcc61a99226cd293830c26ff8869d6a02b0999e4a3eafee",
"md5": "a8b63e3f516042bd0f4ecfca310ab2c5",
"sha256": "04120b7ed2fcff8469c054cbf2888b5cdf4cf693af8aad162c8178fe84f1af52"
},
"downloads": -1,
"filename": "hydra_slurm_rich_launcher-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a8b63e3f516042bd0f4ecfca310ab2c5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 14299,
"upload_time": "2024-07-01T14:24:24",
"upload_time_iso_8601": "2024-07-01T14:24:24.452425Z",
"url": "https://files.pythonhosted.org/packages/27/0a/6f2b0d7c6dd68bcc61a99226cd293830c26ff8869d6a02b0999e4a3eafee/hydra_slurm_rich_launcher-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8f8228bd1962b8841d6d86adbff07dfef673d545c534ee5207ad0cce52e3d2ee",
"md5": "f50efa9217f9590b79739bb902eb99b5",
"sha256": "ac32e03d82d5daa480117228a2897382b70180c90c003a01b884b8a6f812858e"
},
"downloads": -1,
"filename": "hydra_slurm_rich_launcher-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "f50efa9217f9590b79739bb902eb99b5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 13156,
"upload_time": "2024-07-01T14:24:26",
"upload_time_iso_8601": "2024-07-01T14:24:26.403150Z",
"url": "https://files.pythonhosted.org/packages/8f/82/28bd1962b8841d6d86adbff07dfef673d545c534ee5207ad0cce52e3d2ee/hydra_slurm_rich_launcher-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-01 14:24:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "creinders",
"github_project": "hydra-slurm-rich-launcher",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "hydra-slurm-rich-launcher"
}
JSON
