| Name | hydra-lsf-launcher JSON |
| Version |
0.1.2
JSON |
| download |
| home_page | None |
| Summary | Lsf Hydra Launcher plugin |
| upload_time | 2024-11-07 17:16:10 |
| maintainer | None |
| docs_url | None |
| author | Francesco Spinnato |
| requires_python | None |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# hydra-lsf-launcher
This is a simple launcher for launching Hydra jobs on LSF clusters. It is a simple wrapper around the `bsub` command that allows you to launch Hydra jobs with a single command.
### **WARNING: expect bugs and run at your own risk. This is not an official Hydra project.**
## Installation
Install the package with pip:
```bash
pip install hydra-lsf-launcher
```
## Usage
Here is an example using `hydra-zen` to launch a job on an LSF cluster:
```python
from hydra_zen import store, zen
import time
def main_func(sentence):
time.sleep(120)
print(sentence)
store(
main_func,
hydra_defaults=[
"_self_",
]
)
if __name__ == "__main__":
store.add_to_hydra_store()
zen(main_func).hydra_main(
config_name="main_func", version_base="1.1", config_path=None
)
```
Then you can run it with:
```bash
python myscript.py hydra/launcher=lsf sentence="Hello World!" -m
```
## Job groups
To limit the number of concurrent jobs, as for now job arrays are not supported, the user can set job groups:
```bash
bgadd /myJobGroup
bgmod -L 4 /myJobGroup # limit to 4 concurrent jobs
```
Then you can run it with:
```bash
python myscript.py hydra/launcher=lsf hydra.launcher.bsub_args="-g /myJobGroup" sentence="Hello World!" -m
```
Some useful commands.
To change the limit of a job group:
```bash
bgmod -L <new_limit> /myJobGroup
```
To delete a job group:
```bash
bgdel /myJobGroup
```
## More complex examples
Run 5 jobs (4 parallel), each with 2GB of memory, and 4 cores from 2 different nodes.
```bash
python myscript.py hydra/launcher=lsf hydra.launcher.verbose=False hydra.launcher.n=4 'hydra.launcher.bsub_args= -g /myJobGroup -R span\[ptile\=2\]' hydra.launcher.M="2GB" sentence="Hello World!,Hello World!,Hello World!,Hello World!,Hello World!" -m
```
Raw data
{
"_id": null,
"home_page": null,
"name": "hydra-lsf-launcher",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Francesco Spinnato",
"author_email": "francesco.spinnato@di.unipi.it",
"download_url": "https://files.pythonhosted.org/packages/27/75/70cf18c57574661faed6d35d464bf348eab10bdb2b9e3e6053009758073b/hydra_lsf_launcher-0.1.2.tar.gz",
"platform": null,
"description": "# hydra-lsf-launcher\nThis is a simple launcher for launching Hydra jobs on LSF clusters. It is a simple wrapper around the `bsub` command that allows you to launch Hydra jobs with a single command.\n\n### **WARNING: expect bugs and run at your own risk. This is not an official Hydra project.**\n\n## Installation\nInstall the package with pip:\n```bash\npip install hydra-lsf-launcher\n```\n\n## Usage\n\nHere is an example using `hydra-zen` to launch a job on an LSF cluster:\n\n```python\nfrom hydra_zen import store, zen\nimport time\n\n\ndef main_func(sentence):\n time.sleep(120)\n print(sentence)\n\n\nstore(\n main_func,\n hydra_defaults=[\n \"_self_\",\n ]\n)\n\n\nif __name__ == \"__main__\":\n store.add_to_hydra_store()\n zen(main_func).hydra_main(\n config_name=\"main_func\", version_base=\"1.1\", config_path=None\n )\n```\n\nThen you can run it with:\n\n```bash\npython myscript.py hydra/launcher=lsf sentence=\"Hello World!\" -m \n```\n\n## Job groups\n\nTo limit the number of concurrent jobs, as for now job arrays are not supported, the user can set job groups:\n\n```bash\nbgadd /myJobGroup\nbgmod -L 4 /myJobGroup # limit to 4 concurrent jobs\n```\n\n\n\nThen you can run it with:\n\n```bash\npython myscript.py hydra/launcher=lsf hydra.launcher.bsub_args=\"-g /myJobGroup\" sentence=\"Hello World!\" -m \n```\n\nSome useful commands.\nTo change the limit of a job group:\n```bash\nbgmod -L <new_limit> /myJobGroup\n```\n\nTo delete a job group:\n```bash\nbgdel /myJobGroup\n```\n\n\n## More complex examples\nRun 5 jobs (4 parallel), each with 2GB of memory, and 4 cores from 2 different nodes.\n\n```bash\npython myscript.py hydra/launcher=lsf hydra.launcher.verbose=False hydra.launcher.n=4 'hydra.launcher.bsub_args= -g /myJobGroup -R span\\[ptile\\=2\\]' hydra.launcher.M=\"2GB\" sentence=\"Hello World!,Hello World!,Hello World!,Hello World!,Hello World!\" -m\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Lsf Hydra Launcher plugin",
"version": "0.1.2",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b8ed9d822f6d6a47c374be2765cd50e8b5c6518c74b91c6ee39643055fab7bfa",
"md5": "d36d9d84462f0c6abbe00b72887029f6",
"sha256": "31690ac53a28db65618e9969cd45685920daf19f1fe4847b232170f622450b43"
},
"downloads": -1,
"filename": "hydra_lsf_launcher-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d36d9d84462f0c6abbe00b72887029f6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 4788,
"upload_time": "2024-11-07T17:16:09",
"upload_time_iso_8601": "2024-11-07T17:16:09.454902Z",
"url": "https://files.pythonhosted.org/packages/b8/ed/9d822f6d6a47c374be2765cd50e8b5c6518c74b91c6ee39643055fab7bfa/hydra_lsf_launcher-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "277570cf18c57574661faed6d35d464bf348eab10bdb2b9e3e6053009758073b",
"md5": "960147d6aceba89d3717b62ea68133fa",
"sha256": "98191670be92da729c1bb7a79abfc97b275b903e5752f03e6bc0bc9fd3b02a1e"
},
"downloads": -1,
"filename": "hydra_lsf_launcher-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "960147d6aceba89d3717b62ea68133fa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4568,
"upload_time": "2024-11-07T17:16:10",
"upload_time_iso_8601": "2024-11-07T17:16:10.281522Z",
"url": "https://files.pythonhosted.org/packages/27/75/70cf18c57574661faed6d35d464bf348eab10bdb2b9e3e6053009758073b/hydra_lsf_launcher-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-07 17:16:10",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "hydra-lsf-launcher"
}
