# Troika
Submit, monitor and kill jobs on local and remote hosts
## Requirements
* Python 3.8 or higher
* `pyyaml` (https://pypi.org/project/PyYAML/)
* For testing: `pytest` (https://pypi.org/project/pytest/)
* For building the documentation: ``sphinx`` (https://www.sphinx-doc.org)
## Installing
```
python3 -m venv troika
source troika/bin/activate
python3 -m pip install troika
```
### Running the tests
Once Troika is installed in your environment, the tests can be run using `pytest`:
```
python3 -m pytest -v tests/
```
### Building documentation
The documentation uses `sphinx`. To generate the HTML docs:
```
cd docs/
make html
```
## Presentation
Slides and recording of the *"Troika: Submit, monitor, and interrupt jobs on any HPC system with the same interface"*
talk at [FOSDEM'23](https://fosdem.org/2023) are available via https://fosdem.org/2023/schedule/event/troika_hpc_jobs .
## Getting started
### Concepts
Troika holds a list of *sites* onto which jobs can be submitted. A site is
defined by two main parameters: a *connection type* (`local` or `ssh`), and a
*site type* (e.g. `direct` or `slurm`). Every site is identified by a name
given in the configuration file.
### Example configuration file
```yaml
---
sites:
localhost:
type: direct # jobs are run directly on the target
connection: local # the target is the current host
remote:
type: direct # jobs are run directly on the target
connection: ssh # connect to the target via ssh
host: remotebox # ssh host
copy_script: true # if false, the script will be piped through ssh
at_startup: ["check_connection"]
slurm_cluster:
type: slurm # jobs are submitted to Slurm
connection: ssh # connect to the target via ssh
host: remotecluster # ssh host
copy_script: true # if false, the script will be piped through ssh
at_startup: ["check_connection"]
pre_submit: ["create_output_dir"]
at_exit: ["copy_submit_logfile"]
pbs_cluster:
type: pbs # jobs are submitted to PBS
connection: ssh # connect to the target via ssh
host: othercluster # ssh host
copy_script: true # if false, the script will be piped through ssh
at_startup: ["check_connection"]
pre_submit: ["create_output_dir"]
at_exit: ["copy_submit_logfile"]
```
The configuration can be checked using the `list-sites` command:
```
$ troika -c config.yml list-sites
Available sites:
Name Type Connection
------------------------------------------------------------
localhost direct local
remote direct ssh
slurm_cluster slurm ssh
pbs_cluster pbs ssh
```
### Available options
```
$ troika --help
```
### Main commands
Submit a job on `cluster`:
```
$ troika -c config.yaml submit -o /path/to/output/file cluster job.sh
```
Query the status of the job:
```
$ troika -c config.yaml monitor cluster job.sh
```
Kill the job:
```
$ troika -c config.yaml kill cluster job.sh
```
Raw data
{
"_id": null,
"home_page": "https://github.com/ecmwf/troika",
"name": "troika",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Olivier Iffrig",
"author_email": "software.support@ecmwf.int",
"download_url": "https://files.pythonhosted.org/packages/91/5f/396b0b1baafd528ee8b8656b81ee4842c2c5d3b335d450b8195eacc59ba5/troika-0.2.2.tar.gz",
"platform": null,
"description": "# Troika\n\nSubmit, monitor and kill jobs on local and remote hosts\n\n## Requirements\n\n* Python 3.8 or higher\n* `pyyaml` (https://pypi.org/project/PyYAML/)\n* For testing: `pytest` (https://pypi.org/project/pytest/)\n* For building the documentation: ``sphinx`` (https://www.sphinx-doc.org)\n\n## Installing\n\n```\npython3 -m venv troika\nsource troika/bin/activate\npython3 -m pip install troika\n```\n\n### Running the tests\n\nOnce Troika is installed in your environment, the tests can be run using `pytest`:\n\n```\npython3 -m pytest -v tests/\n```\n\n### Building documentation\n\nThe documentation uses `sphinx`. To generate the HTML docs:\n\n```\ncd docs/\nmake html\n```\n\n## Presentation\n\nSlides and recording of the *\"Troika: Submit, monitor, and interrupt jobs on any HPC system with the same interface\"*\ntalk at [FOSDEM'23](https://fosdem.org/2023) are available via https://fosdem.org/2023/schedule/event/troika_hpc_jobs .\n\n## Getting started\n\n### Concepts\n\nTroika holds a list of *sites* onto which jobs can be submitted. A site is\ndefined by two main parameters: a *connection type* (`local` or `ssh`), and a\n*site type* (e.g. `direct` or `slurm`). Every site is identified by a name\ngiven in the configuration file.\n\n### Example configuration file\n\n```yaml\n---\nsites:\n localhost:\n type: direct # jobs are run directly on the target\n connection: local # the target is the current host\n remote:\n type: direct # jobs are run directly on the target\n connection: ssh # connect to the target via ssh\n host: remotebox # ssh host\n copy_script: true # if false, the script will be piped through ssh\n at_startup: [\"check_connection\"]\n slurm_cluster:\n type: slurm # jobs are submitted to Slurm\n connection: ssh # connect to the target via ssh\n host: remotecluster # ssh host\n copy_script: true # if false, the script will be piped through ssh\n at_startup: [\"check_connection\"]\n pre_submit: [\"create_output_dir\"]\n at_exit: [\"copy_submit_logfile\"]\n pbs_cluster:\n type: pbs # jobs are submitted to PBS\n connection: ssh # connect to the target via ssh\n host: othercluster # ssh host\n copy_script: true # if false, the script will be piped through ssh\n at_startup: [\"check_connection\"]\n pre_submit: [\"create_output_dir\"]\n at_exit: [\"copy_submit_logfile\"]\n```\n\nThe configuration can be checked using the `list-sites` command:\n\n```\n$ troika -c config.yml list-sites\nAvailable sites:\nName Type Connection\n------------------------------------------------------------\nlocalhost direct local\nremote direct ssh\nslurm_cluster slurm ssh\npbs_cluster pbs ssh\n```\n\n### Available options\n\n```\n$ troika --help\n```\n\n### Main commands\n\nSubmit a job on `cluster`:\n\n```\n$ troika -c config.yaml submit -o /path/to/output/file cluster job.sh\n```\n\nQuery the status of the job:\n\n```\n$ troika -c config.yaml monitor cluster job.sh\n```\n\nKill the job:\n\n```\n$ troika -c config.yaml kill cluster job.sh\n```\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "Submit, monitor and kill jobs on remote systems",
"version": "0.2.2",
"project_urls": {
"Homepage": "https://github.com/ecmwf/troika"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2bd5ee654d556d56529c7d4742e6e0ded9c8c49bac0160c0c7c053b18615799b",
"md5": "e6abcf4f2ce4e36fde0b3f1dc8d0855e",
"sha256": "2fd50032c2f1cff6dd37e16ecf04fc01f4e6e4140497afa2925b2f77dfb4f9f4"
},
"downloads": -1,
"filename": "troika-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e6abcf4f2ce4e36fde0b3f1dc8d0855e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 43551,
"upload_time": "2023-11-20T16:24:12",
"upload_time_iso_8601": "2023-11-20T16:24:12.085207Z",
"url": "https://files.pythonhosted.org/packages/2b/d5/ee654d556d56529c7d4742e6e0ded9c8c49bac0160c0c7c053b18615799b/troika-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "915f396b0b1baafd528ee8b8656b81ee4842c2c5d3b335d450b8195eacc59ba5",
"md5": "fa35ea7f99494dc89d18011f3f46a8a8",
"sha256": "21c815a04b9d71fafd483c206c1bc3e33d4bac8aefbf3b4d7444efa956f978b6"
},
"downloads": -1,
"filename": "troika-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "fa35ea7f99494dc89d18011f3f46a8a8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 31844,
"upload_time": "2023-11-20T16:24:13",
"upload_time_iso_8601": "2023-11-20T16:24:13.352147Z",
"url": "https://files.pythonhosted.org/packages/91/5f/396b0b1baafd528ee8b8656b81ee4842c2c5d3b335d450b8195eacc59ba5/troika-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-20 16:24:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ecmwf",
"github_project": "troika",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "troika"
}
JSON
