# Trailblazer [![Coverage Status][coveralls-image]][coveralls-url]
### Monitor the progress of analysis workflows submitted to SLURM
Trailblazer is a tool that aims to provide:
- Monitoring of processes that require submission to a workflow manager
- Display metadata for each analysis within a web based user interface
[Here][Trailblazer-UI] you can find a simple web UI for Trailblazer that helps you keep track of the status of multiple runs
## Installation
Trailblazer is written in Python 3.11 and is available on the [Python Package Index][pypi] (PyPI).
```bash
pip install trailblazer
```
If you would like to install the latest development version:
```bash
git clone https://github.com/Clinical-Genomics/trailblazer
cd trailblazer
pip install --editable .
```
With each push to GitHub your files will be automatically verified using [Black] . If you would like to automatically [Black] format your commits on your local machine:
```
pre-commit install
```
## Contributing
Trailblazer uses the GitHub flow branching model as described in Atlas [GitHub Flow].
## Documentation
Here's a brief documentation. Trailblazer functionality can be accessed from the command line interface (CLI), the monitoring web interface, the supporting REST API, as well as using the Python API.
### Command line interface
#### Command: `trailblazer init`
Setup (or reset) a Trailblazer database. The command will set up all the tables in the database. You can reset an existing database by using the `--reset` option.
```shell
trailblazer --database "sqlite:///tb.sqlite3" init --reset
Delete existing tables? [analysis, info, job, user] [y/N]: y
Success! New tables: analysis, info, job, user
```
#### Command: `trailblazer user`
This command can be used both to add a new user to the database (and give them access to the web interface) and view information about an existing user.
```shell
# add a new user
trailblazer user --name "Paul Anderson" paul.anderson@magnolia.com
New user added: paul.anderson@magnolia.com (2)
# check an existing user
trailblazer user paul.anderson@magnolia.com
{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}
```
#### Command: `trailblazer archive-user`
This command archives a user in the database (and removes their access to the web interface).
```shell
# archive a user
trailblazer archive-user paul.anderson@magnolia.com
User archived: paul.anderson@magnolia.com
```
#### Command: `trailblazer users`
This command can be used both to list all users in the database and get a filtered list of users.
```shell
# list all users
trailblazer users
Listing users in database:
{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}
# list all users named 'Anderson' that has an email with 'magnolia' in it
trailblazer users --name Anderson --email magnolia
Listing users in database:
{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}
```
#### Command: `trailblazer log`
Logs the status of a run to the supporting database. You need to point to the analysis config of a specific run.
```shell
trailblazer log path/to/case/analysis/case_config.yaml
```
You can point to the same analysis multiple times, Trailblazer will detect if the same analysis has been added before and skip it if no information has been updated. If an analysis has been added previously as "running" or "pending", those entries will automatically be removed as soon as the same analysis is logged as either "completed" or "failed".
#### Command: `trailblazer scan`
Convenience command to scan an entire directory structure for all analyses and update their status in one go. Assumes the base directory consists of individual case folders:
```shell
trailblazer scan /path/to/analyses/dir/
```
This command can easily be setup in a crontab to run e.g. every hour and keep the analysis statuses up-to-date!
#### Command: `trailblazer ls`
Prints the case id for the most recently completed analyses to the console.
```shell
trailblazer ls
F0013487
F0013362
F0006106
17083
F0013469
17085
```
#### Command: `trailblazer delete`
Deletes an analysis log from the database. The input is the unique analysis id which is printed ones the analysis is initially logged. It's also displayed in the web interface.
```shell
trailblazer delete 4
```
[black]: https://black.readthedocs.io/en/stable/
[coveralls-url]: https://coveralls.io/r/Clinical-Genomics/trailblazer
[coveralls-image]: https://img.shields.io/coveralls/Clinical-Genomics/trailblazer.svg?style=flat-square
[GitHub Flow]: https://atlas.scilifelab.se/infrastructure/github/branching_models/githubflow/
[pypi]: https://pypi.python.org/pypi/trailblazer/
[Trailblazer-UI]: https://github.com/Clinical-Genomics/trailblazer-ui
Raw data
{
"_id": null,
"home_page": "https://github.com/Clinical-Genomics/trailblazer",
"name": "trailblazer",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "analysis monitoring",
"author": "henrik.stranneheim",
"author_email": "henrik.stranneheim@scilifelab.se",
"download_url": "https://files.pythonhosted.org/packages/bd/ee/ac7b5a3d02edd3b738c7c6f5ecc687b57d31cdfa5d1b851a312a8a7a4264/trailblazer-21.5.3.tar.gz",
"platform": null,
"description": "\n# Trailblazer [![Coverage Status][coveralls-image]][coveralls-url]\n\n### Monitor the progress of analysis workflows submitted to SLURM\n\nTrailblazer is a tool that aims to provide:\n- Monitoring of processes that require submission to a workflow manager\n- Display metadata for each analysis within a web based user interface\n\n[Here][Trailblazer-UI] you can find a simple web UI for Trailblazer that helps you keep track of the status of multiple runs\n\n## Installation\n\nTrailblazer is written in Python 3.11 and is available on the [Python Package Index][pypi] (PyPI).\n\n```bash\npip install trailblazer\n```\n\nIf you would like to install the latest development version:\n\n```bash\ngit clone https://github.com/Clinical-Genomics/trailblazer\ncd trailblazer\npip install --editable .\n```\n\nWith each push to GitHub your files will be automatically verified using [Black] . If you would like to automatically [Black] format your commits on your local machine:\n\n```\npre-commit install\n```\n\n## Contributing\n\nTrailblazer uses the GitHub flow branching model as described in Atlas [GitHub Flow].\n\n## Documentation\n\nHere's a brief documentation. Trailblazer functionality can be accessed from the command line interface (CLI), the monitoring web interface, the supporting REST API, as well as using the Python API.\n\n### Command line interface\n\n#### Command: `trailblazer init`\n\nSetup (or reset) a Trailblazer database. The command will set up all the tables in the database. You can reset an existing database by using the `--reset` option.\n\n```shell\ntrailblazer --database \"sqlite:///tb.sqlite3\" init --reset\nDelete existing tables? [analysis, info, job, user] [y/N]: y\nSuccess! New tables: analysis, info, job, user\n```\n\n#### Command: `trailblazer user`\n\nThis command can be used both to add a new user to the database (and give them access to the web interface) and view information about an existing user.\n\n```shell\n# add a new user\ntrailblazer user --name \"Paul Anderson\" paul.anderson@magnolia.com\nNew user added: paul.anderson@magnolia.com (2)\n\n# check an existing user\ntrailblazer user paul.anderson@magnolia.com\n{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}\n```\n\n#### Command: `trailblazer archive-user`\n\nThis command archives a user in the database (and removes their access to the web interface).\n\n```shell\n# archive a user\ntrailblazer archive-user paul.anderson@magnolia.com\nUser archived: paul.anderson@magnolia.com\n```\n\n#### Command: `trailblazer users`\n\nThis command can be used both to list all users in the database and get a filtered list of users.\n\n```shell\n# list all users\ntrailblazer users\nListing users in database:\n{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}\n\n# list all users named 'Anderson' that has an email with 'magnolia' in it\ntrailblazer users --name Anderson --email magnolia\nListing users in database:\n{'created_at': datetime.datetime(2017, 6, 22, 8, 49, 44, 685977), 'google_id': None, 'name': 'Paul Anderson', 'email': 'paul.anderson@magnolia.com', 'avatar': None, 'id': 2}\n```\n\n\n#### Command: `trailblazer log`\n\nLogs the status of a run to the supporting database. You need to point to the analysis config of a specific run.\n\n```shell\ntrailblazer log path/to/case/analysis/case_config.yaml\n```\n\nYou can point to the same analysis multiple times, Trailblazer will detect if the same analysis has been added before and skip it if no information has been updated. If an analysis has been added previously as \"running\" or \"pending\", those entries will automatically be removed as soon as the same analysis is logged as either \"completed\" or \"failed\".\n\n#### Command: `trailblazer scan`\n\nConvenience command to scan an entire directory structure for all analyses and update their status in one go. Assumes the base directory consists of individual case folders:\n\n```shell\ntrailblazer scan /path/to/analyses/dir/\n```\n\nThis command can easily be setup in a crontab to run e.g. every hour and keep the analysis statuses up-to-date!\n\n#### Command: `trailblazer ls`\n\nPrints the case id for the most recently completed analyses to the console.\n\n```shell\ntrailblazer ls\nF0013487\nF0013362\nF0006106\n17083\nF0013469\n17085\n```\n\n#### Command: `trailblazer delete`\n\nDeletes an analysis log from the database. The input is the unique analysis id which is printed ones the analysis is initially logged. It's also displayed in the web interface.\n\n```shell\ntrailblazer delete 4\n```\n\n[black]: https://black.readthedocs.io/en/stable/\n[coveralls-url]: https://coveralls.io/r/Clinical-Genomics/trailblazer\n[coveralls-image]: https://img.shields.io/coveralls/Clinical-Genomics/trailblazer.svg?style=flat-square\n[GitHub Flow]: https://atlas.scilifelab.se/infrastructure/github/branching_models/githubflow/\n[pypi]: https://pypi.python.org/pypi/trailblazer/\n[Trailblazer-UI]: https://github.com/Clinical-Genomics/trailblazer-ui\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Trailblazer is a tool to manage and track state of analyses",
"version": "21.5.3",
"project_urls": {
"Homepage": "https://github.com/Clinical-Genomics/trailblazer"
},
"split_keywords": [
"analysis",
"monitoring"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "157be022588d5b14b0e09ef9694e090385cbc0b46a525d006290622e2d31f73e",
"md5": "1893fe9df96b1fce5d83ff36206781fc",
"sha256": "67aefab75ad7f164bd193f9fbbc08c76ec1238a2dc1b0362ec2a5d122b1400e6"
},
"downloads": -1,
"filename": "trailblazer-21.5.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1893fe9df96b1fce5d83ff36206781fc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 66721,
"upload_time": "2024-04-30T09:13:01",
"upload_time_iso_8601": "2024-04-30T09:13:01.119470Z",
"url": "https://files.pythonhosted.org/packages/15/7b/e022588d5b14b0e09ef9694e090385cbc0b46a525d006290622e2d31f73e/trailblazer-21.5.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bdeeac7b5a3d02edd3b738c7c6f5ecc687b57d31cdfa5d1b851a312a8a7a4264",
"md5": "c11f62b5362a76d98ca92dde7533ceeb",
"sha256": "48c579fc929bfedc4a0040a009a09ffe27c7870578d35563e488b1ed5fb8da16"
},
"downloads": -1,
"filename": "trailblazer-21.5.3.tar.gz",
"has_sig": false,
"md5_digest": "c11f62b5362a76d98ca92dde7533ceeb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 76962,
"upload_time": "2024-04-30T09:13:04",
"upload_time_iso_8601": "2024-04-30T09:13:04.565950Z",
"url": "https://files.pythonhosted.org/packages/bd/ee/ac7b5a3d02edd3b738c7c6f5ecc687b57d31cdfa5d1b851a312a8a7a4264/trailblazer-21.5.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-30 09:13:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Clinical-Genomics",
"github_project": "trailblazer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "trailblazer"
}
JSON
