| Name | hydra-callbacks JSON |
| Version |
0.6.1
JSON |
| download |
| home_page | None |
| Summary | A collection of usefull hydra callbacks for hydra configuration |
| upload_time | 2024-05-24 12:18:03 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | None |
| keywords |
hydra
configuration
callback
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Hydra Callbacks
[](https://github.com/psf/black)
[](https://hydra.cc)
[](https://codecov.io/gh/paquiteau/hydra-callbacks)
[](https://github.com/paquiteau/hydra-callbacks/actions/workflows/master-cd.yml)
[](https://github.com/paquiteau/hydra-callbacks/actions/workflows/test-ci.yml)
[](https://github.com/paquiteau/hydra-callbacks/releases/latest)
A collection of usefulls and simple to use callbacks for the [https://hydra.cc/](hydra) configuration framework.
## Installation
``` shell
pip install hydra-callbacks
```
Development version
``` shell
pip install git+https://github.com/paquiteau/hydra-callbacks
```
## Usage
In your hydra root config file add the following, or analoguous:
``` yaml
hydra:
callbacks:
git_infos:
_target_: hydra_callbacks.GitInfo
clean: true
latest_run:
_target_: hydra_callbacks.LatestRunLink
resource_monitor:
_target_: hydra_callbacks.ResourceMonitor
sample_interval: 0.5
runtime_perf:
_target_: hydra_callbacks.RuntimePerformance
```
This will enrich your script output with:
```console
paquiteau@laptop$ python my_app.py
[hydra] Git sha: 844b9ca1a74d8307ef5331351897cebb18f71b88, dirty: False
## All your app log and outputs ##
[hydra][INFO] - Total runtime: 0.51 seconds
[hydra][INFO] - Writing monitoring data to [...]/outputs/2023-04-06/16-02-46/resource_monitoring.csv
[hydra][INFO] - Latest run is at: [...]/outputs/latest
```
Detailled configuration for each callback is available in the `tests/test_app/` folder.
## Available Callbacks
| Name | Action |
|:-------------------|:---------------------------------------------------|
| GitInfo | Check status of Repository |
| LatestRunLink | Get a link to the latest run |
| MultiRunGatherer | Gather results json file in a single table |
| RuntimePerformance | Get Execution time for each run |
| ResourceMonitor | Monitor resources of running jobs (CPU and Memory) |
And more to come !
## Also Available
- `PerfLogger` : A simple to use performance logger
```python
from hydra_callbacks import PerfLogger
import logging
log = logging.getLogger(__name__)
def main_app(cfg):
with PerfLogger(log, "step1"):
sleep(1)
with PerfLogger(log, "step2"):
sleep(2)
PerfLogger.recap(log)
```
- `RessourceMonitorService` : A simple CPU and GPU usage and memory sampler. It launches an extra process to monitor everything.
```python
from hydra_callbacks.monitor import RessourceMonitorService
import os
monitor = RessourceMonitorService(interval=0.2, gpu_monit=True)
monitor.start()
# Launch heavy stuff
metrics = monitor.stop()
# Or use it as a context manager
with RessourceMonitorService(interval=0.2, gpu_monit=True) as monitor:
# launch heavy stuff
metrics_values = monitor.get_values()
```
## You too, have cool Callbacks, or idea for one ?
Open a PR or an issue !
### Possible Ideas
- A callback that summarize log from multiple runs
- Monitoring of GPU using nvitop
<p align=center> :star2: If you like this work, don't forget to star it and share it 🌟 </p>
Raw data
{
"_id": null,
"home_page": null,
"name": "hydra-callbacks",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "hydra, configuration, callback",
"author": null,
"author_email": "Pierre-Antoine Comby <pierre-antoine.comby@cea.fr>",
"download_url": "https://files.pythonhosted.org/packages/59/81/2c73ffba354d5aa9144431033a548d497586f71c631254a4883fb498c787/hydra_callbacks-0.6.1.tar.gz",
"platform": null,
"description": "# Hydra Callbacks \n\n\n[](https://github.com/psf/black)\n[](https://hydra.cc)\n[](https://codecov.io/gh/paquiteau/hydra-callbacks)\n[](https://github.com/paquiteau/hydra-callbacks/actions/workflows/master-cd.yml)\n[](https://github.com/paquiteau/hydra-callbacks/actions/workflows/test-ci.yml)\n[](https://github.com/paquiteau/hydra-callbacks/releases/latest)\n\nA collection of usefulls and simple to use callbacks for the [https://hydra.cc/](hydra) configuration framework.\n\n\n## Installation \n``` shell \npip install hydra-callbacks\n```\n\nDevelopment version \n``` shell\npip install git+https://github.com/paquiteau/hydra-callbacks\n```\n\n## Usage \n\nIn your hydra root config file add the following, or analoguous:\n\n``` yaml\nhydra:\n callbacks:\n git_infos:\n _target_: hydra_callbacks.GitInfo\n clean: true\n latest_run:\n _target_: hydra_callbacks.LatestRunLink\n resource_monitor:\n _target_: hydra_callbacks.ResourceMonitor\n sample_interval: 0.5\n runtime_perf:\n _target_: hydra_callbacks.RuntimePerformance \n```\n\nThis will enrich your script output with: \n\n```console\npaquiteau@laptop$ python my_app.py\n[hydra] Git sha: 844b9ca1a74d8307ef5331351897cebb18f71b88, dirty: False\n\n## All your app log and outputs ##\n\n[hydra][INFO] - Total runtime: 0.51 seconds\n[hydra][INFO] - Writing monitoring data to [...]/outputs/2023-04-06/16-02-46/resource_monitoring.csv\n[hydra][INFO] - Latest run is at: [...]/outputs/latest\n```\n\n\nDetailled configuration for each callback is available in the `tests/test_app/` folder.\n\n## Available Callbacks \n\n| Name | Action |\n|:-------------------|:---------------------------------------------------|\n| GitInfo | Check status of Repository |\n| LatestRunLink | Get a link to the latest run |\n| MultiRunGatherer | Gather results json file in a single table |\n| RuntimePerformance | Get Execution time for each run |\n| ResourceMonitor | Monitor resources of running jobs (CPU and Memory) |\n\nAnd more to come ! \n\n## Also Available \n \n - `PerfLogger` : A simple to use performance logger\n \n```python\n \nfrom hydra_callbacks import PerfLogger \nimport logging\n\nlog = logging.getLogger(__name__)\ndef main_app(cfg):\n with PerfLogger(log, \"step1\"):\n sleep(1)\n\n with PerfLogger(log, \"step2\"):\n sleep(2)\n PerfLogger.recap(log)\n\n```\n - `RessourceMonitorService` : A simple CPU and GPU usage and memory sampler. It launches an extra process to monitor everything.\n \n ```python\nfrom hydra_callbacks.monitor import RessourceMonitorService\nimport os \n \nmonitor = RessourceMonitorService(interval=0.2, gpu_monit=True)\n \nmonitor.start()\n# Launch heavy stuff \nmetrics = monitor.stop()\n\n# Or use it as a context manager\nwith RessourceMonitorService(interval=0.2, gpu_monit=True) as monitor: \n # launch heavy stuff\n \nmetrics_values = monitor.get_values()\n```\n\n\n## You too, have cool Callbacks, or idea for one ? \n\nOpen a PR or an issue !\n\n### Possible Ideas\n- A callback that summarize log from multiple runs\n- Monitoring of GPU using nvitop\n \n<p align=center> :star2: If you like this work, don't forget to star it and share it \ud83c\udf1f </p>\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A collection of usefull hydra callbacks for hydra configuration",
"version": "0.6.1",
"project_urls": {
"Bug Reports": "https://github.com/paquiteau/hydra-callbacks/issues",
"Homepage": "https://github.com/paquiteau/hydra-callbacks",
"Sources": "https://github.com/paquiteau/hydra-callbacks"
},
"split_keywords": [
"hydra",
" configuration",
" callback"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c7897d39bb3349ebbf944eceb94410314c24f14b8adcff0c5aeebd671ad848f0",
"md5": "7d2e8b1743ae56583c50ed51fc94b4b2",
"sha256": "90cdb8f0e9cea500a6f0b25d86af46d65606f5b20138aea259b1fc58cf615db3"
},
"downloads": -1,
"filename": "hydra_callbacks-0.6.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7d2e8b1743ae56583c50ed51fc94b4b2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11931,
"upload_time": "2024-05-24T12:18:01",
"upload_time_iso_8601": "2024-05-24T12:18:01.641193Z",
"url": "https://files.pythonhosted.org/packages/c7/89/7d39bb3349ebbf944eceb94410314c24f14b8adcff0c5aeebd671ad848f0/hydra_callbacks-0.6.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "59812c73ffba354d5aa9144431033a548d497586f71c631254a4883fb498c787",
"md5": "88ae12ebd55d2d438b25e8500b2966c4",
"sha256": "24bc3adaf6d328966b8baa0fc5ea22357c86ea332a57408858ddca068e2e0f99"
},
"downloads": -1,
"filename": "hydra_callbacks-0.6.1.tar.gz",
"has_sig": false,
"md5_digest": "88ae12ebd55d2d438b25e8500b2966c4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 18509,
"upload_time": "2024-05-24T12:18:03",
"upload_time_iso_8601": "2024-05-24T12:18:03.618129Z",
"url": "https://files.pythonhosted.org/packages/59/81/2c73ffba354d5aa9144431033a548d497586f71c631254a4883fb498c787/hydra_callbacks-0.6.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-24 12:18:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "paquiteau",
"github_project": "hydra-callbacks",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "hydra-callbacks"
}
