| Name | hydraflow JSON |
| Version |
0.17.0
JSON |
| download |
| home_page | None |
| Summary | HydraFlow seamlessly integrates Hydra and MLflow to streamline ML experiment management, combining Hydra's configuration management with MLflow's tracking capabilities. |
| upload_time | 2025-04-25 23:13:16 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.13 |
| license | MIT License Copyright (c) 2024 Daizu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
ai
data-science
deep-learning
experiment-tracking
hydra
machine-learning
mlflow
mlops
research
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# HydraFlow
[![PyPI Version][pypi-v-image]][pypi-v-link]
[![Build Status][GHAction-image]][GHAction-link]
[![Coverage Status][codecov-image]][codecov-link]
[![Documentation Status][docs-image]][docs-link]
[![Python Version][python-v-image]][python-v-link]
<!-- Badges -->
[pypi-v-image]: https://img.shields.io/pypi/v/hydraflow.svg
[pypi-v-link]: https://pypi.org/project/hydraflow/
[GHAction-image]: https://github.com/daizutabi/hydraflow/actions/workflows/ci.yaml/badge.svg?branch=main&event=push
[GHAction-link]: https://github.com/daizutabi/hydraflow/actions?query=event%3Apush+branch%3Amain
[codecov-image]: https://codecov.io/github/daizutabi/hydraflow/coverage.svg?branch=main
[codecov-link]: https://codecov.io/github/daizutabi/hydraflow?branch=main
[docs-image]: https://img.shields.io/badge/docs-latest-blue.svg
[docs-link]: https://daizutabi.github.io/hydraflow/
[python-v-image]: https://img.shields.io/pypi/pyversions/hydraflow.svg
[python-v-link]: https://pypi.org/project/hydraflow
## Overview
HydraFlow seamlessly integrates [Hydra](https://hydra.cc/) and [MLflow](https://mlflow.org/) to streamline machine learning experiment workflows. By combining Hydra's powerful configuration management with MLflow's robust experiment tracking, HydraFlow provides a comprehensive solution for defining, executing, and analyzing machine learning experiments.
## Design Principles
HydraFlow is built on the following design principles:
1. **Type Safety** - Utilizing Python dataclasses for configuration type checking and IDE support
2. **Reproducibility** - Automatically tracking all experiment configurations for fully reproducible experiments
3. **Analysis Capabilities** - Providing powerful APIs for easily analyzing experiment results
4. **Workflow Integration** - Creating a cohesive workflow by integrating Hydra's configuration management with MLflow's experiment tracking
## Key Features
- **Type-safe Configuration Management** - Define experiment parameters using Python dataclasses with full IDE support and validation
- **Seamless Hydra-MLflow Integration** - Automatically register configurations with Hydra and track experiments with MLflow
- **Advanced Parameter Sweeps** - Define complex parameter spaces using extended sweep syntax for numerical ranges, combinations, and SI prefixes
- **Workflow Automation** - Create reusable experiment workflows with YAML-based job definitions
- **Powerful Analysis Tools** - Filter, group, and analyze experiment results with type-aware APIs
- **Custom Implementation Support** - Extend experiment analysis with domain-specific functionality
## Installation
```bash
pip install hydraflow
```
**Requirements:** Python 3.13+
## Quick Example
```python
from dataclasses import dataclass
from mlflow.entities import Run
import hydraflow
@dataclass
class Config:
width: int = 1024
height: int = 768
@hydraflow.main(Config)
def app(run: Run, cfg: Config) -> None:
# Your experiment code here
print(f"Running with width={cfg.width}, height={cfg.height}")
# Log metrics
hydraflow.log_metric("area", cfg.width * cfg.height)
if __name__ == "__main__":
app()
```
Execute a parameter sweep with:
```bash
python app.py -m width=800,1200 height=600,900
```
## Core Components
HydraFlow consists of the following key components:
### Configuration Management
Define type-safe configurations using Python dataclasses:
```python
@dataclass
class Config:
learning_rate: float = 0.001
batch_size: int = 32
epochs: int = 10
```
### Main Decorator
The `@hydraflow.main` decorator integrates Hydra and MLflow:
```python
@hydraflow.main(Config)
def train(run: Run, cfg: Config) -> None:
# Your experiment code
```
### Workflow Automation
Define reusable experiment workflows in YAML:
```yaml
jobs:
train_models:
run: python train.py
sets:
- each: model=small,medium,large
all: learning_rate=0.001,0.01,0.1
```
### Analysis Tools
Analyze experiment results with powerful APIs:
```python
from hydraflow import Run, iter_run_dirs
# Load runs
runs = Run.load(iter_run_dirs("mlruns"))
# Filter and analyze
best_runs = runs.filter(model_type="transformer").to_frame("learning_rate", "accuracy")
```
## Documentation
For detailed documentation, visit our [documentation site](https://daizutabi.github.io/hydraflow/):
- [Getting Started](https://daizutabi.github.io/hydraflow/getting-started/) - Installation and core concepts
- [Practical Tutorials](https://daizutabi.github.io/hydraflow/practical-tutorials/) - Learn through hands-on examples
- [User Guide](https://daizutabi.github.io/hydraflow/part1-applications/) - Detailed documentation of HydraFlow's capabilities
- [API Reference](https://daizutabi.github.io/hydraflow/api/hydraflow/) - Complete API documentation
## License
This project is licensed under the MIT License.
Raw data
{
"_id": null,
"home_page": null,
"name": "hydraflow",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.13",
"maintainer_email": null,
"keywords": "ai, data-science, deep-learning, experiment-tracking, hydra, machine-learning, mlflow, mlops, research",
"author": null,
"author_email": "daizutabi <daizutabi@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/94/3b/23a5d63a72539d06ea415e02c190c20c666bd927f5c494606aa2e09c7dda/hydraflow-0.17.0.tar.gz",
"platform": null,
"description": "# HydraFlow\n\n[![PyPI Version][pypi-v-image]][pypi-v-link]\n[![Build Status][GHAction-image]][GHAction-link]\n[![Coverage Status][codecov-image]][codecov-link]\n[![Documentation Status][docs-image]][docs-link]\n[![Python Version][python-v-image]][python-v-link]\n\n<!-- Badges -->\n\n[pypi-v-image]: https://img.shields.io/pypi/v/hydraflow.svg\n[pypi-v-link]: https://pypi.org/project/hydraflow/\n[GHAction-image]: https://github.com/daizutabi/hydraflow/actions/workflows/ci.yaml/badge.svg?branch=main&event=push\n[GHAction-link]: https://github.com/daizutabi/hydraflow/actions?query=event%3Apush+branch%3Amain\n[codecov-image]: https://codecov.io/github/daizutabi/hydraflow/coverage.svg?branch=main\n[codecov-link]: https://codecov.io/github/daizutabi/hydraflow?branch=main\n[docs-image]: https://img.shields.io/badge/docs-latest-blue.svg\n[docs-link]: https://daizutabi.github.io/hydraflow/\n[python-v-image]: https://img.shields.io/pypi/pyversions/hydraflow.svg\n[python-v-link]: https://pypi.org/project/hydraflow\n\n## Overview\n\nHydraFlow seamlessly integrates [Hydra](https://hydra.cc/) and [MLflow](https://mlflow.org/) to streamline machine learning experiment workflows. By combining Hydra's powerful configuration management with MLflow's robust experiment tracking, HydraFlow provides a comprehensive solution for defining, executing, and analyzing machine learning experiments.\n\n## Design Principles\n\nHydraFlow is built on the following design principles:\n\n1. **Type Safety** - Utilizing Python dataclasses for configuration type checking and IDE support\n2. **Reproducibility** - Automatically tracking all experiment configurations for fully reproducible experiments\n3. **Analysis Capabilities** - Providing powerful APIs for easily analyzing experiment results\n4. **Workflow Integration** - Creating a cohesive workflow by integrating Hydra's configuration management with MLflow's experiment tracking\n\n## Key Features\n\n- **Type-safe Configuration Management** - Define experiment parameters using Python dataclasses with full IDE support and validation\n- **Seamless Hydra-MLflow Integration** - Automatically register configurations with Hydra and track experiments with MLflow\n- **Advanced Parameter Sweeps** - Define complex parameter spaces using extended sweep syntax for numerical ranges, combinations, and SI prefixes\n- **Workflow Automation** - Create reusable experiment workflows with YAML-based job definitions\n- **Powerful Analysis Tools** - Filter, group, and analyze experiment results with type-aware APIs\n- **Custom Implementation Support** - Extend experiment analysis with domain-specific functionality\n\n## Installation\n\n```bash\npip install hydraflow\n```\n\n**Requirements:** Python 3.13+\n\n## Quick Example\n\n```python\nfrom dataclasses import dataclass\nfrom mlflow.entities import Run\nimport hydraflow\n\n@dataclass\nclass Config:\n width: int = 1024\n height: int = 768\n\n@hydraflow.main(Config)\ndef app(run: Run, cfg: Config) -> None:\n # Your experiment code here\n print(f\"Running with width={cfg.width}, height={cfg.height}\")\n\n # Log metrics\n hydraflow.log_metric(\"area\", cfg.width * cfg.height)\n\nif __name__ == \"__main__\":\n app()\n```\n\nExecute a parameter sweep with:\n\n```bash\npython app.py -m width=800,1200 height=600,900\n```\n\n## Core Components\n\nHydraFlow consists of the following key components:\n\n### Configuration Management\n\nDefine type-safe configurations using Python dataclasses:\n\n```python\n@dataclass\nclass Config:\n learning_rate: float = 0.001\n batch_size: int = 32\n epochs: int = 10\n```\n\n### Main Decorator\n\nThe `@hydraflow.main` decorator integrates Hydra and MLflow:\n\n```python\n@hydraflow.main(Config)\ndef train(run: Run, cfg: Config) -> None:\n # Your experiment code\n```\n\n### Workflow Automation\n\nDefine reusable experiment workflows in YAML:\n\n```yaml\njobs:\n train_models:\n run: python train.py\n sets:\n - each: model=small,medium,large\n all: learning_rate=0.001,0.01,0.1\n```\n\n### Analysis Tools\n\nAnalyze experiment results with powerful APIs:\n\n```python\nfrom hydraflow import Run, iter_run_dirs\n\n# Load runs\nruns = Run.load(iter_run_dirs(\"mlruns\"))\n\n# Filter and analyze\nbest_runs = runs.filter(model_type=\"transformer\").to_frame(\"learning_rate\", \"accuracy\")\n```\n\n## Documentation\n\nFor detailed documentation, visit our [documentation site](https://daizutabi.github.io/hydraflow/):\n\n- [Getting Started](https://daizutabi.github.io/hydraflow/getting-started/) - Installation and core concepts\n- [Practical Tutorials](https://daizutabi.github.io/hydraflow/practical-tutorials/) - Learn through hands-on examples\n- [User Guide](https://daizutabi.github.io/hydraflow/part1-applications/) - Detailed documentation of HydraFlow's capabilities\n- [API Reference](https://daizutabi.github.io/hydraflow/api/hydraflow/) - Complete API documentation\n\n## License\n\nThis project is licensed under the MIT License.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Daizu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "HydraFlow seamlessly integrates Hydra and MLflow to streamline ML experiment management, combining Hydra's configuration management with MLflow's tracking capabilities.",
"version": "0.17.0",
"project_urls": {
"Documentation": "https://daizutabi.github.io/hydraflow/",
"Issues": "https://github.com/daizutabi/hydraflow/issues",
"Source": "https://github.com/daizutabi/hydraflow"
},
"split_keywords": [
"ai",
" data-science",
" deep-learning",
" experiment-tracking",
" hydra",
" machine-learning",
" mlflow",
" mlops",
" research"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e95e978611ab0403566970b2f962d27fd3e668a92ddb2f99f605f8f9046c3183",
"md5": "2a8a005f28aa979adc1af31d56bd5da1",
"sha256": "97e1316142f4153740103efdba7332c7b1c05d464fbb3e7f4c0285ac0e3565a6"
},
"downloads": -1,
"filename": "hydraflow-0.17.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2a8a005f28aa979adc1af31d56bd5da1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.13",
"size": 32183,
"upload_time": "2025-04-25T23:13:15",
"upload_time_iso_8601": "2025-04-25T23:13:15.325728Z",
"url": "https://files.pythonhosted.org/packages/e9/5e/978611ab0403566970b2f962d27fd3e668a92ddb2f99f605f8f9046c3183/hydraflow-0.17.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "943b23a5d63a72539d06ea415e02c190c20c666bd927f5c494606aa2e09c7dda",
"md5": "76960c5be3444945dbfa75f033bfc51c",
"sha256": "089582efe4dde72f0f89b62ba9cfaee930251a0af653a1dec23a62557a202b70"
},
"downloads": -1,
"filename": "hydraflow-0.17.0.tar.gz",
"has_sig": false,
"md5_digest": "76960c5be3444945dbfa75f033bfc51c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.13",
"size": 73049,
"upload_time": "2025-04-25T23:13:16",
"upload_time_iso_8601": "2025-04-25T23:13:16.380535Z",
"url": "https://files.pythonhosted.org/packages/94/3b/23a5d63a72539d06ea415e02c190c20c666bd927f5c494606aa2e09c7dda/hydraflow-0.17.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-04-25 23:13:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "daizutabi",
"github_project": "hydraflow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "hydraflow"
}
