<div align="center" markdown="1">
# Hydralette
**Create complex configurations in a simple, pythonic way!**
[](https://github.com/pre-commit/pre-commit)
[](https://black.readthedocs.io/en/stable/)
[](https://pycqa.github.io/isort/)
[](https://github.com/ashleve/lightning-hydra-template#license)
<a href="https://valeknappich.github.io/hydralette" style="font-size: 14pt">Documentation</a>
</div>
## Installation
```bash
pip install hydralette
# OR
poetry add hydralette
```
## Changelog
- v0.2.3
- change version of `dill` dependency
- v0.2.2
- support mixing _fields, _groups and _from_signature
- support literal values as groups
- add boolean flags as `--flag` or `--no-flag`
- v0.2.1
- fix yaml representation
- support setting value to None in automatic conversion
- v0.2.0
- complete re-design --> breaking changes!
- easier creation of hierarchical configs
- v0.1.5
- Add yaml overrides
- Add end-to-end tests with pytest
- v0.1.4
- Add documentation at [https://valeknappich.github.io/hydralette](https://valeknappich.github.io/hydralette)
- v0.1.3
- Add automatic generation of configs from signatures via `from_signature` and `config_from_signature`
- v0.1.2
- Add support for `config groups`, `references`, `validation` and `type conversion`
- Add CLI help pages
## Features
### Basics
Create a config using the `Config` class, passing the config fields as keyword arguments.
```python
from hydralette import Config
cfg = Config(a=1, b=2, c="abc")
```
`Config` objects can be create from and exported to `dict`s.
```python
from hydralette import Config
cfg = Config.from_dict({"a": 1, "b": 2, "c": "abc"})
print(cfg.to_dict())
```
Every hydralette `Config` be overriden via CLI:
```python
from hydralette import Config, Field
cfg = Config(
a=Field(default=1, help="Lorem ipsum"),
b=Config(
c=Field(default=2, help="Lorem ipsum")
)
)
cfg.apply(["--help"]) # or cfg.print_help()
# prints:
# Usage python script.py [OPTIONS]
# --a 1 Lorem ipsum
# --b.c 1 Lorem ipsum
```
After creation, you can override, resolve references, validate and check for missing required arguments.
```python
cfg.override(["--a", "1"]) # overrides defaults to sys.argv[1:]
cfg.resolve_references()
cfg.validate()
cfg.check_required_args()
# OR
cfg.apply(["--a", "1"]) # shorthand for the above
```
### Groups
Config groups can be very handy to make components interchangeable without requiring them to share a config.
```python
from hydralette import Config, Field
cfg = Config(
model=Config(
_groups={
"_default": "rnn",
"rnn": Config(
n_layers=2,
bidirectional=False
),
"transformer": Config(
n_layers=16,
num_attention_heads=8
)
}
)
)
cfg.apply(["--model", "transformer"])
```
### From signature / existing config
Often times, part of your configuration is already implemented somewhere else and duplicating this information creates a source of failure. Instead, you can automatically generate your hydralette `Config` based on an existing interface
```python
def calc(a: int, b=2):
pass
from hydralette import Config, Field
cfg = Config(_from_signature=calc)
# equivalent to
cfg = Config(a=Field(type=int), b=Field(default=2, type=int))
```
### Fields
When you directly pass a value to `Config`s constructor, hydralette will create a `Field` under the hood. To use additional features, you can create it explicitly
`convert`: Specify how command-line overrides should be converted to the target type. If not explicitly specified, hydralette tries to use the field's `type` as conversion function (`type` is either explicitly specified or automatically derived from `default` / `default_factory`).
```python
import json
from hydralette import Config, Field
cfg = Config(my_dict=Field(default={"a": 1, "b": {"c": 2}}, convert=json.loads))
cfg.apply(['--my_dict', r'{"a": 2, "b": {"c": 3}}'])
```
`validate`: Constrain what values are valid for your field. If the validity of a value depends on the rest of the config, use `_validate` in the `Config` constructor instead.
```python
from hydralette import Config, Field
cfg = Config(n=Field(default=1, validate=lambda n: n > 0))
cfg.apply(['--n', '-1'])
# throws: ValidationError: Field validation failed for -1
cfg = Config(_validate=lambda cfg: cfg.a > cfg.b, a=1, b=2)
cfg.apply() # or cfg.validate()
# throws: ValidationError: Config validation failed for {'a': 1, 'b': 2}
```
`reference` / `reference_root`: Refer to any other value in the config
```python
from hydralette import Config, Field
from pathlib import Path
cfg = Config(
dir=Path("outputs"),
train=Config(
checkpoint_dir=Field(reference_root=lambda cfg: cfg.dir / "checkpoints"), # relative to current config
metrics_dir=Field(reference=lambda cfg: cfg.checkpoint_dir.parent / "metrics") # relative to root config
)
)
cfg.resolve_references()
```
## Backlog
- [x] CLI
- [x] groups
- [x] from signatures
- [x] validation
- [x] conversion
- [x] references
- [x] yaml representation
- [x] pickle serialization
- [x] allow combining _groups, _fields and _from_signature
- [x] special support for boolean flags in CLI
## Dev Info
Steps on new release:
1. Run tests `pytest`
2. Edit docs
3. Increment version in `pyproject.toml`
4. Add changelog to `README.md`
5. Push increment to GitHub
6. Publish to PyPI `poetry publish --build`
7. Publish docs `mkdocs gh-deploy`
8. Create release and tag on GitHub
Raw data
{
"_id": null,
"home_page": "https://github.com/ValeKnappich/hydralette",
"name": "hydralette",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "config,cfg,hydra",
"author": "ValeKnappich",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/c4/b8/a9579ae185c4dc82952e92f27d33bbb9ad254421f7c21149fb257fb37c88/hydralette-0.2.3.tar.gz",
"platform": null,
"description": "<div align=\"center\" markdown=\"1\">\n\n# Hydralette\n\n**Create complex configurations in a simple, pythonic way!**\n\n[](https://github.com/pre-commit/pre-commit)\n[](https://black.readthedocs.io/en/stable/)\n[](https://pycqa.github.io/isort/)\n[](https://github.com/ashleve/lightning-hydra-template#license)\n\n<a href=\"https://valeknappich.github.io/hydralette\" style=\"font-size: 14pt\">Documentation</a>\n\n</div>\n\n## Installation\n\n```bash\npip install hydralette\n# OR\npoetry add hydralette\n```\n\n## Changelog\n\n- v0.2.3\n - change version of `dill` dependency\n- v0.2.2\n - support mixing _fields, _groups and _from_signature\n - support literal values as groups\n - add boolean flags as `--flag` or `--no-flag`\n- v0.2.1\n - fix yaml representation\n - support setting value to None in automatic conversion\n- v0.2.0\n - complete re-design --> breaking changes!\n - easier creation of hierarchical configs\n- v0.1.5\n - Add yaml overrides\n - Add end-to-end tests with pytest\n- v0.1.4\n - Add documentation at [https://valeknappich.github.io/hydralette](https://valeknappich.github.io/hydralette)\n- v0.1.3\n - Add automatic generation of configs from signatures via `from_signature` and `config_from_signature`\n- v0.1.2\n - Add support for `config groups`, `references`, `validation` and `type conversion`\n - Add CLI help pages\n\n\n## Features\n\n### Basics\n\nCreate a config using the `Config` class, passing the config fields as keyword arguments.\n\n```python\nfrom hydralette import Config\ncfg = Config(a=1, b=2, c=\"abc\")\n```\n\n`Config` objects can be create from and exported to `dict`s.\n\n```python\nfrom hydralette import Config\ncfg = Config.from_dict({\"a\": 1, \"b\": 2, \"c\": \"abc\"})\nprint(cfg.to_dict())\n```\n\nEvery hydralette `Config` be overriden via CLI:\n\n```python\nfrom hydralette import Config, Field\ncfg = Config(\n a=Field(default=1, help=\"Lorem ipsum\"),\n b=Config(\n c=Field(default=2, help=\"Lorem ipsum\")\n )\n)\ncfg.apply([\"--help\"]) # or cfg.print_help()\n\n# prints:\n\n# Usage python script.py [OPTIONS]\n# --a 1 Lorem ipsum\n# --b.c 1 Lorem ipsum\n```\n\nAfter creation, you can override, resolve references, validate and check for missing required arguments.\n\n```python\ncfg.override([\"--a\", \"1\"]) # overrides defaults to sys.argv[1:]\ncfg.resolve_references()\ncfg.validate()\ncfg.check_required_args()\n# OR\ncfg.apply([\"--a\", \"1\"]) # shorthand for the above\n```\n\n\n### Groups\n\nConfig groups can be very handy to make components interchangeable without requiring them to share a config.\n\n```python\nfrom hydralette import Config, Field\ncfg = Config(\n model=Config(\n _groups={\n \"_default\": \"rnn\",\n \"rnn\": Config(\n n_layers=2,\n bidirectional=False\n ),\n \"transformer\": Config(\n n_layers=16,\n num_attention_heads=8\n )\n }\n )\n)\ncfg.apply([\"--model\", \"transformer\"])\n```\n\n### From signature / existing config\n\nOften times, part of your configuration is already implemented somewhere else and duplicating this information creates a source of failure. Instead, you can automatically generate your hydralette `Config` based on an existing interface\n\n```python\n\ndef calc(a: int, b=2):\n pass\n\nfrom hydralette import Config, Field\ncfg = Config(_from_signature=calc)\n# equivalent to\ncfg = Config(a=Field(type=int), b=Field(default=2, type=int))\n```\n\n\n### Fields\n\nWhen you directly pass a value to `Config`s constructor, hydralette will create a `Field` under the hood. To use additional features, you can create it explicitly\n\n`convert`: Specify how command-line overrides should be converted to the target type. If not explicitly specified, hydralette tries to use the field's `type` as conversion function (`type` is either explicitly specified or automatically derived from `default` / `default_factory`).\n\n```python\nimport json\nfrom hydralette import Config, Field\ncfg = Config(my_dict=Field(default={\"a\": 1, \"b\": {\"c\": 2}}, convert=json.loads))\ncfg.apply(['--my_dict', r'{\"a\": 2, \"b\": {\"c\": 3}}'])\n```\n\n`validate`: Constrain what values are valid for your field. If the validity of a value depends on the rest of the config, use `_validate` in the `Config` constructor instead.\n\n```python\nfrom hydralette import Config, Field\ncfg = Config(n=Field(default=1, validate=lambda n: n > 0))\ncfg.apply(['--n', '-1'])\n# throws: ValidationError: Field validation failed for -1\ncfg = Config(_validate=lambda cfg: cfg.a > cfg.b, a=1, b=2)\ncfg.apply() # or cfg.validate()\n# throws: ValidationError: Config validation failed for {'a': 1, 'b': 2}\n```\n\n`reference` / `reference_root`: Refer to any other value in the config\n\n```python\nfrom hydralette import Config, Field\nfrom pathlib import Path\ncfg = Config(\n dir=Path(\"outputs\"),\n train=Config(\n checkpoint_dir=Field(reference_root=lambda cfg: cfg.dir / \"checkpoints\"), # relative to current config\n metrics_dir=Field(reference=lambda cfg: cfg.checkpoint_dir.parent / \"metrics\") # relative to root config\n )\n)\ncfg.resolve_references()\n```\n\n\n\n\n\n## Backlog\n\n- [x] CLI\n- [x] groups\n- [x] from signatures\n- [x] validation\n- [x] conversion\n- [x] references\n- [x] yaml representation\n- [x] pickle serialization\n- [x] allow combining _groups, _fields and _from_signature\n- [x] special support for boolean flags in CLI\n\n## Dev Info\n\nSteps on new release:\n\n1. Run tests `pytest`\n2. Edit docs\n3. Increment version in `pyproject.toml`\n4. Add changelog to `README.md`\n5. Push increment to GitHub\n6. Publish to PyPI `poetry publish --build`\n7. Publish docs `mkdocs gh-deploy`\n8. Create release and tag on GitHub",
"bugtrack_url": null,
"license": "MIT",
"summary": "",
"version": "0.2.3",
"project_urls": {
"Homepage": "https://github.com/ValeKnappich/hydralette",
"Repository": "https://github.com/ValeKnappich/hydralette"
},
"split_keywords": [
"config",
"cfg",
"hydra"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c39d6b4a8965574b169d7a3d52cc8045924dc2e7d698836be2a3e5fbb157241d",
"md5": "4b87edf884ee2c5d3ac738fdb3a1e6fb",
"sha256": "4d708ea0c8bbbb5724d1ddd8b726479766ab2f6d5d459865b7d4d3c61beddc37"
},
"downloads": -1,
"filename": "hydralette-0.2.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4b87edf884ee2c5d3ac738fdb3a1e6fb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 9631,
"upload_time": "2024-02-08T15:26:17",
"upload_time_iso_8601": "2024-02-08T15:26:17.686765Z",
"url": "https://files.pythonhosted.org/packages/c3/9d/6b4a8965574b169d7a3d52cc8045924dc2e7d698836be2a3e5fbb157241d/hydralette-0.2.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c4b8a9579ae185c4dc82952e92f27d33bbb9ad254421f7c21149fb257fb37c88",
"md5": "2b5ab3adcb42f9544d6607ab625b83b7",
"sha256": "ec5b3eb87a1a8c5c71b055b2b8de016176f67f1cdd5d51e1381ad00010f1f02f"
},
"downloads": -1,
"filename": "hydralette-0.2.3.tar.gz",
"has_sig": false,
"md5_digest": "2b5ab3adcb42f9544d6607ab625b83b7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 8407,
"upload_time": "2024-02-08T15:26:18",
"upload_time_iso_8601": "2024-02-08T15:26:18.905182Z",
"url": "https://files.pythonhosted.org/packages/c4/b8/a9579ae185c4dc82952e92f27d33bbb9ad254421f7c21149fb257fb37c88/hydralette-0.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-08 15:26:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ValeKnappich",
"github_project": "hydralette",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "hydralette"
}
JSON
