| Name | hydra-filter-sweeper JSON |
| Version |
1.1.0
JSON |
| download |
| home_page | None |
| Summary | A Hydra plugin to extend the basic sweeper with customizable filters. |
| upload_time | 2024-09-25 14:35:30 |
| maintainer | None |
| docs_url | None |
| author | Simon Rampp |
| requires_python | <4.0,>=3.9 |
| license | MIT |
| keywords |
hydra
plugin
sweeper
filter
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Hydra Filter Sweeper Plugin
[](https://pypi.org/project/hydra-filter-sweeper/)
[](https://pypi.org/project/hydra-filter-sweeper/)
[](https://github.com/autrainer/hydra-filter-sweeper/blob/main/LICENSE)
`hydra-filter-sweeper` is a plugin for [Hydra](https://hydra.cc/) that extends the [basic sweeper](https://hydra.cc/docs/tutorials/basic/running_your_app/multi-run/#sweeper) by the addition of filters, enabling more targeted parameter sweeps.
The plugin is compatible with any [Hydra launcher plugin](https://hydra.cc/docs/tutorials/basic/running_your_app/multi-run/#launcher).
The minimum required Hydra version is `1.3.2`.
## Features
**Customizable Filters**
- Apply [expressions](#expression-filter-expr), [existence checks](#exists-filter-exists), or [custom filter classes](#class-filter-class) to the sweep.
**Flexible Filter Conditions**
- Include fail-safe conditions to gracefully handle possible exceptions raised by filters.
**Interpolation Support**
- Utilize [OmegaConf's interpolation syntax](https://omegaconf.readthedocs.io/en/latest/usage.html#variable-interpolation) to reference configuration values.
## Installation
To install the plugin, use pip:
```bash
pip install hydra-filter-sweeper
```
## Usage
To use `hydra-filter-sweeper`, override the default sweeper with `filter` at the end of the defaults list.
Filters are specified as a list of dictionaries and can be of type `expr`, `exists`, or `class`.
If any filter evaluates to `True`, the current configuration **is excluded** from the sweep.
If no `filters` list is provided or all filters evaluate to `False`, all configurations are included and the
sweeper resembles the default behavior of Hydra's basic sweeper.
**Example Configuration**
```yaml
defaults:
- _self_
- override hydra/sweeper: filter
some_value: four
hydra:
mode: MULTIRUN
sweeper:
params:
+foo: 1,2,3
+bar: one, two, three
filters:
- type: expr
expr: foo == 1 and bar == "two"
- type: exists
path: some_directory/some.file
- type: class
target: some_filter.SomeFilter
some_arg: ${some_value}
```
## Filters
### Expression Filter (`expr`)
Filter configurations based on a Python expression that evaluates to `True` or `False`.
The context of the expression is the configuration itself.
The configuration is excluded if the expression evaluates to `True`.
**Parameters**:
- `expr` (_str_): Python expression to evaluate.
- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.
- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.
**Example Configuration**
```yaml
hydra/sweeper/filters:
- type: expr
expr: foo == 1 and bar == "two"
- type: expr
expr: bar == ${some_value}
- type: expr
expr: undefined == 1 and bar == "two"
fail: false
log: false
```
### Exists Filter (`exists`)
Checks if a specified file or directory exists in the run's directory.
The configuration is excluded if the file or directory exists.
**Parameters**:
- `path` (_str_): Path to the file or directory to check if it exists in the run's directory.
- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.
- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.
**Example Configuration**
```yaml
hydra/sweeper/filters:
- type: exists
path: some_directory/some.file
- type: exists
path: some_directory
log: false
- type: exists
path: some_directory/${some_value}.file
- type: exists
path: null
fail: false
```
### Class Filter (`class`)
Applies a custom filter class to the sweep.
The configuration is excluded if the filter method returns `True`.
**Parameters**:
- `target` (_str_): Python relative import path to the class.
- `*` (_Any_): Additional keyword arguments passed to the filter method of the class.
- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.
- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.
**Example Configuration**
```yaml
hydra/sweeper/filters:
- type: class
target: some_filter.SomeFilter
some_arg: ${some_value}
log: false
- type: class
target: some_filter.NonExistentFilter
some_arg: ${some_value}
fail: false
```
The `SomeFilter` class should inherit from `AbstractFilter` and implement the `filter`
method that returns `True` if the configuration should be excluded.
The `filter` method receives the configuration, the run's directory, and any additional keyword arguments as parameters.
```python
from omegaconf import DictConfig
from hydra_filter_sweeper import AbstractFilter
class SomeFilter(AbstractFilter):
def filter(self, config: DictConfig, directory: str, some_arg: str) -> bool:
return config.foo == 1 and config.bar == "two" and some_arg == "four"
```
## Contributing
Contributions are welcome!
For bug reports or requests, please [submit an issue](https://github.com/autrainer/hydra-filter-sweeper/issues).
To contribute, please fork the repository and submit a [pull request](https://github.com/autrainer/hydra-filter-sweeper/pulls).
We use [Poetry](https://python-poetry.org/) for dependency management,
[Ruff](https://astral.sh/ruff) for code formatting,
[codespell](https://github.com/codespell-project/codespell) for spell checking,
[pytest](https://docs.pytest.org/en/stable/) for testing,
and [pre-commit](https://pre-commit.com/) for managing the hooks.
Install the development dependencies with:
```bash
poetry install
pre-commit install
```
Both formatting and spell checking are enforced by pre-commit hooks.
We strive for 100% test coverage. To run the tests locally, use:
```bash
pytest
```
Raw data
{
"_id": null,
"home_page": null,
"name": "hydra-filter-sweeper",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "hydra, plugin, sweeper, filter",
"author": "Simon Rampp",
"author_email": "simon.rampp@tum.de",
"download_url": "https://files.pythonhosted.org/packages/60/32/cd35dbddf6fad0cb71f2e1640053190ca6bd151ae0e795e51d974c1102bd/hydra_filter_sweeper-1.1.0.tar.gz",
"platform": null,
"description": "# Hydra Filter Sweeper Plugin\n\n[](https://pypi.org/project/hydra-filter-sweeper/)\n[](https://pypi.org/project/hydra-filter-sweeper/)\n[](https://github.com/autrainer/hydra-filter-sweeper/blob/main/LICENSE)\n\n`hydra-filter-sweeper` is a plugin for [Hydra](https://hydra.cc/) that extends the [basic sweeper](https://hydra.cc/docs/tutorials/basic/running_your_app/multi-run/#sweeper) by the addition of filters, enabling more targeted parameter sweeps.\n\nThe plugin is compatible with any [Hydra launcher plugin](https://hydra.cc/docs/tutorials/basic/running_your_app/multi-run/#launcher).\nThe minimum required Hydra version is `1.3.2`.\n\n## Features\n\n**Customizable Filters**\n\n- Apply [expressions](#expression-filter-expr), [existence checks](#exists-filter-exists), or [custom filter classes](#class-filter-class) to the sweep.\n\n**Flexible Filter Conditions**\n\n- Include fail-safe conditions to gracefully handle possible exceptions raised by filters.\n\n**Interpolation Support**\n\n- Utilize [OmegaConf's interpolation syntax](https://omegaconf.readthedocs.io/en/latest/usage.html#variable-interpolation) to reference configuration values.\n\n## Installation\n\nTo install the plugin, use pip:\n\n```bash\npip install hydra-filter-sweeper\n```\n\n## Usage\n\nTo use `hydra-filter-sweeper`, override the default sweeper with `filter` at the end of the defaults list.\n\nFilters are specified as a list of dictionaries and can be of type `expr`, `exists`, or `class`.\n\nIf any filter evaluates to `True`, the current configuration **is excluded** from the sweep.\n\nIf no `filters` list is provided or all filters evaluate to `False`, all configurations are included and the\nsweeper resembles the default behavior of Hydra's basic sweeper.\n\n**Example Configuration**\n\n```yaml\ndefaults:\n - _self_\n - override hydra/sweeper: filter\n\nsome_value: four\n\nhydra:\n mode: MULTIRUN\n sweeper:\n params:\n +foo: 1,2,3\n +bar: one, two, three\n filters:\n - type: expr\n expr: foo == 1 and bar == \"two\"\n - type: exists\n path: some_directory/some.file\n - type: class\n target: some_filter.SomeFilter\n some_arg: ${some_value}\n```\n\n## Filters\n\n### Expression Filter (`expr`)\n\nFilter configurations based on a Python expression that evaluates to `True` or `False`.\nThe context of the expression is the configuration itself.\nThe configuration is excluded if the expression evaluates to `True`.\n\n**Parameters**:\n\n- `expr` (_str_): Python expression to evaluate.\n- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.\n- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.\n\n**Example Configuration**\n\n```yaml\nhydra/sweeper/filters:\n - type: expr\n expr: foo == 1 and bar == \"two\"\n - type: expr\n expr: bar == ${some_value}\n - type: expr\n expr: undefined == 1 and bar == \"two\"\n fail: false\n log: false\n```\n\n### Exists Filter (`exists`)\n\nChecks if a specified file or directory exists in the run's directory.\nThe configuration is excluded if the file or directory exists.\n\n**Parameters**:\n\n- `path` (_str_): Path to the file or directory to check if it exists in the run's directory.\n- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.\n- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.\n\n**Example Configuration**\n\n```yaml\nhydra/sweeper/filters:\n - type: exists\n path: some_directory/some.file\n - type: exists\n path: some_directory\n log: false\n - type: exists\n path: some_directory/${some_value}.file\n - type: exists\n path: null\n fail: false\n```\n\n### Class Filter (`class`)\n\nApplies a custom filter class to the sweep.\nThe configuration is excluded if the filter method returns `True`.\n\n**Parameters**:\n\n- `target` (_str_): Python relative import path to the class.\n- `*` (_Any_): Additional keyword arguments passed to the filter method of the class.\n- `fail` (_bool_): Whether to fail if the filter raises an exception. Default is `True`.\n- `log` (_bool_): Whether to log the configuration if the filter evaluates to `True`. Default is `True`.\n\n**Example Configuration**\n\n```yaml\nhydra/sweeper/filters:\n - type: class\n target: some_filter.SomeFilter\n some_arg: ${some_value}\n log: false\n - type: class\n target: some_filter.NonExistentFilter\n some_arg: ${some_value}\n fail: false\n```\n\nThe `SomeFilter` class should inherit from `AbstractFilter` and implement the `filter`\nmethod that returns `True` if the configuration should be excluded.\n\nThe `filter` method receives the configuration, the run's directory, and any additional keyword arguments as parameters.\n\n```python\nfrom omegaconf import DictConfig\n\nfrom hydra_filter_sweeper import AbstractFilter\n\n\nclass SomeFilter(AbstractFilter):\n def filter(self, config: DictConfig, directory: str, some_arg: str) -> bool:\n return config.foo == 1 and config.bar == \"two\" and some_arg == \"four\"\n```\n\n## Contributing\n\nContributions are welcome!\nFor bug reports or requests, please [submit an issue](https://github.com/autrainer/hydra-filter-sweeper/issues).\n\nTo contribute, please fork the repository and submit a [pull request](https://github.com/autrainer/hydra-filter-sweeper/pulls).\n\nWe use [Poetry](https://python-poetry.org/) for dependency management,\n[Ruff](https://astral.sh/ruff) for code formatting,\n[codespell](https://github.com/codespell-project/codespell) for spell checking,\n[pytest](https://docs.pytest.org/en/stable/) for testing,\nand [pre-commit](https://pre-commit.com/) for managing the hooks.\n\nInstall the development dependencies with:\n\n```bash\npoetry install\npre-commit install\n```\n\nBoth formatting and spell checking are enforced by pre-commit hooks.\n\nWe strive for 100% test coverage. To run the tests locally, use:\n\n```bash\npytest\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Hydra plugin to extend the basic sweeper with customizable filters.",
"version": "1.1.0",
"project_urls": null,
"split_keywords": [
"hydra",
" plugin",
" sweeper",
" filter"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7ad83ce40ba804896de7712157eb235e03d04301c4c2ba1812b99295c9ba70a8",
"md5": "309089b04b3af2d174413c51698a5672",
"sha256": "52ec5d71b45c058e67efc529923cd4e94823c93ec750b5550c0929c26dd9c1e9"
},
"downloads": -1,
"filename": "hydra_filter_sweeper-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "309089b04b3af2d174413c51698a5672",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 6941,
"upload_time": "2024-09-25T14:35:29",
"upload_time_iso_8601": "2024-09-25T14:35:29.098184Z",
"url": "https://files.pythonhosted.org/packages/7a/d8/3ce40ba804896de7712157eb235e03d04301c4c2ba1812b99295c9ba70a8/hydra_filter_sweeper-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6032cd35dbddf6fad0cb71f2e1640053190ca6bd151ae0e795e51d974c1102bd",
"md5": "9dab75e059bb0b3a252bfc594db55245",
"sha256": "099f6f57abcdc2763bb4f4bed41d8a9f0f228ec536793aff15d832f3ca9399b0"
},
"downloads": -1,
"filename": "hydra_filter_sweeper-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "9dab75e059bb0b3a252bfc594db55245",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 6234,
"upload_time": "2024-09-25T14:35:30",
"upload_time_iso_8601": "2024-09-25T14:35:30.504961Z",
"url": "https://files.pythonhosted.org/packages/60/32/cd35dbddf6fad0cb71f2e1640053190ca6bd151ae0e795e51d974c1102bd/hydra_filter_sweeper-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-25 14:35:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "hydra-filter-sweeper"
}
