# Neural Pipeline Search (NePS)
[](https://pypi.org/project/neural-pipeline-search/)
[](https://pypi.org/project/neural-pipeline-search/)
[](LICENSE)
[](https://github.com/automl/neps/actions)
Welcome to NePS, a powerful and flexible Python library for hyperparameter optimization (HPO) and neural architecture search (NAS) with its primary goal: enable HPO adoption in practice for deep learners!
NePS houses recently published and some more well-established algorithms that are all capable of being run massively parallel on any distributed setup, with tools to analyze runs, restart runs, etc.
Take a look at our [documentation](https://automl.github.io/neps/latest/) and continue following through current README for instructions on how to use NePS!
## Key Features
In addition to the common features offered by traditional HPO and NAS libraries, NePS stands out with the following key features:
1. [**Hyperparameter Optimization (HPO) With Prior Knowledge:**](neps_examples/template/priorband_template.py)
- NePS excels in efficiently tuning hyperparameters using algorithms that enable users to make use of their prior knowledge within the search space. This is leveraged by the insights presented in:
- [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://arxiv.org/abs/2306.12370)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://arxiv.org/abs/2204.11051)
2. [**Neural Architecture Search (NAS) With Context-free Grammar Search Spaces:**](neps_examples/basic_usage/architecture.py)
- NePS is equipped to handle context-free grammar search spaces, providing advanced capabilities for designing and optimizing architectures. this is leveraged by the insights presented in:
- [Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars](https://arxiv.org/abs/2211.01842)
3. [**Easy Parallelization and Resumption of Runs:**](docs/parallelization.md)
- NePS simplifies the process of parallelizing optimization tasks both on individual computers and in distributed
computing environments. It also allows users to conveniently resume these optimization tasks after completion to
ensure a seamless and efficient workflow for long-running experiments.
4. [**Seamless User Code Integration:**](neps_examples/template/)
- NePS's modular design ensures flexibility and extensibility. Integrate NePS effortlessly into existing machine learning workflows.
## Getting Started
### 1. Installation
Using pip:
```bash
pip install neural-pipeline-search
```
> Note: As indicated with the `v0.x.x` version number, NePS is early stage code and APIs might change in the future.
### 2. Basic Usage
Using `neps` always follows the same pattern:
1. Define a `run_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations
for your problem.
2. Define a search space named `pipeline_space` of those Parameters e.g. via a dictionary
3. Call `neps.run` to optimize `run_pipeline` over `pipeline_space`
In code, the usage pattern can look like this:
```python
import neps
import logging
# 1. Define a function that accepts hyperparameters and computes the validation error
def run_pipeline(
hyperparameter_a: float, hyperparameter_b: int, architecture_parameter: str
) -> dict:
# Create your model
model = MyModel(architecture_parameter)
# Train and evaluate the model with your training pipeline
validation_error, training_error = train_and_eval(
model, hyperparameter_a, hyperparameter_b
)
return { # dict or float(validation error)
"loss": validation_error,
"info_dict": {
"training_error": training_error
# + Other metrics
},
}
# 2. Define a search space of parameters; use the same names for the parameters as in run_pipeline
pipeline_space = dict(
hyperparameter_b=neps.IntegerParameter(
lower=1, upper=42, is_fidelity=True
), # Mark 'is_fidelity' as true for a multi-fidelity approach.
hyperparameter_a=neps.FloatParameter(
lower=0.001, upper=0.1, log=True
), # If True, the search space is sampled in log space.
architecture_parameter=neps.CategoricalParameter(
["option_a", "option_b", "option_c"]
),
)
if __name__ == "__main__":
# 3. Run the NePS optimization
logging.basicConfig(level=logging.INFO)
neps.run(
run_pipeline=run_pipeline,
pipeline_space=pipeline_space,
root_directory="path/to/save/results", # Replace with the actual path.
max_evaluations_total=100,
searcher="hyperband" # Optional specifies the search strategy,
# otherwise NePs decides based on your data.
)
```
## Examples
Discover how NePS works through these practical examples:
* **[Pipeline Space via YAML](neps_examples/basic_usage/defining_search_space)**: Explore how to define the `pipeline_space` using a
YAML file instead of a dictionary.
* **[Hyperparameter Optimization (HPO)](neps_examples/basic_usage/hyperparameters.py)**: Learn the essentials of hyperparameter optimization with NePS.
* **[Architecture Search with Primitives](neps_examples/basic_usage/architecture.py)**: Dive into architecture search using primitives in NePS.
* **[Multi-Fidelity Optimization](neps_examples/efficiency/multi_fidelity.py)**: Understand how to leverage multi-fidelity optimization for efficient model tuning.
* **[Utilizing Expert Priors for Hyperparameters](neps_examples/efficiency/expert_priors_for_hyperparameters.py)**: Learn how to incorporate expert priors for more efficient hyperparameter selection.
* **[Additional NePS Examples](neps_examples/)**: Explore more examples, including various use cases and advanced configurations in NePS.
## Documentation
For more details and features please have a look at our [documentation](https://automl.github.io/neps/latest/)
## Analysing runs
See our [documentation on analysing runs](https://automl.github.io/neps/latest/analyse).
## Contributing
Please see the [documentation for contributors](https://automl.github.io/neps/latest/contributing/).
## Citations
Please consider citing us if you use our tool!
Refer to our [documentation on citations](https://automl.github.io/neps/latest/citations/).
## Alternatives
NePS does not cover your use-case? Have a look at [some alternatives](https://automl.github.io/neps/latest/alternatives).
Raw data
{
"_id": null,
"home_page": "https://github.com/automl/neps",
"name": "neural-pipeline-search",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.12",
"maintainer_email": "",
"keywords": "Neural Pipeline Search,Neural Architecture Search,Hyperparameter Optimization,AutoML",
"author": "Danny Stoll",
"author_email": "stolld@cs.uni-freiburg.de",
"download_url": "https://files.pythonhosted.org/packages/53/92/d4c0ec7526dfa880e2041fa099f4f626ad0381b7a382fc0a841c757a2654/neural_pipeline_search-0.11.1.tar.gz",
"platform": null,
"description": "# Neural Pipeline Search (NePS)\n\n[](https://pypi.org/project/neural-pipeline-search/)\n[](https://pypi.org/project/neural-pipeline-search/)\n[](LICENSE)\n[](https://github.com/automl/neps/actions)\n\nWelcome to NePS, a powerful and flexible Python library for hyperparameter optimization (HPO) and neural architecture search (NAS) with its primary goal: enable HPO adoption in practice for deep learners!\n\nNePS houses recently published and some more well-established algorithms that are all capable of being run massively parallel on any distributed setup, with tools to analyze runs, restart runs, etc.\n\nTake a look at our [documentation](https://automl.github.io/neps/latest/) and continue following through current README for instructions on how to use NePS!\n\n\n## Key Features\n\nIn addition to the common features offered by traditional HPO and NAS libraries, NePS stands out with the following key features:\n\n1. [**Hyperparameter Optimization (HPO) With Prior Knowledge:**](neps_examples/template/priorband_template.py)\n - NePS excels in efficiently tuning hyperparameters using algorithms that enable users to make use of their prior knowledge within the search space. This is leveraged by the insights presented in:\n - [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://arxiv.org/abs/2306.12370)\n - [\u03c0BO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://arxiv.org/abs/2204.11051)\n\n2. [**Neural Architecture Search (NAS) With Context-free Grammar Search Spaces:**](neps_examples/basic_usage/architecture.py)\n - NePS is equipped to handle context-free grammar search spaces, providing advanced capabilities for designing and optimizing architectures. this is leveraged by the insights presented in:\n - [Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars](https://arxiv.org/abs/2211.01842)\n\n3. [**Easy Parallelization and Resumption of Runs:**](docs/parallelization.md)\n - NePS simplifies the process of parallelizing optimization tasks both on individual computers and in distributed\n computing environments. It also allows users to conveniently resume these optimization tasks after completion to\n ensure a seamless and efficient workflow for long-running experiments.\n\n4. [**Seamless User Code Integration:**](neps_examples/template/)\n - NePS's modular design ensures flexibility and extensibility. Integrate NePS effortlessly into existing machine learning workflows.\n\n## Getting Started\n\n### 1. Installation\n\nUsing pip:\n\n```bash\npip install neural-pipeline-search\n```\n\n> Note: As indicated with the `v0.x.x` version number, NePS is early stage code and APIs might change in the future.\n\n### 2. Basic Usage\n\nUsing `neps` always follows the same pattern:\n\n1. Define a `run_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations\n for your problem.\n2. Define a search space named `pipeline_space` of those Parameters e.g. via a dictionary\n3. Call `neps.run` to optimize `run_pipeline` over `pipeline_space`\n\nIn code, the usage pattern can look like this:\n\n```python\nimport neps\nimport logging\n\n\n# 1. Define a function that accepts hyperparameters and computes the validation error\ndef run_pipeline(\n hyperparameter_a: float, hyperparameter_b: int, architecture_parameter: str\n) -> dict:\n # Create your model\n model = MyModel(architecture_parameter)\n\n # Train and evaluate the model with your training pipeline\n validation_error, training_error = train_and_eval(\n model, hyperparameter_a, hyperparameter_b\n )\n\n return { # dict or float(validation error)\n \"loss\": validation_error,\n \"info_dict\": {\n \"training_error\": training_error\n # + Other metrics\n },\n }\n\n\n# 2. Define a search space of parameters; use the same names for the parameters as in run_pipeline\npipeline_space = dict(\n hyperparameter_b=neps.IntegerParameter(\n lower=1, upper=42, is_fidelity=True\n ), # Mark 'is_fidelity' as true for a multi-fidelity approach.\n hyperparameter_a=neps.FloatParameter(\n lower=0.001, upper=0.1, log=True\n ), # If True, the search space is sampled in log space.\n architecture_parameter=neps.CategoricalParameter(\n [\"option_a\", \"option_b\", \"option_c\"]\n ),\n)\n\nif __name__ == \"__main__\":\n # 3. Run the NePS optimization\n logging.basicConfig(level=logging.INFO)\n neps.run(\n run_pipeline=run_pipeline,\n pipeline_space=pipeline_space,\n root_directory=\"path/to/save/results\", # Replace with the actual path.\n max_evaluations_total=100,\n searcher=\"hyperband\" # Optional specifies the search strategy,\n # otherwise NePs decides based on your data.\n )\n```\n\n## Examples\n\nDiscover how NePS works through these practical examples:\n* **[Pipeline Space via YAML](neps_examples/basic_usage/defining_search_space)**: Explore how to define the `pipeline_space` using a\n YAML file instead of a dictionary.\n\n* **[Hyperparameter Optimization (HPO)](neps_examples/basic_usage/hyperparameters.py)**: Learn the essentials of hyperparameter optimization with NePS.\n\n* **[Architecture Search with Primitives](neps_examples/basic_usage/architecture.py)**: Dive into architecture search using primitives in NePS.\n\n* **[Multi-Fidelity Optimization](neps_examples/efficiency/multi_fidelity.py)**: Understand how to leverage multi-fidelity optimization for efficient model tuning.\n\n* **[Utilizing Expert Priors for Hyperparameters](neps_examples/efficiency/expert_priors_for_hyperparameters.py)**: Learn how to incorporate expert priors for more efficient hyperparameter selection.\n\n* **[Additional NePS Examples](neps_examples/)**: Explore more examples, including various use cases and advanced configurations in NePS.\n\n## Documentation\n\nFor more details and features please have a look at our [documentation](https://automl.github.io/neps/latest/)\n\n## Analysing runs\n\nSee our [documentation on analysing runs](https://automl.github.io/neps/latest/analyse).\n\n## Contributing\n\nPlease see the [documentation for contributors](https://automl.github.io/neps/latest/contributing/).\n\n## Citations\n\nPlease consider citing us if you use our tool!\n\nRefer to our [documentation on citations](https://automl.github.io/neps/latest/citations/).\n\n## Alternatives\n\nNePS does not cover your use-case? Have a look at [some alternatives](https://automl.github.io/neps/latest/alternatives).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Neural Pipeline Search helps deep learning experts find the best neural pipeline.",
"version": "0.11.1",
"project_urls": {
"Documentation": "https://automl.github.io/neps/",
"Homepage": "https://github.com/automl/neps",
"Repository": "https://github.com/automl/neps"
},
"split_keywords": [
"neural pipeline search",
"neural architecture search",
"hyperparameter optimization",
"automl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "eb9c295793eceee267fe686c99c55599bc2978ed80a447af39e13c1c178ce377",
"md5": "8697386ee5e3097564e9a799e3cae338",
"sha256": "58712f9ea851410eb6269deed0a471aada0a3676abcb8297b89b80081c5b352f"
},
"downloads": -1,
"filename": "neural_pipeline_search-0.11.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8697386ee5e3097564e9a799e3cae338",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.12",
"size": 301364,
"upload_time": "2024-02-19T18:13:13",
"upload_time_iso_8601": "2024-02-19T18:13:13.890104Z",
"url": "https://files.pythonhosted.org/packages/eb/9c/295793eceee267fe686c99c55599bc2978ed80a447af39e13c1c178ce377/neural_pipeline_search-0.11.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5392d4c0ec7526dfa880e2041fa099f4f626ad0381b7a382fc0a841c757a2654",
"md5": "3c6b4b4a9abcba3549001e44e29f2834",
"sha256": "e118fc4173e00fd5008dfd37aee77c54bed4bee4dcfeaedb6880bd389a0161c0"
},
"downloads": -1,
"filename": "neural_pipeline_search-0.11.1.tar.gz",
"has_sig": false,
"md5_digest": "3c6b4b4a9abcba3549001e44e29f2834",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.12",
"size": 232247,
"upload_time": "2024-02-19T18:13:15",
"upload_time_iso_8601": "2024-02-19T18:13:15.595254Z",
"url": "https://files.pythonhosted.org/packages/53/92/d4c0ec7526dfa880e2041fa099f4f626ad0381b7a382fc0a841c757a2654/neural_pipeline_search-0.11.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-19 18:13:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "automl",
"github_project": "neps",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "neural-pipeline-search"
}
JSON
