<p align="center">
<a href="https://evotorch.ai" rel="nofollow">
<img src="https://raw.githubusercontent.com/nnaisense/evotorch/master/docs/assets/evotorch.svg" />
</a>
</p>
<div align="center">
<a href="https://www.python.org/" rel="nofollow">
<img src="https://img.shields.io/pypi/pyversions/evotorch" alt="Python" />
</a>
<a href="https://pypi.org/project/evotorch/" rel="nofollow">
<img src="https://img.shields.io/pypi/v/evotorch" alt="PyPI" />
</a>
<a href="https://github.com/nnaisense/evotorch/blob/master/LICENSE" rel="nofollow">
<img src="https://img.shields.io/pypi/l/evotorch" alt="License" />
</a>
<a href="https://docs.evotorch.ai" rel="nofollow">
<img src="https://github.com/nnaisense/evotorch/actions/workflows/docs.yaml/badge.svg" alt="Build" />
</a>
<a href="https://github.com/nnaisense/evotorch/actions/workflows/test.yaml" rel="nofollow">
<img src="https://github.com/nnaisense/evotorch/actions/workflows/test.yaml/badge.svg" alt="Test" />
</a>
<a href="https://github.com/psf/black" rel="nofollow">
<img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="Code style: black" />
</a>
<a href="https://results.pre-commit.ci/latest/github/nnaisense/evotorch/master" rel="nofollow">
<img src="https://results.pre-commit.ci/badge/github/nnaisense/evotorch/master.svg" alt="pre-commit.ci status" />
</a>
</div>
---
Welcome to the EvoTorch project!
EvoTorch is an open source evolutionary computation library developed at [NNAISENSE](https://nnaisense.com), built on top of [PyTorch](https://pytorch.org/).
See the [documentation](https://docs.evotorch.ai) for in-depth guidance about using EvoTorch, and [join us on Slack](https://join.slack.com/t/evotorch/shared_invite/zt-1hcj9prrl-wQBMX4JtaB6WdGKSDjZGXw) for discussions.
Get started by installing EvoTorch:
```
pip install evotorch
```
With EvoTorch, one can solve various optimization problems, regardless of whether they are differentiable (i.e. allow gradient descent). Among the problem types that are solvable with EvoTorch are:
- Black-box optimization problems (continuous or discrete)
- Reinforcement learning tasks
- Supervised learning tasks
Various evolutionary computation algorithms are available in EvoTorch:
- **Distribution-based search algorithms:**
- **PGPE:** Policy Gradients with Parameter-based Exploration.
- **XNES:** Exponential Natural Evolution Strategies.
- **CMA-ES:** Covariance Matrix Adaptation Evolution Strategies.
- **SNES:** Separable Natural Evolution Strategies.
- **CEM:** Cross Entropy Method.
- **Population-based search algorithms:**
- **GeneticAlgorithm:** A genetic algorithm implementation. Also supports multiple objectives, in which case it behaves like **NSGA-II**.
- **CoSyNE:** Cooperative Synapse Neuroevolution.
- **MAPElites:** Multi-dimensional Archive of Phenotypic Elites
Since all of these algorithms are implemented in PyTorch, they benefit from use of vectorization and parallelization on GPUs, drastically speeding up optimization when GPUs are available.
Using [Ray](https://github.com/ray-project/ray), EvoTorch scales these algorithms even further by splitting the workload across:
- multiple CPUs
- multiple GPUs
- multiple computers in a Ray cluster
# Examples
Below are some code examples that demonstrate the API of EvoTorch.
## A black-box optimization example
Any objective function defined to work with PyTorch can be used directly with EvoTorch.
A non-vectorized objective function simply receives a solution as a 1-dimensional torch tensor, and returns a fitness as a scalar.
A vectorized objective function receives a batch of solutions as a 2-dimensional torch tensor, and returns a 1-dimensional tensor of fitnesses.
The following example demonstrates how to define and solve the classical Rastrigin problem.
```python
from evotorch import Problem
from evotorch.algorithms import SNES
from evotorch.logging import StdOutLogger, PandasLogger
import math
import matplotlib.pyplot as plt
import torch
# Declare the objective function
def rastrigin(x: torch.Tensor) -> torch.Tensor:
A = 10
(_, n) = x.shape
return A * n + torch.sum((x**2) - A * torch.cos(2 * math.pi * x), 1)
# Declare the problem
problem = Problem(
"min",
rastrigin,
initial_bounds=(-5.12, 5.12),
solution_length=100,
vectorized=True,
# device="cuda:0" # enable this line if you wish to use GPU
)
# Initialize the SNES algorithm to solve the problem
searcher = SNES(problem, popsize=1000, stdev_init=10.0)
# Initialize a standard output logger, and a pandas logger
_ = StdOutLogger(searcher, interval=10)
pandas_logger = PandasLogger(searcher)
# Run SNES for the specified amount of generations
searcher.run(2000)
# Get the progress of the evolution into a DataFrame with the
# help of the PandasLogger, and then plot the progress.
pandas_frame = pandas_logger.to_dataframe()
pandas_frame["best_eval"].plot()
plt.show()
```
## A reinforcement learning example
The following example demonstrates how to solve reinforcement learning tasks that are available through the gym library.
```python
from evotorch.algorithms import PGPE
from evotorch.logging import StdOutLogger, PicklingLogger
from evotorch.neuroevolution import GymNE
# Declare the problem to solve
problem = GymNE(
env="Humanoid-v4", # Solve the Humanoid-v4 task
network="Linear(obs_length, act_length)", # Linear policy
observation_normalization=True, # Normalize the policy inputs
decrease_rewards_by=5.0, # Decrease each reward by 5.0
num_actors="max", # Use all available CPUs
# num_actors=4, # Explicit setting. Use 4 actors.
)
# Instantiate a PGPE algorithm to solve the problem
searcher = PGPE(
problem,
# Base population size
popsize=200,
# For each generation, sample more solutions until the
# number of simulator interactions reaches this threshold
num_interactions=int(200 * 1000 * 0.75),
# Stop re-sampling solutions if the current population size
# reaches or exceeds this number.
popsize_max=3200,
# Learning rates
center_learning_rate=0.0075,
stdev_learning_rate=0.1,
# Radius of the initial search distribution
radius_init=0.27,
# Use the ClipUp optimizer with the specified maximum speed
optimizer="clipup",
optimizer_config={"max_speed": 0.15},
)
# Instantiate a standard output logger
_ = StdOutLogger(searcher)
# Optional: Instantiate a logger to pickle and save the results periodically.
# In this example, among the saved results will be the center of the search
# distribution, since we are using PGPE which is distribution-based.
_ = PicklingLogger(searcher, interval=10)
# Run the algorithm for the specified amount of generations
searcher.run(500)
# Get the center point of the search distribution,
# obtain a policy out of that point, and visualize the
# agent using that policy.
center_solution = searcher.status["center"]
trained_policy = problem.make_net(center_solution)
problem.visualize(trained_policy)
```
More examples can be found [here](examples/).
# How to cite
If you use EvoTorch in your research, please consider citing our [paper](https://arxiv.org/abs/2302.12600).
```bibtex
@article{evotorch2023arxiv,
title={{EvoTorch}: Scalable Evolutionary Computation in {Python}},
author={Toklu, Nihat Engin and Atkinson, Timothy and Micka, Vojt\v{e}ch and Liskowski, Pawe\l{} and Srivastava, Rupesh Kumar},
journal={arXiv preprint},
year={2023},
note={https://arxiv.org/abs/2302.12600}
}
```
# How to Contribute
Please see our [contribution guidelines](CONTRIBUTING.md).
# Authors
- [Nihat Engin Toklu](https://github.com/engintoklu)
- [Timothy Atkinson](https://github.com/NaturalGradient)
- [Vojtech Micka](https://github.com/Higgcz)
- [Pawel Liskowski](https://github.com/pliskowski)
- [Rupesh Kumar Srivastava](https://github.com/flukeskywalker)
Raw data
{
"_id": null,
"home_page": "https://evotorch.ai",
"name": "evotorch",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "python,evolution,neuroevolution,evolutionary computation,genetic algorithm,reinforcement learning",
"author": "Nihat Engin Toklu, Timothy Atkinson, Vojtech Micka, Pawel Liskowski, Rupesh Kumar Srivastava",
"author_email": "engin@nnaisense.com, timothy@nnaisense.com, vojtech@nnaisense.com, pawel@nnaisense.com, rupesh@nnaisense.com",
"download_url": "https://files.pythonhosted.org/packages/95/87/a79b431c29de1ee57a3d217975b542f7565f956b6b25dbdf96ebb0cbfd55/evotorch-0.5.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://evotorch.ai\" rel=\"nofollow\">\n <img src=\"https://raw.githubusercontent.com/nnaisense/evotorch/master/docs/assets/evotorch.svg\" />\n </a>\n</p>\n\n<div align=\"center\">\n <a href=\"https://www.python.org/\" rel=\"nofollow\">\n <img src=\"https://img.shields.io/pypi/pyversions/evotorch\" alt=\"Python\" />\n </a>\n <a href=\"https://pypi.org/project/evotorch/\" rel=\"nofollow\">\n <img src=\"https://img.shields.io/pypi/v/evotorch\" alt=\"PyPI\" />\n </a>\n <a href=\"https://github.com/nnaisense/evotorch/blob/master/LICENSE\" rel=\"nofollow\">\n <img src=\"https://img.shields.io/pypi/l/evotorch\" alt=\"License\" />\n </a>\n <a href=\"https://docs.evotorch.ai\" rel=\"nofollow\">\n <img src=\"https://github.com/nnaisense/evotorch/actions/workflows/docs.yaml/badge.svg\" alt=\"Build\" />\n </a>\n <a href=\"https://github.com/nnaisense/evotorch/actions/workflows/test.yaml\" rel=\"nofollow\">\n <img src=\"https://github.com/nnaisense/evotorch/actions/workflows/test.yaml/badge.svg\" alt=\"Test\" />\n </a>\n <a href=\"https://github.com/psf/black\" rel=\"nofollow\">\n <img src=\"https://img.shields.io/badge/code%20style-black-000000.svg\" alt=\"Code style: black\" />\n </a>\n <a href=\"https://results.pre-commit.ci/latest/github/nnaisense/evotorch/master\" rel=\"nofollow\">\n <img src=\"https://results.pre-commit.ci/badge/github/nnaisense/evotorch/master.svg\" alt=\"pre-commit.ci status\" />\n </a>\n</div>\n\n---\n\nWelcome to the EvoTorch project!\nEvoTorch is an open source evolutionary computation library developed at [NNAISENSE](https://nnaisense.com), built on top of [PyTorch](https://pytorch.org/).\nSee the [documentation](https://docs.evotorch.ai) for in-depth guidance about using EvoTorch, and [join us on Slack](https://join.slack.com/t/evotorch/shared_invite/zt-1hcj9prrl-wQBMX4JtaB6WdGKSDjZGXw) for discussions.\n\nGet started by installing EvoTorch:\n```\npip install evotorch\n```\n\nWith EvoTorch, one can solve various optimization problems, regardless of whether they are differentiable (i.e. allow gradient descent). Among the problem types that are solvable with EvoTorch are:\n- Black-box optimization problems (continuous or discrete)\n- Reinforcement learning tasks\n- Supervised learning tasks\n\nVarious evolutionary computation algorithms are available in EvoTorch:\n- **Distribution-based search algorithms:**\n - **PGPE:** Policy Gradients with Parameter-based Exploration.\n - **XNES:** Exponential Natural Evolution Strategies.\n - **CMA-ES:** Covariance Matrix Adaptation Evolution Strategies.\n - **SNES:** Separable Natural Evolution Strategies.\n - **CEM:** Cross Entropy Method.\n- **Population-based search algorithms:**\n - **GeneticAlgorithm:** A genetic algorithm implementation. Also supports multiple objectives, in which case it behaves like **NSGA-II**.\n - **CoSyNE:** Cooperative Synapse Neuroevolution.\n - **MAPElites:** Multi-dimensional Archive of Phenotypic Elites\n\nSince all of these algorithms are implemented in PyTorch, they benefit from use of vectorization and parallelization on GPUs, drastically speeding up optimization when GPUs are available.\nUsing [Ray](https://github.com/ray-project/ray), EvoTorch scales these algorithms even further by splitting the workload across:\n- multiple CPUs\n- multiple GPUs\n- multiple computers in a Ray cluster\n\n# Examples\n\nBelow are some code examples that demonstrate the API of EvoTorch.\n\n## A black-box optimization example\n\nAny objective function defined to work with PyTorch can be used directly with EvoTorch.\nA non-vectorized objective function simply receives a solution as a 1-dimensional torch tensor, and returns a fitness as a scalar.\nA vectorized objective function receives a batch of solutions as a 2-dimensional torch tensor, and returns a 1-dimensional tensor of fitnesses.\nThe following example demonstrates how to define and solve the classical Rastrigin problem.\n\n```python\nfrom evotorch import Problem\nfrom evotorch.algorithms import SNES\nfrom evotorch.logging import StdOutLogger, PandasLogger\nimport math\nimport matplotlib.pyplot as plt\nimport torch\n\n\n# Declare the objective function\ndef rastrigin(x: torch.Tensor) -> torch.Tensor:\n A = 10\n (_, n) = x.shape\n return A * n + torch.sum((x**2) - A * torch.cos(2 * math.pi * x), 1)\n\n\n# Declare the problem\nproblem = Problem(\n \"min\",\n rastrigin,\n initial_bounds=(-5.12, 5.12),\n solution_length=100,\n vectorized=True,\n # device=\"cuda:0\" # enable this line if you wish to use GPU\n)\n\n# Initialize the SNES algorithm to solve the problem\nsearcher = SNES(problem, popsize=1000, stdev_init=10.0)\n\n# Initialize a standard output logger, and a pandas logger\n_ = StdOutLogger(searcher, interval=10)\npandas_logger = PandasLogger(searcher)\n\n# Run SNES for the specified amount of generations\nsearcher.run(2000)\n\n# Get the progress of the evolution into a DataFrame with the\n# help of the PandasLogger, and then plot the progress.\npandas_frame = pandas_logger.to_dataframe()\npandas_frame[\"best_eval\"].plot()\nplt.show()\n```\n\n## A reinforcement learning example\n\nThe following example demonstrates how to solve reinforcement learning tasks that are available through the gym library.\n\n```python\nfrom evotorch.algorithms import PGPE\nfrom evotorch.logging import StdOutLogger, PicklingLogger\nfrom evotorch.neuroevolution import GymNE\n\n# Declare the problem to solve\nproblem = GymNE(\n env=\"Humanoid-v4\", # Solve the Humanoid-v4 task\n network=\"Linear(obs_length, act_length)\", # Linear policy\n observation_normalization=True, # Normalize the policy inputs\n decrease_rewards_by=5.0, # Decrease each reward by 5.0\n num_actors=\"max\", # Use all available CPUs\n # num_actors=4, # Explicit setting. Use 4 actors.\n)\n\n# Instantiate a PGPE algorithm to solve the problem\nsearcher = PGPE(\n problem,\n # Base population size\n popsize=200,\n # For each generation, sample more solutions until the\n # number of simulator interactions reaches this threshold\n num_interactions=int(200 * 1000 * 0.75),\n # Stop re-sampling solutions if the current population size\n # reaches or exceeds this number.\n popsize_max=3200,\n # Learning rates\n center_learning_rate=0.0075,\n stdev_learning_rate=0.1,\n # Radius of the initial search distribution\n radius_init=0.27,\n # Use the ClipUp optimizer with the specified maximum speed\n optimizer=\"clipup\",\n optimizer_config={\"max_speed\": 0.15},\n)\n\n# Instantiate a standard output logger\n_ = StdOutLogger(searcher)\n\n# Optional: Instantiate a logger to pickle and save the results periodically.\n# In this example, among the saved results will be the center of the search\n# distribution, since we are using PGPE which is distribution-based.\n_ = PicklingLogger(searcher, interval=10)\n\n# Run the algorithm for the specified amount of generations\nsearcher.run(500)\n\n# Get the center point of the search distribution,\n# obtain a policy out of that point, and visualize the\n# agent using that policy.\ncenter_solution = searcher.status[\"center\"]\ntrained_policy = problem.make_net(center_solution)\nproblem.visualize(trained_policy)\n```\n\nMore examples can be found [here](examples/).\n\n# How to cite\n\nIf you use EvoTorch in your research, please consider citing our [paper](https://arxiv.org/abs/2302.12600).\n\n```bibtex\n@article{evotorch2023arxiv,\n title={{EvoTorch}: Scalable Evolutionary Computation in {Python}},\n author={Toklu, Nihat Engin and Atkinson, Timothy and Micka, Vojt\\v{e}ch and Liskowski, Pawe\\l{} and Srivastava, Rupesh Kumar},\n journal={arXiv preprint},\n year={2023},\n note={https://arxiv.org/abs/2302.12600}\n}\n```\n\n# How to Contribute\n\nPlease see our [contribution guidelines](CONTRIBUTING.md).\n\n# Authors\n\n- [Nihat Engin Toklu](https://github.com/engintoklu)\n- [Timothy Atkinson](https://github.com/NaturalGradient)\n- [Vojtech Micka](https://github.com/Higgcz)\n- [Pawel Liskowski](https://github.com/pliskowski)\n- [Rupesh Kumar Srivastava](https://github.com/flukeskywalker)\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "EvoTorch is an advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.",
"version": "0.5.0",
"project_urls": {
"Documentation": "https://docs.evotorch.ai",
"Homepage": "https://evotorch.ai",
"Source Code": "https://github.com/nnaisense/evotorch"
},
"split_keywords": [
"python",
"evolution",
"neuroevolution",
"evolutionary computation",
"genetic algorithm",
"reinforcement learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "efcdfe3cfffaca1ed880bab1613f8889dab92279e6c6f903f017a107aa835340",
"md5": "8a16b02ff9dfdb332c58ff771f2f839a",
"sha256": "ea6a1d8938aef1123ec0d01df6604292b34686d2d07dbc0f3cc5ade111b75510"
},
"downloads": -1,
"filename": "evotorch-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8a16b02ff9dfdb332c58ff771f2f839a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 275635,
"upload_time": "2023-11-02T10:11:06",
"upload_time_iso_8601": "2023-11-02T10:11:06.926439Z",
"url": "https://files.pythonhosted.org/packages/ef/cd/fe3cfffaca1ed880bab1613f8889dab92279e6c6f903f017a107aa835340/evotorch-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9587a79b431c29de1ee57a3d217975b542f7565f956b6b25dbdf96ebb0cbfd55",
"md5": "839a8deb31d0a05c0ec83cf5208738a0",
"sha256": "93c83c2ed4c79163b3a8c430fdfb4c01d786a369b454ff8100c8669320d951fd"
},
"downloads": -1,
"filename": "evotorch-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "839a8deb31d0a05c0ec83cf5208738a0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 1748034,
"upload_time": "2023-11-02T10:11:09",
"upload_time_iso_8601": "2023-11-02T10:11:09.174412Z",
"url": "https://files.pythonhosted.org/packages/95/87/a79b431c29de1ee57a3d217975b542f7565f956b6b25dbdf96ebb0cbfd55/evotorch-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-02 10:11:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nnaisense",
"github_project": "evotorch",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "evotorch"
}
JSON
