[](https://travis-ci.org/cvxgrp/cvxpylayers)
[](https://ci.appveyor.com/project/sbarratt/cvxpylayers/branch/master)
# cvxpylayers
cvxpylayers is a Python library for constructing differentiable convex
optimization layers in PyTorch, JAX, and TensorFlow using CVXPY.
A convex optimization layer solves a parametrized convex optimization problem
in the forward pass to produce a solution.
It computes the derivative of the solution with respect to
the parameters in the backward pass.
This library accompanies our [NeurIPS 2019 paper](https://web.stanford.edu/~boyd/papers/pdf/diff_cvxpy.pdf)
on differentiable convex optimization layers.
For an informal introduction to convex optimization layers, see our
[blog post](https://locuslab.github.io/2019-10-28-cvxpylayers/).
Our package uses [CVXPY](https://github.com/cvxgrp/cvxpy) for specifying
parametrized convex optimization problems.
- [Installation](#installation)
- [Usage](#usage)
- [Examples](#examples)
- [Contributing](#contributing)
- [Projects using cvxpylayers](#projects-using-cvxpylayers)
- [License](#contributing)
- [Citing](#citing)
## Installation
Use the package manager [pip](https://pip.pypa.io/en/stable/) to install
cvxpylayers.
```bash
pip install cvxpylayers
```
Our package includes convex optimization layers for
PyTorch, JAX, and TensorFlow 2.0;
the layers are functionally equivalent. You will need to install
[PyTorch](https://pytorch.org),
[JAX](https://github.com/google/jax), or
[TensorFlow](https://www.tensorflow.org)
separately, which can be done by following the instructions on their websites.
cvxpylayers has the following dependencies:
* Python 3
* [NumPy](https://pypi.org/project/numpy/)
* [CVXPY](https://github.com/cvxgrp/cvxpy) >= 1.1.a4
* [PyTorch](https://pytorch.org) >= 1.0, [JAX](https://github.com/google/jax) >= 0.2.12, or [TensorFlow](https://tensorflow.org) >= 2.0
* [diffcp](https://github.com/cvxgrp/diffcp) >= 1.0.13
## Usage
Below are usage examples of our PyTorch, JAX, and TensorFlow layers.
Note that the parametrized convex optimization problems must be constructed
in CVXPY, using
[DPP](https://www.cvxpy.org/tutorial/advanced/index.html#disciplined-parametrized-programming).
### PyTorch
```python
import cvxpy as cp
import torch
from cvxpylayers.torch import CvxpyLayer
n, m = 2, 3
x = cp.Variable(n)
A = cp.Parameter((m, n))
b = cp.Parameter(m)
constraints = [x >= 0]
objective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))
problem = cp.Problem(objective, constraints)
assert problem.is_dpp()
cvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])
A_tch = torch.randn(m, n, requires_grad=True)
b_tch = torch.randn(m, requires_grad=True)
# solve the problem
solution, = cvxpylayer(A_tch, b_tch)
# compute the gradient of the sum of the solution with respect to A, b
solution.sum().backward()
```
Note: `CvxpyLayer` cannot be traced with `torch.jit`.
### JAX
```python
import cvxpy as cp
import jax
from cvxpylayers.jax import CvxpyLayer
n, m = 2, 3
x = cp.Variable(n)
A = cp.Parameter((m, n))
b = cp.Parameter(m)
constraints = [x >= 0]
objective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))
problem = cp.Problem(objective, constraints)
assert problem.is_dpp()
cvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])
key = jax.random.PRNGKey(0)
key, k1, k2 = jax.random.split(key, 3)
A_jax = jax.random.normal(k1, shape=(m, n))
b_jax = jax.random.normal(k2, shape=(m,))
solution, = cvxpylayer(A_jax, b_jax)
# compute the gradient of the summed solution with respect to A, b
dcvxpylayer = jax.grad(lambda A, b: sum(cvxpylayer(A, b)[0]), argnums=[0, 1])
gradA, gradb = dcvxpylayer(A_jax, b_jax)
```
Note: `CvxpyLayer` cannot be traced with the JAX `jit` or `vmap` operations.
### TensorFlow 2
```python
import cvxpy as cp
import tensorflow as tf
from cvxpylayers.tensorflow import CvxpyLayer
n, m = 2, 3
x = cp.Variable(n)
A = cp.Parameter((m, n))
b = cp.Parameter(m)
constraints = [x >= 0]
objective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))
problem = cp.Problem(objective, constraints)
assert problem.is_dpp()
cvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])
A_tf = tf.Variable(tf.random.normal((m, n)))
b_tf = tf.Variable(tf.random.normal((m,)))
with tf.GradientTape() as tape:
# solve the problem, setting the values of A, b to A_tf, b_tf
solution, = cvxpylayer(A_tf, b_tf)
summed_solution = tf.math.reduce_sum(solution)
# compute the gradient of the summed solution with respect to A, b
gradA, gradb = tape.gradient(summed_solution, [A_tf, b_tf])
```
Note: `CvxpyLayer` cannot be traced with `tf.function`.
### Log-log convex programs
Starting with version 0.1.3, cvxpylayers can also differentiate through log-log convex programs (LLCPs), which generalize geometric programs. Use the keyword argument `gp=True` when constructing a `CvxpyLayer` for an LLCP. Below is a simple usage example
```python
import cvxpy as cp
import torch
from cvxpylayers.torch import CvxpyLayer
x = cp.Variable(pos=True)
y = cp.Variable(pos=True)
z = cp.Variable(pos=True)
a = cp.Parameter(pos=True, value=2.)
b = cp.Parameter(pos=True, value=1.)
c = cp.Parameter(value=0.5)
objective_fn = 1/(x*y*z)
objective = cp.Minimize(objective_fn)
constraints = [a*(x*y + x*z + y*z) <= b, x >= y**c]
problem = cp.Problem(objective, constraints)
assert problem.is_dgp(dpp=True)
layer = CvxpyLayer(problem, parameters=[a, b, c],
variables=[x, y, z], gp=True)
a_tch = torch.tensor(a.value, requires_grad=True)
b_tch = torch.tensor(b.value, requires_grad=True)
c_tch = torch.tensor(c.value, requires_grad=True)
x_star, y_star, z_star = layer(a_tch, b_tch, c_tch)
sum_of_solution = x_star + y_star + z_star
sum_of_solution.backward()
```
## Solvers
At this time, we support three open-source solvers: [Clarabel](https://github.com/oxfordcontrol/Clarabel.rs),
[SCS](https://github.com/cvxgrp/scs), and [ECOS](https://github.com/embotech/ecos).
Clarabel and SCS can be used to solve any problem expressible in CVXPY; ECOS can be used to solve problems that don't use
the positive semidefinite or exponential cone (this roughly means that if you have positive semidefinite matrices
or use atoms like `cp.log`, ECOS cannot be used to solve your problem via `cvxpylayers`).
By default, `cvxpylayers` uses SCS to solve the problem.
### Using ECOS
First make sure that you have `cvxpylayers` and `diffcp` up to date,
by running
```
pip install --upgrade cvxpylayers diffcp
```
Then, to use ECOS, you would pass the `solver_args` argument to the layer:
```
solution = layer(*params, solver_args={"solve_method": "ECOS"})
```
### Passing arguments to the solvers
One can pass arguments to both SCS and ECOS by adding the argument as a key-value pair
in the `solver_args` argument.
For example, to increase the tolerance of SCS to `1e-8` one would write:
```
layer(*parameters, solver_args={"eps": 1e-8})
```
If SCS is not converging, we highly recommend switching to ECOS (if possible),
and if not, using the following arguments to `SCS`:
```
solver_args={"eps": 1e-8, "max_iters": 10000, "acceleration_lookback": 0}
```
## Examples
Our [examples](examples) subdirectory contains simple applications of convex optimization
layers in IPython notebooks.
## Contributing
Pull requests are welcome. For major changes, please open an issue first to
discuss what you would like to change.
Please make sure to update tests as appropriate.
Please lint the code with `flake8`.
```bash
pip install flake8 # if not already installed
flake8
```
### Running tests
cvxpylayers uses the `pytest` framework for running tests.
To install `pytest`, run:
```bash
pip install pytest
```
Execute the tests from the main directory of this repository with:
```bash
pytest cvxpylayers/{torch,jax,tensorflow}
```
## Projects using cvxpylayers
Below is a list of projects using cvxpylayers. If you have used cvxpylayers in a project, you're welcome to make a PR to add it to this list.
* [Learning Convex Optimization Control Policies](https://web.stanford.edu/~boyd/papers/learning_cocps.html)
* [Learning Convex Optimization Models](https://web.stanford.edu/~boyd/papers/learning_copt_models.html)
## License
cvxpylayers carries an Apache 2.0 license.
## Citing
If you use cvxpylayers for research, please cite our accompanying [NeurIPS paper](https://web.stanford.edu/~boyd/papers/pdf/diff_cvxpy.pdf):
```
@inproceedings{cvxpylayers2019,
author={Agrawal, A. and Amos, B. and Barratt, S. and Boyd, S. and Diamond, S. and Kolter, Z.},
title={Differentiable Convex Optimization Layers},
booktitle={Advances in Neural Information Processing Systems},
year={2019},
}
```
If you use cvxpylayers to differentiate through a log-log convex program, please cite the accompanying [paper](https://web.stanford.edu/~boyd/papers/diff_dgp.html):
```
@article{agrawal2020differentiating,
title={Differentiating through log-log convex programs},
author={Agrawal, Akshay and Boyd, Stephen},
journal={arXiv},
archivePrefix={arXiv},
eprint={2004.12553},
primaryClass={math.OC},
year={2020},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/cvxgrp/cvxpylayers",
"name": "cvxpylayers",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/cc/37/9ce623f5f583fe116d6d784f4a90b64e3d291802bc6f207f8653ae896f5f/cvxpylayers-0.1.9.tar.gz",
"platform": null,
"description": "\n[](https://travis-ci.org/cvxgrp/cvxpylayers)\n[](https://ci.appveyor.com/project/sbarratt/cvxpylayers/branch/master)\n\n# cvxpylayers\n\ncvxpylayers is a Python library for constructing differentiable convex\noptimization layers in PyTorch, JAX, and TensorFlow using CVXPY.\nA convex optimization layer solves a parametrized convex optimization problem\nin the forward pass to produce a solution.\nIt computes the derivative of the solution with respect to\nthe parameters in the backward pass.\n\nThis library accompanies our [NeurIPS 2019 paper](https://web.stanford.edu/~boyd/papers/pdf/diff_cvxpy.pdf)\non differentiable convex optimization layers.\nFor an informal introduction to convex optimization layers, see our\n[blog post](https://locuslab.github.io/2019-10-28-cvxpylayers/).\n\nOur package uses [CVXPY](https://github.com/cvxgrp/cvxpy) for specifying\nparametrized convex optimization problems.\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Examples](#examples)\n- [Contributing](#contributing)\n- [Projects using cvxpylayers](#projects-using-cvxpylayers)\n- [License](#contributing)\n- [Citing](#citing)\n\n## Installation\n\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install\ncvxpylayers.\n\n```bash\npip install cvxpylayers\n```\n\nOur package includes convex optimization layers for\nPyTorch, JAX, and TensorFlow 2.0;\nthe layers are functionally equivalent. You will need to install\n[PyTorch](https://pytorch.org),\n[JAX](https://github.com/google/jax), or\n[TensorFlow](https://www.tensorflow.org)\nseparately, which can be done by following the instructions on their websites.\n\ncvxpylayers has the following dependencies:\n* Python 3\n* [NumPy](https://pypi.org/project/numpy/)\n* [CVXPY](https://github.com/cvxgrp/cvxpy) >= 1.1.a4\n* [PyTorch](https://pytorch.org) >= 1.0, [JAX](https://github.com/google/jax) >= 0.2.12, or [TensorFlow](https://tensorflow.org) >= 2.0\n* [diffcp](https://github.com/cvxgrp/diffcp) >= 1.0.13\n\n## Usage\nBelow are usage examples of our PyTorch, JAX, and TensorFlow layers.\nNote that the parametrized convex optimization problems must be constructed\nin CVXPY, using\n[DPP](https://www.cvxpy.org/tutorial/advanced/index.html#disciplined-parametrized-programming).\n\n### PyTorch\n\n```python\nimport cvxpy as cp\nimport torch\nfrom cvxpylayers.torch import CvxpyLayer\n\nn, m = 2, 3\nx = cp.Variable(n)\nA = cp.Parameter((m, n))\nb = cp.Parameter(m)\nconstraints = [x >= 0]\nobjective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))\nproblem = cp.Problem(objective, constraints)\nassert problem.is_dpp()\n\ncvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])\nA_tch = torch.randn(m, n, requires_grad=True)\nb_tch = torch.randn(m, requires_grad=True)\n\n# solve the problem\nsolution, = cvxpylayer(A_tch, b_tch)\n\n# compute the gradient of the sum of the solution with respect to A, b\nsolution.sum().backward()\n```\n\nNote: `CvxpyLayer` cannot be traced with `torch.jit`.\n\n### JAX\n```python\nimport cvxpy as cp\nimport jax\nfrom cvxpylayers.jax import CvxpyLayer\n\nn, m = 2, 3\nx = cp.Variable(n)\nA = cp.Parameter((m, n))\nb = cp.Parameter(m)\nconstraints = [x >= 0]\nobjective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))\nproblem = cp.Problem(objective, constraints)\nassert problem.is_dpp()\n\ncvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])\nkey = jax.random.PRNGKey(0)\nkey, k1, k2 = jax.random.split(key, 3)\nA_jax = jax.random.normal(k1, shape=(m, n))\nb_jax = jax.random.normal(k2, shape=(m,))\n\nsolution, = cvxpylayer(A_jax, b_jax)\n\n# compute the gradient of the summed solution with respect to A, b\ndcvxpylayer = jax.grad(lambda A, b: sum(cvxpylayer(A, b)[0]), argnums=[0, 1])\ngradA, gradb = dcvxpylayer(A_jax, b_jax)\n```\n\nNote: `CvxpyLayer` cannot be traced with the JAX `jit` or `vmap` operations.\n\n### TensorFlow 2\n```python\nimport cvxpy as cp\nimport tensorflow as tf\nfrom cvxpylayers.tensorflow import CvxpyLayer\n\nn, m = 2, 3\nx = cp.Variable(n)\nA = cp.Parameter((m, n))\nb = cp.Parameter(m)\nconstraints = [x >= 0]\nobjective = cp.Minimize(0.5 * cp.pnorm(A @ x - b, p=1))\nproblem = cp.Problem(objective, constraints)\nassert problem.is_dpp()\n\ncvxpylayer = CvxpyLayer(problem, parameters=[A, b], variables=[x])\nA_tf = tf.Variable(tf.random.normal((m, n)))\nb_tf = tf.Variable(tf.random.normal((m,)))\n\nwith tf.GradientTape() as tape:\n # solve the problem, setting the values of A, b to A_tf, b_tf\n solution, = cvxpylayer(A_tf, b_tf)\n summed_solution = tf.math.reduce_sum(solution)\n# compute the gradient of the summed solution with respect to A, b\ngradA, gradb = tape.gradient(summed_solution, [A_tf, b_tf])\n```\n\nNote: `CvxpyLayer` cannot be traced with `tf.function`.\n\n\n### Log-log convex programs\nStarting with version 0.1.3, cvxpylayers can also differentiate through log-log convex programs (LLCPs), which generalize geometric programs. Use the keyword argument `gp=True` when constructing a `CvxpyLayer` for an LLCP. Below is a simple usage example\n\n```python\nimport cvxpy as cp\nimport torch\nfrom cvxpylayers.torch import CvxpyLayer\n\nx = cp.Variable(pos=True)\ny = cp.Variable(pos=True)\nz = cp.Variable(pos=True)\n\na = cp.Parameter(pos=True, value=2.)\nb = cp.Parameter(pos=True, value=1.)\nc = cp.Parameter(value=0.5)\n\nobjective_fn = 1/(x*y*z)\nobjective = cp.Minimize(objective_fn)\nconstraints = [a*(x*y + x*z + y*z) <= b, x >= y**c]\nproblem = cp.Problem(objective, constraints)\nassert problem.is_dgp(dpp=True)\n\nlayer = CvxpyLayer(problem, parameters=[a, b, c],\n variables=[x, y, z], gp=True)\na_tch = torch.tensor(a.value, requires_grad=True)\nb_tch = torch.tensor(b.value, requires_grad=True)\nc_tch = torch.tensor(c.value, requires_grad=True)\n\nx_star, y_star, z_star = layer(a_tch, b_tch, c_tch)\nsum_of_solution = x_star + y_star + z_star\nsum_of_solution.backward()\n```\n\n## Solvers\n\nAt this time, we support three open-source solvers: [Clarabel](https://github.com/oxfordcontrol/Clarabel.rs),\n[SCS](https://github.com/cvxgrp/scs), and [ECOS](https://github.com/embotech/ecos).\nClarabel and SCS can be used to solve any problem expressible in CVXPY; ECOS can be used to solve problems that don't use\nthe positive semidefinite or exponential cone (this roughly means that if you have positive semidefinite matrices\nor use atoms like `cp.log`, ECOS cannot be used to solve your problem via `cvxpylayers`).\nBy default, `cvxpylayers` uses SCS to solve the problem.\n\n### Using ECOS\nFirst make sure that you have `cvxpylayers` and `diffcp` up to date,\nby running\n```\npip install --upgrade cvxpylayers diffcp\n```\nThen, to use ECOS, you would pass the `solver_args` argument to the layer:\n```\nsolution = layer(*params, solver_args={\"solve_method\": \"ECOS\"})\n```\n\n### Passing arguments to the solvers\nOne can pass arguments to both SCS and ECOS by adding the argument as a key-value pair\nin the `solver_args` argument.\nFor example, to increase the tolerance of SCS to `1e-8` one would write:\n```\nlayer(*parameters, solver_args={\"eps\": 1e-8})\n```\nIf SCS is not converging, we highly recommend switching to ECOS (if possible),\nand if not, using the following arguments to `SCS`:\n```\nsolver_args={\"eps\": 1e-8, \"max_iters\": 10000, \"acceleration_lookback\": 0}\n``` \n\n## Examples\nOur [examples](examples) subdirectory contains simple applications of convex optimization\nlayers in IPython notebooks.\n\n## Contributing\nPull requests are welcome. For major changes, please open an issue first to\ndiscuss what you would like to change.\n\nPlease make sure to update tests as appropriate.\n\nPlease lint the code with `flake8`.\n```bash\npip install flake8 # if not already installed\nflake8\n```\n\n### Running tests\n\ncvxpylayers uses the `pytest` framework for running tests.\nTo install `pytest`, run:\n```bash\npip install pytest\n```\n\nExecute the tests from the main directory of this repository with:\n```bash\npytest cvxpylayers/{torch,jax,tensorflow}\n```\n\n## Projects using cvxpylayers\nBelow is a list of projects using cvxpylayers. If you have used cvxpylayers in a project, you're welcome to make a PR to add it to this list.\n* [Learning Convex Optimization Control Policies](https://web.stanford.edu/~boyd/papers/learning_cocps.html)\n* [Learning Convex Optimization Models](https://web.stanford.edu/~boyd/papers/learning_copt_models.html)\n\n## License\ncvxpylayers carries an Apache 2.0 license.\n\n## Citing\nIf you use cvxpylayers for research, please cite our accompanying [NeurIPS paper](https://web.stanford.edu/~boyd/papers/pdf/diff_cvxpy.pdf):\n\n```\n@inproceedings{cvxpylayers2019,\n author={Agrawal, A. and Amos, B. and Barratt, S. and Boyd, S. and Diamond, S. and Kolter, Z.},\n title={Differentiable Convex Optimization Layers},\n booktitle={Advances in Neural Information Processing Systems},\n year={2019},\n}\n```\n\nIf you use cvxpylayers to differentiate through a log-log convex program, please cite the accompanying [paper](https://web.stanford.edu/~boyd/papers/diff_dgp.html):\n\n```\n@article{agrawal2020differentiating,\n title={Differentiating through log-log convex programs},\n author={Agrawal, Akshay and Boyd, Stephen},\n journal={arXiv},\n archivePrefix={arXiv},\n eprint={2004.12553},\n primaryClass={math.OC},\n year={2020},\n}\n```\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "Differentiable convex optimization layers",
"version": "0.1.9",
"project_urls": {
"Homepage": "https://github.com/cvxgrp/cvxpylayers"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fc01e2fd0ee99b0d24297fc3c1469b27d36084f72c411778e30ae29786230bc1",
"md5": "f75df110bee970bf4a9c2e8ee8893493",
"sha256": "8b3d547ca1862462c290e4767b12b67beeda2bacc8db7106f7c8aec35f17be59"
},
"downloads": -1,
"filename": "cvxpylayers-0.1.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f75df110bee970bf4a9c2e8ee8893493",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 31689,
"upload_time": "2024-12-06T01:44:38",
"upload_time_iso_8601": "2024-12-06T01:44:38.163249Z",
"url": "https://files.pythonhosted.org/packages/fc/01/e2fd0ee99b0d24297fc3c1469b27d36084f72c411778e30ae29786230bc1/cvxpylayers-0.1.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cc379ce623f5f583fe116d6d784f4a90b64e3d291802bc6f207f8653ae896f5f",
"md5": "6e9a6861a496df5bcb70a814aa8d0820",
"sha256": "aa72a475b28930b09855acbddce21ef3c1226aa3d86fec3dedc27afe5329f5f8"
},
"downloads": -1,
"filename": "cvxpylayers-0.1.9.tar.gz",
"has_sig": false,
"md5_digest": "6e9a6861a496df5bcb70a814aa8d0820",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29696,
"upload_time": "2024-12-06T01:44:40",
"upload_time_iso_8601": "2024-12-06T01:44:40.127928Z",
"url": "https://files.pythonhosted.org/packages/cc/37/9ce623f5f583fe116d6d784f4a90b64e3d291802bc6f207f8653ae896f5f/cvxpylayers-0.1.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-06 01:44:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cvxgrp",
"github_project": "cvxpylayers",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"appveyor": true,
"lcname": "cvxpylayers"
}
JSON
