Q-MM: A Python toolbox for Quadratic Majorization-Minimization
==============================================================
[](https://doi.org/10.5281/zenodo.6373070)
[](https://qmm.readthedocs.io/en/latest/?badge=latest)
Q-MM is a Python implementation of Majorize-Minimize Quadratic
optimization algorithms. Algorithms provided here come from
[1] C. Labat and J. Idier, “Convergence of Conjugate Gradient Methods with a
Closed-Form Stepsize Formula,” J Optim Theory Appl, p. 18, 2008.
and
[2] E. Chouzenoux, J. Idier, and S. Moussaoui, “A Majorize–Minimize Strategy
for Subspace Optimization Applied to Image Restoration,” IEEE Trans. on
Image Process., vol. 20, no. 6, pp. 1517–1528, Jun. 2011, doi:
10.1109/TIP.2010.2103083.
See [documentation](https://qmm.readthedocs.io/en/stable/index.html) for more
background. If you use this code, please cite the references above and a
citation of this toolbox will also be appreciated, see [below](#citation). You
can also click ⭐ on the repo.
Quadratic Majorize-Minimize
---------------------------
The Q-MM optimization algorithms compute the minimizer of objective
function like
J(x) = ∑ₖ μₖ ψₖ(Vₖ·x - ωₖ)
where x is the unknown vector, Vₖ a linear operator, ωₖ a fixed data, μₖ
a scalar, ψₖ(u) = ∑ᵢφₖ(uᵢ), and φₖ a function that must be
differentiable, even, coercive, φ(√·) concave, and 0 \< φ\'(u) / u \<
+∞.
The optimization is done thanks to quadratic sugorate function. In
particular, no linesearch or sub-iteration is necessary, and close form
formula for the step are used with guaranteed convergence.
A classical example, like in the figure below that show an image
deconvolution problem, is the resolution of an inverse problem with the
minimization of
J(x) = \|\|² + μ ψ(V·x)
where H is a low-pass forward model, V a regularization operator that
approximate gradient (kind of high-pass filter) and ψ an edge preserving
function like Huber. The above objective is obtained with k ∈ {1, 2},
ψ₁(·) = \|\|², V₁ = H, ω₁ = y, and ω₂ = 0.
Features
--------
- The `mmmg`, Majorize-Minimize Memory Gradient algorithm. See
documentation and \[2\] for details.
- The `mmcg`, Majorize-Minimize Conjugate Gradient algorithm. See
documentation and \[1\] for details.
- **No linesearch**: the step is obtained from a close form formula
without sub-iteration.
- **No conjugacy choice**: a conjugacy strategy is not necessary
thanks to the subspace nature of the algorithms. The `mmcg`
algorithm use a Polak-Ribière formula.
- Generic and flexible: there is no restriction on the number of
regularizer, their type, ..., as well as for data adequacy.
- Provided base class for objectives and losses allowing easy and fast
implementation.
- Just one file if you like quick and dirty installation, but
available with `pip`.
- Comes with examples of implemented linear operator.
Installation and documentation
------------------------------
Q-MM is essentially just one file `qmm.py`. We recommend using poetry
for installation
``` {.sourceCode .sh}
poetry add qmm
```
The package can also be installed with pip. More options are described
in the [documentation](https://qmm.readthedocs.io/en/stable/index.html).
Q-MM only depends on `numpy` and Python 3.6.
Example
-------
The `demo.py` presents an example on image deconvolution. The first step
is to implement the operators `V` and the adjoint `Vᵀ` as callable
(function or methods). The user is in charge of these operators and
these callable must accept a unique Numpy array `x` and a unique return
value
([partial](https://docs.python.org/fr/3.9/library/functools.html#functools.partial)
in the `functools` module in the standard library is usefull here).
There is no constraints on the shape, everything is vectorized
internally.
After import of `qmm`, user must instantiate `Potential` objects that
implement `φ` and `Objective` objects that implement `μ ψ(V·x - ω)`
``` {.sourceCode .python}
import qmm
phi = qmm.Huber(delta=10) # φ
data_adeq = qmm.QuadObjective(H, Ht, HtH, data=data) # ||y - H·x||²
prior = qmm.Objective(V, Vt, phi, hyper=0.01) # μ ψ(V·x) = μ ∑ᵢ φ(vᵢᵗ·x)
```
Then you can run the algorithm
``` {.sourceCode .python}
res = qmm.mmmg([data_adeq, prior], init, max_iter=200)
```
where `[data_adeq, prior]`{.sourceCode} means that the two objective
functions are summed. For more details, see
[documentation](https://qmm.readthedocs.io/en/stable/index.html).
Contribute
----------
- Source code: <https://github.com/forieux/qmm>
- Issue tracker: <https://github.com/forieux/qmm/issues>
Author
------
If you are having issues, please let us know
orieux AT l2s.centralesupelec.fr
More information about me [here](https://pro.orieux.fr). F. Orieux and
R. Abirizk are affiliated to the Signal and Systems Laboratory
[L2S](https://l2s.centralesupelec.fr/).
Citation
--------
Q-MM has a DOI with Zenodo
[](https://doi.org/10.5281/zenodo.6373069).
Specific version can also be cited. Citation can be
François Orieux, & Ralph Abirizk. (2022). Q-MM: The Quadratic Majorize-Minimize
Python toolbox (v0.12.0). Zenodo. https://doi.org/10.5281/zenodo.6373070
A example of bibtex is
@software{francois_orieux_2022_6373070,
author = {François Orieux and Ralph Abirizk},
title = {Q-MM: The Quadratic Majorize-Minimize Python toolbox},
month = mar,
year = 2022,
publisher = {Zenodo},
version = {0.12.0},
doi = {10.5281/zenodo.6373069},
url = {https://doi.org/10.5281/zenodo.6373069}
}
Acknowledgement
---------------
Author would like to thanks [J.
Idier](https://pagespersowp.ls2n.fr/jeromeidier/en/jerome-idier-3/), [S.
Moussaoui](https://scholar.google.fr/citations?user=Vkr8yxkAAAAJ&hl=fr)
and [É. Chouzenoux](http://www-syscom.univ-mlv.fr/~chouzeno/). É.
Chouzenoux has also a Matlab package that implements 3MG for image
deconvolution that can be found on her
[webpage](http://www-syscom.univ-mlv.fr/~chouzeno/Logiciel.html).
License
-------
The project is licensed under the GPLv3 license.
Raw data
{
"_id": null,
"home_page": "https://qmm.readthedocs.io/en/stable/",
"name": "qmm",
"maintainer": "Fran\u00e7ois Orieux",
"docs_url": null,
"requires_python": ">=3.7,<3.12",
"maintainer_email": "francois.orieux@universite-paris-saclay.fr",
"keywords": "optimization,MM,surogate,criterion,objective,learning",
"author": "Fran\u00e7ois Orieux",
"author_email": "francois.orieux@universite-paris-saclay.fr",
"download_url": "https://files.pythonhosted.org/packages/23/f2/74e3d5eeb974c1d4efff8d348a155183c0d805580b56eb2b526ae44b59b4/qmm-0.14.0.tar.gz",
"platform": null,
"description": "Q-MM: A Python toolbox for Quadratic Majorization-Minimization\n==============================================================\n\n[](https://doi.org/10.5281/zenodo.6373070)\n\n\n\n\n\n[](https://qmm.readthedocs.io/en/latest/?badge=latest)\n\n\n\nQ-MM is a Python implementation of Majorize-Minimize Quadratic\noptimization algorithms. Algorithms provided here come from\n\n [1] C. Labat and J. Idier, \u201cConvergence of Conjugate Gradient Methods with a\n Closed-Form Stepsize Formula,\u201d J Optim Theory Appl, p. 18, 2008.\n\nand\n\n [2] E. Chouzenoux, J. Idier, and S. Moussaoui, \u201cA Majorize\u2013Minimize Strategy\n for Subspace Optimization Applied to Image Restoration,\u201d IEEE Trans. on\n Image Process., vol. 20, no. 6, pp. 1517\u20131528, Jun. 2011, doi:\n 10.1109/TIP.2010.2103083.\n\nSee [documentation](https://qmm.readthedocs.io/en/stable/index.html) for more\nbackground. If you use this code, please cite the references above and a\ncitation of this toolbox will also be appreciated, see [below](#citation). You\ncan also click \u2b50 on the repo.\n\nQuadratic Majorize-Minimize\n---------------------------\n\nThe Q-MM optimization algorithms compute the minimizer of objective\nfunction like\n\nJ(x) = \u2211\u2096 \u03bc\u2096 \u03c8\u2096(V\u2096\u00b7x - \u03c9\u2096)\n\nwhere x is the unknown vector, V\u2096 a linear operator, \u03c9\u2096 a fixed data, \u03bc\u2096\na scalar, \u03c8\u2096(u) = \u2211\u1d62\u03c6\u2096(u\u1d62), and \u03c6\u2096 a function that must be\ndifferentiable, even, coercive, \u03c6(\u221a\u00b7) concave, and 0 \\< \u03c6\\'(u) / u \\<\n+\u221e.\n\nThe optimization is done thanks to quadratic sugorate function. In\nparticular, no linesearch or sub-iteration is necessary, and close form\nformula for the step are used with guaranteed convergence.\n\nA classical example, like in the figure below that show an image\ndeconvolution problem, is the resolution of an inverse problem with the\nminimization of\n\nJ(x) = \\|\\|\u00b2 + \u03bc \u03c8(V\u00b7x)\n\nwhere H is a low-pass forward model, V a regularization operator that\napproximate gradient (kind of high-pass filter) and \u03c8 an edge preserving\nfunction like Huber. The above objective is obtained with k \u2208 {1, 2},\n\u03c8\u2081(\u00b7) = \\|\\|\u00b2, V\u2081 = H, \u03c9\u2081 = y, and \u03c9\u2082 = 0.\n\n\n\nFeatures\n--------\n\n- The `mmmg`, Majorize-Minimize Memory Gradient algorithm. See\n documentation and \\[2\\] for details.\n- The `mmcg`, Majorize-Minimize Conjugate Gradient algorithm. See\n documentation and \\[1\\] for details.\n- **No linesearch**: the step is obtained from a close form formula\n without sub-iteration.\n- **No conjugacy choice**: a conjugacy strategy is not necessary\n thanks to the subspace nature of the algorithms. The `mmcg`\n algorithm use a Polak-Ribi\u00e8re formula.\n- Generic and flexible: there is no restriction on the number of\n regularizer, their type, ..., as well as for data adequacy.\n- Provided base class for objectives and losses allowing easy and fast\n implementation.\n- Just one file if you like quick and dirty installation, but\n available with `pip`.\n- Comes with examples of implemented linear operator.\n\nInstallation and documentation\n------------------------------\n\nQ-MM is essentially just one file `qmm.py`. We recommend using poetry\nfor installation\n\n``` {.sourceCode .sh}\npoetry add qmm\n```\n\nThe package can also be installed with pip. More options are described\nin the [documentation](https://qmm.readthedocs.io/en/stable/index.html).\n\nQ-MM only depends on `numpy` and Python 3.6.\n\nExample\n-------\n\nThe `demo.py` presents an example on image deconvolution. The first step\nis to implement the operators `V` and the adjoint `V\u1d40` as callable\n(function or methods). The user is in charge of these operators and\nthese callable must accept a unique Numpy array `x` and a unique return\nvalue\n([partial](https://docs.python.org/fr/3.9/library/functools.html#functools.partial)\nin the `functools` module in the standard library is usefull here).\nThere is no constraints on the shape, everything is vectorized\ninternally.\n\nAfter import of `qmm`, user must instantiate `Potential` objects that\nimplement `\u03c6` and `Objective` objects that implement `\u03bc \u03c8(V\u00b7x - \u03c9)`\n\n``` {.sourceCode .python}\nimport qmm\nphi = qmm.Huber(delta=10) # \u03c6\n\ndata_adeq = qmm.QuadObjective(H, Ht, HtH, data=data) # ||y - H\u00b7x||\u00b2\nprior = qmm.Objective(V, Vt, phi, hyper=0.01) # \u03bc \u03c8(V\u00b7x) = \u03bc \u2211\u1d62 \u03c6(v\u1d62\u1d57\u00b7x)\n```\n\nThen you can run the algorithm\n\n``` {.sourceCode .python}\nres = qmm.mmmg([data_adeq, prior], init, max_iter=200)\n```\n\nwhere `[data_adeq, prior]`{.sourceCode} means that the two objective\nfunctions are summed. For more details, see\n[documentation](https://qmm.readthedocs.io/en/stable/index.html).\n\nContribute\n----------\n\n- Source code: <https://github.com/forieux/qmm>\n- Issue tracker: <https://github.com/forieux/qmm/issues>\n\nAuthor\n------\n\nIf you are having issues, please let us know\n\norieux AT l2s.centralesupelec.fr\n\nMore information about me [here](https://pro.orieux.fr). F. Orieux and\nR. Abirizk are affiliated to the Signal and Systems Laboratory\n[L2S](https://l2s.centralesupelec.fr/).\n\nCitation\n--------\n\nQ-MM has a DOI with Zenodo\n[](https://doi.org/10.5281/zenodo.6373069).\nSpecific version can also be cited. Citation can be\n\n Fran\u00e7ois Orieux, & Ralph Abirizk. (2022). Q-MM: The Quadratic Majorize-Minimize\n Python toolbox (v0.12.0). Zenodo. https://doi.org/10.5281/zenodo.6373070\n\nA example of bibtex is\n\n @software{francois_orieux_2022_6373070,\n author = {Fran\u00e7ois Orieux and Ralph Abirizk},\n title = {Q-MM: The Quadratic Majorize-Minimize Python toolbox},\n month = mar,\n year = 2022,\n publisher = {Zenodo},\n version = {0.12.0},\n doi = {10.5281/zenodo.6373069},\n url = {https://doi.org/10.5281/zenodo.6373069}\n }\n\nAcknowledgement\n---------------\n\nAuthor would like to thanks [J.\nIdier](https://pagespersowp.ls2n.fr/jeromeidier/en/jerome-idier-3/), [S.\nMoussaoui](https://scholar.google.fr/citations?user=Vkr8yxkAAAAJ&hl=fr)\nand [\u00c9. Chouzenoux](http://www-syscom.univ-mlv.fr/~chouzeno/). \u00c9.\nChouzenoux has also a Matlab package that implements 3MG for image\ndeconvolution that can be found on her\n[webpage](http://www-syscom.univ-mlv.fr/~chouzeno/Logiciel.html).\n\nLicense\n-------\n\nThe project is licensed under the GPLv3 license.\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Quadratic Majorize-Minimize Python toolbox",
"version": "0.14.0",
"project_urls": {
"Author webpage": "https://pro.orieux.fr/",
"Bug Tracker": "https://github.com/forieux/qmm/issues",
"Documentation": "https://qmm.readthedocs.io/en/stable/",
"Homepage": "https://qmm.readthedocs.io/en/stable/",
"Repository": "https://github.com/forieux/qmm/"
},
"split_keywords": [
"optimization",
"mm",
"surogate",
"criterion",
"objective",
"learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a2ae0924a1a5d29e77622008b7adf96b761b5d3d60e7826b07ad3c8641299eb9",
"md5": "fe2a0d1813d8b6c739bcb9c31b905419",
"sha256": "38c205d4c3afd8f6987995ef5d0c7bb9c99b8415b5df9d0dd2f3cebf32e37260"
},
"downloads": -1,
"filename": "qmm-0.14.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fe2a0d1813d8b6c739bcb9c31b905419",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7,<3.12",
"size": 42536,
"upload_time": "2023-06-06T14:22:52",
"upload_time_iso_8601": "2023-06-06T14:22:52.581130Z",
"url": "https://files.pythonhosted.org/packages/a2/ae/0924a1a5d29e77622008b7adf96b761b5d3d60e7826b07ad3c8641299eb9/qmm-0.14.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "23f274e3d5eeb974c1d4efff8d348a155183c0d805580b56eb2b526ae44b59b4",
"md5": "4a6a97687956a9d9ebac8c0439965483",
"sha256": "990fa9d4f04aa05b6f6e5587d5ab0a3965dec56b4b9aca872f49bcd919c3c654"
},
"downloads": -1,
"filename": "qmm-0.14.0.tar.gz",
"has_sig": false,
"md5_digest": "4a6a97687956a9d9ebac8c0439965483",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<3.12",
"size": 31167,
"upload_time": "2023-06-06T14:22:54",
"upload_time_iso_8601": "2023-06-06T14:22:54.615875Z",
"url": "https://files.pythonhosted.org/packages/23/f2/74e3d5eeb974c1d4efff8d348a155183c0d805580b56eb2b526ae44b59b4/qmm-0.14.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-06 14:22:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "forieux",
"github_project": "qmm",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "qmm"
}
JSON
