# distgfs
Distributed computing framework for the
[Global Function Search](http://dlib.net/optimization.html#global_function_search)
(GFS) hyperparameter optimizer from the [Dlib](http://dlib.net) library.
Based on [gfsopt](https://github.com/tsoernes/gfsopt).
Provides the following features:
* Parallel optimization: Run distributed hyperparameter searches via [mpi4py](https://github.com/mpi4py/mpi4py).
* Save and restore progress: Save/restore settings, parameters and optimization progress to/from HDF5 file.
* Average over multiple runs: Run a stochastic objective function using the same
parameters multiple times and report the average to Dlib's Global Function
Search. Useful in highly stochastic domains to avoid biasing the search towards
lucky runs.
For theoretical background of GFS, see ['A Global Optimization Algorithm Worth Using'](http://blog.dlib.net/2017/12/a-global-optimization-algorithm-worth.html) and [Malherbe & Vayatis 2017: Global optimization of Lipschitz functions](https://arxiv.org/abs/1703.02628)
# Example usage
A basic example where we maximize Levi's function with as many parallel processes as there are logical cores, and save progress to file.
```python
import math, distgfs
def levi(x, y):
"""
Levi's function (see https://en.wikipedia.org/wiki/Test_functions_for_optimization).
Has a global _minimum_ of 0 at x=1, y=1.
"""
a = math.sin(3. * math.pi * x)**2
b = (x - 1)**2 * (1 + math.sin(3. * math.pi * y)**2)
c = (y - 1)**2 * (1 + math.sin(2. * math.pi * y)**2)
return a + b + c
def obj_fun(pp, pid):
""" Objective function to be _maximized_ by GFS. """
x = pp['x']
y = pp['y']
res = levi(0.4*x, y)
print(f"Iter: {pid}\t x:{x}, y:{y}, result:{res}")
# Since Dlib maximizes, but we want to find the minimum,
# we negate the result before passing it to the Dlib optimizer.
return -res
# For this example, we pretend that we want to keep 'y' fixed at 1.0
# while optimizing 'x' in the range -4.5 to 4.5
space = {'x': [-4.5, 4.5]}
problem_parameters = {'y': 1.}
# Create an optimizer parameter set
distgfs_params = {'opt_id': 'distgfs_levi',
'obj_fun_name': 'obj_fun',
'obj_fun_module': 'example_distgfs_levi_file',
'problem_parameters': problem_parameters,
'space': space,
'n_iter': 10,
'file_path': 'distgfs.levi.h5',
'save': True,
}
distgfs.run(distgfs_params, verbose=True)
```
For additional examples, see [examples](https://github.com/iraikov/distgfs/tree/master/examples).
Raw data
{
"_id": null,
"home_page": "",
"name": "distgfs",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "optimization,distributed optimization,GFS,MPI,mpi4py,distributed computing",
"author": "Ivan Raikov",
"author_email": "ivan.g.raikov@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e8/72/0efb4f5944f44daaf3e6c2a1761d82856b88d60fa06c6a035d8f829e5148/distgfs-1.1.0.tar.gz",
"platform": null,
"description": "# distgfs\n\nDistributed computing framework for the\n[Global Function Search](http://dlib.net/optimization.html#global_function_search) \n(GFS) hyperparameter optimizer from the [Dlib](http://dlib.net) library.\nBased on [gfsopt](https://github.com/tsoernes/gfsopt).\n\nProvides the following features:\n* Parallel optimization: Run distributed hyperparameter searches via [mpi4py](https://github.com/mpi4py/mpi4py).\n* Save and restore progress: Save/restore settings, parameters and optimization progress to/from HDF5 file. \n* Average over multiple runs: Run a stochastic objective function using the same\nparameters multiple times and report the average to Dlib's Global Function\nSearch. Useful in highly stochastic domains to avoid biasing the search towards\nlucky runs.\n\nFor theoretical background of GFS, see ['A Global Optimization Algorithm Worth Using'](http://blog.dlib.net/2017/12/a-global-optimization-algorithm-worth.html) and [Malherbe & Vayatis 2017: Global optimization of Lipschitz functions](https://arxiv.org/abs/1703.02628)\n\n# Example usage\nA basic example where we maximize Levi's function with as many parallel processes as there are logical cores, and save progress to file.\n\n```python\nimport math, distgfs\n\ndef levi(x, y):\n \"\"\"\n Levi's function (see https://en.wikipedia.org/wiki/Test_functions_for_optimization).\n Has a global _minimum_ of 0 at x=1, y=1.\n \"\"\"\n a = math.sin(3. * math.pi * x)**2\n b = (x - 1)**2 * (1 + math.sin(3. * math.pi * y)**2)\n c = (y - 1)**2 * (1 + math.sin(2. * math.pi * y)**2)\n return a + b + c\n\n\ndef obj_fun(pp, pid):\n \"\"\" Objective function to be _maximized_ by GFS. \"\"\"\n x = pp['x']\n y = pp['y']\n\n res = levi(0.4*x, y)\n print(f\"Iter: {pid}\\t x:{x}, y:{y}, result:{res}\")\n # Since Dlib maximizes, but we want to find the minimum,\n # we negate the result before passing it to the Dlib optimizer.\n return -res\n\n# For this example, we pretend that we want to keep 'y' fixed at 1.0\n# while optimizing 'x' in the range -4.5 to 4.5\nspace = {'x': [-4.5, 4.5]}\nproblem_parameters = {'y': 1.}\n \n# Create an optimizer parameter set\ndistgfs_params = {'opt_id': 'distgfs_levi',\n 'obj_fun_name': 'obj_fun',\n 'obj_fun_module': 'example_distgfs_levi_file',\n 'problem_parameters': problem_parameters,\n 'space': space,\n 'n_iter': 10,\n 'file_path': 'distgfs.levi.h5',\n 'save': True,\n }\n\ndistgfs.run(distgfs_params, verbose=True)\n```\n\nFor additional examples, see [examples](https://github.com/iraikov/distgfs/tree/master/examples).\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Distributed global function search via dlib",
"version": "1.1.0",
"split_keywords": [
"optimization",
"distributed optimization",
"gfs",
"mpi",
"mpi4py",
"distributed computing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "66e605a74d64406cdd3c8774b2dd551437a6f684bf7898f5728f5faaef20befc",
"md5": "fc62e2cab96a7e4bf9b1f43bc4d77009",
"sha256": "3a872c8d0b9052ef70b040aafdafa8b7a7f94add19e73ad58ee592552ae514eb"
},
"downloads": -1,
"filename": "distgfs-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fc62e2cab96a7e4bf9b1f43bc4d77009",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 23322,
"upload_time": "2023-02-04T01:55:05",
"upload_time_iso_8601": "2023-02-04T01:55:05.660161Z",
"url": "https://files.pythonhosted.org/packages/66/e6/05a74d64406cdd3c8774b2dd551437a6f684bf7898f5728f5faaef20befc/distgfs-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e8720efb4f5944f44daaf3e6c2a1761d82856b88d60fa06c6a035d8f829e5148",
"md5": "aa72f03a9e0f0e3b2d6ea5211604e4d0",
"sha256": "0771308c2c3b2f8bd06b2d09e6a945b5db9e0b72e02777a2a215d3c483799afc"
},
"downloads": -1,
"filename": "distgfs-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "aa72f03a9e0f0e3b2d6ea5211604e4d0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 24574,
"upload_time": "2023-02-04T01:55:07",
"upload_time_iso_8601": "2023-02-04T01:55:07.595511Z",
"url": "https://files.pythonhosted.org/packages/e8/72/0efb4f5944f44daaf3e6c2a1761d82856b88d60fa06c6a035d8f829e5148/distgfs-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-04 01:55:07",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "distgfs"
}
JSON