<H1 align="center">
Search Data Explorer
</H1>
<br>
<H2 align="center">
Visualize optimization search-data via plotly in a streamlit dashboard
</H2>
The Search-Data-Explorer is a simple application specialized to visualize search-data generated from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive). It is designed as an easy-to-use tool to gain insights into multi-dimensional data, as commonly found in optimization.
I created this package, because I needed a convenient tool to visually analyse search-data during the development of gradient-free-optimization algorithms. My goal for this package is to help users get insight into the search-data and its corresponding objective-function and search-space. Building on this insight could help improve the selection of the search-space, compare models in the objective-function or explain the behaviour of the optimization algorithm.
<br>
## Disclaimer
This project is in an early development stage and is only tested manually. If you encounter bugs or have suggestions for improvements, then please open an issue.
<br>
## Installation
```console
pip install search-data-explorer
```
<br>
## How to use
The Search Data Explorer has a very simple API, that can be explained by the examples below or just execute the command "`search-data-explorer` [file]" to open the Search Data Explorer without executing a python script.
<br>
### search-data requirements
The Search Data Explorer is used by loading the search-data with a few lines of code. The search data that is loaded from file must follow the pattern below. The columns can have any name but must contain the `score`, which is always included in search-data from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive).
<table class="table">
<thead class="table-head">
<tr class="row">
<td class="cell">first column name</td>
<td class="cell">another column name</td>
<td class="cell">...</td>
<td class="cell">score</td>
</tr>
</thead>
<tbody class="table-body">
<tr class="row">
<td class="cell">0.756</td>
<td class="cell">0.1</td>
<td class="cell">0.2</td>
<td class="cell">-3</td>
</tr>
<tr class="row">
<td class="cell">0.823</td>
<td class="cell">0.3</td>
<td class="cell">0.1</td>
<td class="cell">-10</td>
</tr>
<tr class="row">
<td class="cell">...</td>
<td class="cell">...</td>
<td class="cell">...</td>
<td class="cell">...</td>
</tr>
<tr class="row">
<td class="cell">...</td>
<td class="cell">...</td>
<td class="cell">...</td>
<td class="cell">...</td>
</tr>
</tbody>
</table>
<br>
## Examples
<br>
### Load search-data by passing dataframe
You can pass the search-data directly, if you do not want to save your search-data to disk and just explore it one time after the optimization has finished.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open(search_data) # pass search-data
```
<br>
### Load search-data by passing path to file
If you already have a search-data file on disk you can pass the path to the file to the search-data-explorer.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
search_data.to_csv("search_data.csv", index=False)
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open("model1.csv") # pass path to file on disk
```
<br>
### Load search-data by browsing for file
You can just open the search-data-explorer without passing a file or path. In this case you can browse for the file via a menu inside the search-data-explorer.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
search_data.to_csv("search_data.csv", index=False)
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open() # start without passing anything and use the file explorer within the search-data-explorer
```
Raw data
{
"_id": null,
"home_page": "https://github.com/SimonBlanke/search-data-explorer",
"name": "search-data-explorer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": "",
"keywords": "visualization,data-science",
"author": "Simon Blanke",
"author_email": "simon.blanke@yahoo.com",
"download_url": "",
"platform": null,
"description": "<H1 align=\"center\">\n Search Data Explorer\n</H1>\n\n<br>\n\n<H2 align=\"center\">\n Visualize optimization search-data via plotly in a streamlit dashboard\n</H2>\n\nThe Search-Data-Explorer is a simple application specialized to visualize search-data generated from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive). It is designed as an easy-to-use tool to gain insights into multi-dimensional data, as commonly found in optimization.\n\nI created this package, because I needed a convenient tool to visually analyse search-data during the development of gradient-free-optimization algorithms. My goal for this package is to help users get insight into the search-data and its corresponding objective-function and search-space. Building on this insight could help improve the selection of the search-space, compare models in the objective-function or explain the behaviour of the optimization algorithm.\n\n\n<br>\n\n## Disclaimer\n\nThis project is in an early development stage and is only tested manually. If you encounter bugs or have suggestions for improvements, then please open an issue.\n\n\n<br>\n\n## Installation\n\n```console\npip install search-data-explorer\n```\n\n<br>\n\n## How to use\n\nThe Search Data Explorer has a very simple API, that can be explained by the examples below or just execute the command \"`search-data-explorer` [file]\" to open the Search Data Explorer without executing a python script.\n\n\n<br>\n\n### search-data requirements\n\nThe Search Data Explorer is used by loading the search-data with a few lines of code. The search data that is loaded from file must follow the pattern below. The columns can have any name but must contain the `score`, which is always included in search-data from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive).\n\n<table class=\"table\">\n<thead class=\"table-head\">\n <tr class=\"row\">\n <td class=\"cell\">first column name</td>\n <td class=\"cell\">another column name</td>\n <td class=\"cell\">...</td>\n <td class=\"cell\">score</td>\n </tr>\n</thead>\n<tbody class=\"table-body\">\n <tr class=\"row\">\n <td class=\"cell\">0.756</td>\n <td class=\"cell\">0.1</td>\n <td class=\"cell\">0.2</td>\n <td class=\"cell\">-3</td>\n </tr>\n <tr class=\"row\">\n <td class=\"cell\">0.823</td>\n <td class=\"cell\">0.3</td>\n <td class=\"cell\">0.1</td>\n <td class=\"cell\">-10</td>\n </tr>\n <tr class=\"row\">\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n </tr>\n <tr class=\"row\">\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n <td class=\"cell\">...</td>\n </tr>\n</tbody>\n</table>\n\n\n\n<br>\n\n## Examples\n\n\n<br>\n\n### Load search-data by passing dataframe\n\nYou can pass the search-data directly, if you do not want to save your search-data to disk and just explore it one time after the optimization has finished.\n\n```python\nimport numpy as np\nfrom gradient_free_optimizers import RandomSearchOptimizer\n\nfrom search_data_explorer import SearchDataExplorer\n\n\ndef parabola_function(para):\n loss = para[\"x\"] * para[\"x\"] + para[\"y\"] * para[\"y\"] + para[\"y\"] * para[\"y\"]\n return -loss\n\n\nsearch_space = {\n \"x\": np.arange(-10, 10, 0.1),\n \"y\": np.arange(-10, 10, 0.1),\n \"z\": np.arange(-10, 10, 0.1),\n}\n\n# generate search-data for this example with gradient-free-optimizers\n\nopt = RandomSearchOptimizer(search_space)\nopt.search(parabola_function, n_iter=1000)\n\nsearch_data = opt.search_data\n\n\n# Open Search-Data-Explorer\n\nsde = SearchDataExplorer()\nsde.open(search_data) # pass search-data\n```\n\n\n<br>\n\n### Load search-data by passing path to file\n\nIf you already have a search-data file on disk you can pass the path to the file to the search-data-explorer.\n\n```python\nimport numpy as np\nfrom gradient_free_optimizers import RandomSearchOptimizer\n\nfrom search_data_explorer import SearchDataExplorer\n\n\ndef parabola_function(para):\n loss = para[\"x\"] * para[\"x\"] + para[\"y\"] * para[\"y\"] + para[\"y\"] * para[\"y\"]\n return -loss\n\n\nsearch_space = {\n \"x\": np.arange(-10, 10, 0.1),\n \"y\": np.arange(-10, 10, 0.1),\n \"z\": np.arange(-10, 10, 0.1),\n}\n\n# generate search-data for this example with gradient-free-optimizers\n\nopt = RandomSearchOptimizer(search_space)\nopt.search(parabola_function, n_iter=1000)\n\nsearch_data = opt.search_data\nsearch_data.to_csv(\"search_data.csv\", index=False)\n\n\n# Open Search-Data-Explorer\n\nsde = SearchDataExplorer()\nsde.open(\"model1.csv\") # pass path to file on disk\n```\n\n\n<br>\n\n### Load search-data by browsing for file\n\nYou can just open the search-data-explorer without passing a file or path. In this case you can browse for the file via a menu inside the search-data-explorer.\n\n```python\nimport numpy as np\nfrom gradient_free_optimizers import RandomSearchOptimizer\n\nfrom search_data_explorer import SearchDataExplorer\n\n\ndef parabola_function(para):\n loss = para[\"x\"] * para[\"x\"] + para[\"y\"] * para[\"y\"] + para[\"y\"] * para[\"y\"]\n return -loss\n\n\nsearch_space = {\n \"x\": np.arange(-10, 10, 0.1),\n \"y\": np.arange(-10, 10, 0.1),\n \"z\": np.arange(-10, 10, 0.1),\n}\n\n# generate search-data for this example with gradient-free-optimizers\n\nopt = RandomSearchOptimizer(search_space)\nopt.search(parabola_function, n_iter=1000)\n\nsearch_data = opt.search_data\nsearch_data.to_csv(\"search_data.csv\", index=False)\n\n\n# Open Search-Data-Explorer\n\nsde = SearchDataExplorer()\nsde.open() # start without passing anything and use the file explorer within the search-data-explorer\n```\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/SimonBlanke/search-data-explorer"
},
"split_keywords": [
"visualization",
"data-science"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d427fa6fd6a6abe61a8da9b9f833ecb47f9de4c2dae0c6e46060648a967c5d83",
"md5": "7db65f596b89a2aeadb1ef7a57dc598b",
"sha256": "ee12f4697d6bf5e2811819fcad1e255d365ded6a0854ddc2c8bef0b45d117695"
},
"downloads": -1,
"filename": "search_data_explorer-0.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7db65f596b89a2aeadb1ef7a57dc598b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 25475,
"upload_time": "2024-03-02T07:53:39",
"upload_time_iso_8601": "2024-03-02T07:53:39.429532Z",
"url": "https://files.pythonhosted.org/packages/d4/27/fa6fd6a6abe61a8da9b9f833ecb47f9de4c2dae0c6e46060648a967c5d83/search_data_explorer-0.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-02 07:53:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SimonBlanke",
"github_project": "search-data-explorer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "search-data-explorer"
}
JSON
