# Overview
The `dbnd-mlflow` plugin allows storing [mlflow](https://github.com/mlflow/mlflow) metrics to DBND tracker together with duplicating them to the mlflow store.
# Install
```bash
pip install dbnd-mlflow
# or
pip install databand[mlflow]
```
# Config
```ini
[core]
# Databand store url should be defined
databand_url=http://localhost:8080
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
```
# Run example
You might need to install examples at first `pip install dbnd-examples`.
```bash
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow
# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True
```
# Explanation
<details><summary>mlflow_example code</summary>
<p>
```python
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
```
</p>
</details>
## Execution flow:
1. Run `dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True`
2. dbnd creates a new dbnd context
3. `dbnd_on_pre_init_context` hook from `dbnd_mlflow` is triggered
- a new uri is computed to be used by mlflow, e.g.:
- `dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F`
- the new uri is set to be used with `mlflow.set_tracking_uri()`
4. `mlflow_example` task starts:
1. `mlflow.start_run()`
1. `mlflow` reads `entry_points` for each installed package and finds:
- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
2. `mlflow` creates `TrackingStoreClient` using the new uri
3. uri schema instructs to use `dbnd_mlflow.tracking_store:get_dbnd_store`
- `get_dbnd_store` creates dbnd `TrackingAPIClient`
- `get_dbnd_store` creates mlflow tracking store to duplicate tracking to
- `get_dbnd_store` returns `DatabandStore` instance
2. `log_param()`/`log_metric()`
- calls to `DatabandStore`
- calls to `TrackingAPIClient`
- calls to mlflow tracking store to duplicate tracking to
3. `mlflow.end_run()`
5. `mlflow_example` ends
6. `dbnd_on_exit_context` hook from `dbnd_mlflow` is triggered
- restore original mlflow tracking uri
Raw data
{
"_id": null,
"home_page": "https://github.com/databand-ai/dbnd",
"name": "dbnd-mlflow",
"maintainer": "Evgeny Shulman",
"docs_url": null,
"requires_python": "",
"maintainer_email": "evgeny.shulman@databand.ai",
"keywords": "orchestration,data,machinelearning",
"author": "Evgeny Shulman",
"author_email": "evgeny.shulman@databand.ai",
"download_url": "https://files.pythonhosted.org/packages/13/9c/07b77b07566d74f857cb416b2fbd332a3e22c34e1ec141f0fd952b493f0a/dbnd-mlflow-1.0.22.9.tar.gz",
"platform": "any",
"description": "# Overview\n\nThe `dbnd-mlflow` plugin allows storing [mlflow](https://github.com/mlflow/mlflow) metrics to DBND tracker together with duplicating them to the mlflow store.\n\n# Install\n\n```bash\npip install dbnd-mlflow\n# or\npip install databand[mlflow]\n```\n\n# Config\n\n```ini\n[core]\n# Databand store url should be defined\ndataband_url=http://localhost:8080\n\n[mlflow_tracking]\n# Enable tracking to Databand store\ndataband_tracking=True\n\n# Optionally, define a URI for mlflow store,\n# mlflow.get_tracking_uri() is used by default\n; duplicate_tracking_to=http://mlflow-store/\n```\n\n# Run example\n\nYou might need to install examples at first `pip install dbnd-examples`.\n\n```bash\ndbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow\n\n# or set configs manually\ndbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True\n```\n\n# Explanation\n\n<details><summary>mlflow_example code</summary>\n<p>\n\n```python\nfrom dbnd import task\nfrom mlflow import start_run, end_run\nfrom mlflow import log_metric, log_param\n\n@task\ndef mlflow_example():\n start_run()\n # params\n log_param(\"param1\", randint(0, 100))\n log_param(\"param2\", randint(0, 100))\n # metrics\n log_metric(\"foo1\", random())\n log_metric(\"foo2\", random())\n end_run()\n```\n\n</p>\n</details>\n\n## Execution flow:\n\n1. Run `dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True`\n2. dbnd creates a new dbnd context\n3. `dbnd_on_pre_init_context` hook from `dbnd_mlflow` is triggered\n - a new uri is computed to be used by mlflow, e.g.:\n - `dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F`\n - the new uri is set to be used with `mlflow.set_tracking_uri()`\n4. `mlflow_example` task starts:\n 1. `mlflow.start_run()`\n 1. `mlflow` reads `entry_points` for each installed package and finds:\n - \"dbnd = dbnd_mlflow.tracking_store:get_dbnd_store\",\n - \"dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store\",\n - \"databand = dbnd_mlflow.tracking_store:get_dbnd_store\",\n - \"databand+s = dbnd_mlflow.tracking_store:get_dbnd_store\",\n 2. `mlflow` creates `TrackingStoreClient` using the new uri\n 3. uri schema instructs to use `dbnd_mlflow.tracking_store:get_dbnd_store`\n - `get_dbnd_store` creates dbnd `TrackingAPIClient`\n - `get_dbnd_store` creates mlflow tracking store to duplicate tracking to\n - `get_dbnd_store` returns `DatabandStore` instance\n 2. `log_param()`/`log_metric()`\n - calls to `DatabandStore`\n - calls to `TrackingAPIClient`\n - calls to mlflow tracking store to duplicate tracking to\n 3. `mlflow.end_run()`\n5. `mlflow_example` ends\n6. `dbnd_on_exit_context` hook from `dbnd_mlflow` is triggered\n - restore original mlflow tracking uri\n",
"bugtrack_url": null,
"license": "",
"summary": "Machine Learning Orchestration",
"version": "1.0.22.9",
"project_urls": {
"Bug-Tracker": "https://github.com/databand-ai/dbnd/issues",
"Documentation": "https://dbnd.readme.io/",
"Homepage": "https://github.com/databand-ai/dbnd",
"Source-Code": "https://github.com/databand-ai/dbnd"
},
"split_keywords": [
"orchestration",
"data",
"machinelearning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e74ece06f3a8c6a1d893389d788b0117b761839a395c5f19b8074892b5708eed",
"md5": "15886c5c4397b12051cd45cd595b68d0",
"sha256": "7ee5215ada2fc1b44a72cfceeeb1af87382a686dfd3714d1ba2d0a75ff771a80"
},
"downloads": -1,
"filename": "dbnd_mlflow-1.0.22.9-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "15886c5c4397b12051cd45cd595b68d0",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 12740,
"upload_time": "2024-03-14T15:51:03",
"upload_time_iso_8601": "2024-03-14T15:51:03.628733Z",
"url": "https://files.pythonhosted.org/packages/e7/4e/ce06f3a8c6a1d893389d788b0117b761839a395c5f19b8074892b5708eed/dbnd_mlflow-1.0.22.9-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "139c07b77b07566d74f857cb416b2fbd332a3e22c34e1ec141f0fd952b493f0a",
"md5": "c116bc8d30db3495cbf5840d3b31a821",
"sha256": "7741f5cbed0b57c69cf1638fcf985393f87d5db5227595a8fe4f77ee8f8863be"
},
"downloads": -1,
"filename": "dbnd-mlflow-1.0.22.9.tar.gz",
"has_sig": false,
"md5_digest": "c116bc8d30db3495cbf5840d3b31a821",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12314,
"upload_time": "2024-03-14T15:53:05",
"upload_time_iso_8601": "2024-03-14T15:53:05.548593Z",
"url": "https://files.pythonhosted.org/packages/13/9c/07b77b07566d74f857cb416b2fbd332a3e22c34e1ec141f0fd952b493f0a/dbnd-mlflow-1.0.22.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-14 15:53:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "databand-ai",
"github_project": "dbnd",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "dbnd-mlflow"
}
JSON
