# Huawei Cloud OBS store plugin for MLflow
This repository provides a MLflow plugin that allows users to use a Huawei Cloud OBS as the artifact store for MLflow.
## Implementation overview
* `huaweicloudstoreplugin`: this package includes the `HuaweiCloudObsArtifactRepository` class that is used to read and write artifacts from Huawei Cloud OBS storage.
* `setup.py` file defines entrypoints that tell MLflow to automatically associate the `obs` URIs with the `HuaweiCloudObsArtifactRepository` implementation when the `huaweicloudstoreplugin` library is installed. The entrypoints are configured as follows:
```
entry_points={
"mlflow.artifact_repository":
"obs=huaweicloudstoreplugin.store.artifact.huaweicloud_obs_artifact_repo.py:HuaweiCloudObsArtifactRepository"
]
},
```
# Usage
Install by pip on both your client and the server, and then use MLflow as normal. The Huawei Cloud OBS artifact store support will be provided automatically.
```bash
pip install huaweicloudstoreplugin
```
The plugin implements all of the MLflow artifact store APIs.
It expects Huawei Cloud Storage access credentials in the ``MLFLOW_OBS_REGION``, ``MLFLOW_OBS_ACCESS_KEY_ID`` and ``MLFLOW_OBS_SECRET_ACCESS_KEY`` environment variables,
so you must set these variables on both your client application and your MLflow tracking server.
To use Huawei Cloud OBS as an artifact store, an OBS URI of the form ``obs://<bucket>/<path>`` must be provided, as shown in the example below:
```python
import mlflow
class DemoModel(mlflow.pyfunc.PythonModel):
def predict(self, ctx, inp) -> int:
return 1
experiment = "demo"
mlflow.create_experiment(experiment, artifact_location="obs://mlflow-test/")
mlflow.set_experiment(experiment)
mlflow.pyfunc.log_model('model_test', python_model=DemoModel())
```
In the example provided above, the ``log_model`` operation creates three entries in the OBS storage ``obs://mlflow-test/$RUN_ID/artifacts/model_test/``, the MLmodel file
and the conda.yaml file associated with the model.
Raw data
{
"_id": null,
"home_page": "https://gitee.com/HuaweiCloudDeveloper/huaweicloud-mlflow-plugins",
"name": "huaweicloudstoreplugin",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "HuaweiCloud",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/06/f1/bf5f44cd6f5cbc6e9261b61d45879fef0386b9fb0fdd3cee7ab79479a248/huaweicloudstoreplugin-0.2.0.tar.gz",
"platform": null,
"description": "# Huawei Cloud OBS store plugin for MLflow\r\nThis repository provides a MLflow plugin that allows users to use a Huawei Cloud OBS as the artifact store for MLflow.\r\n\r\n## Implementation overview\r\n* `huaweicloudstoreplugin`: this package includes the `HuaweiCloudObsArtifactRepository` class that is used to read and write artifacts from Huawei Cloud OBS storage.\r\n* `setup.py` file defines entrypoints that tell MLflow to automatically associate the `obs` URIs with the `HuaweiCloudObsArtifactRepository` implementation when the `huaweicloudstoreplugin` library is installed. The entrypoints are configured as follows:\r\n\r\n```\r\nentry_points={\r\n \"mlflow.artifact_repository\":\r\n \"obs=huaweicloudstoreplugin.store.artifact.huaweicloud_obs_artifact_repo.py:HuaweiCloudObsArtifactRepository\"\r\n ]\r\n },\r\n```\r\n\r\n\r\n# Usage\r\n\r\nInstall by pip on both your client and the server, and then use MLflow as normal. The Huawei Cloud OBS artifact store support will be provided automatically.\r\n\r\n```bash\r\npip install huaweicloudstoreplugin\r\n```\r\n\r\n\r\nThe plugin implements all of the MLflow artifact store APIs.\r\nIt expects Huawei Cloud Storage access credentials in the ``MLFLOW_OBS_REGION``, ``MLFLOW_OBS_ACCESS_KEY_ID`` and ``MLFLOW_OBS_SECRET_ACCESS_KEY`` environment variables,\r\nso you must set these variables on both your client application and your MLflow tracking server.\r\nTo use Huawei Cloud OBS as an artifact store, an OBS URI of the form ``obs://<bucket>/<path>`` must be provided, as shown in the example below:\r\n\r\n```python\r\nimport mlflow\r\n\r\n\r\nclass DemoModel(mlflow.pyfunc.PythonModel):\r\n def predict(self, ctx, inp) -> int:\r\n return 1\r\n\r\n \r\nexperiment = \"demo\"\r\nmlflow.create_experiment(experiment, artifact_location=\"obs://mlflow-test/\")\r\nmlflow.set_experiment(experiment)\r\nmlflow.pyfunc.log_model('model_test', python_model=DemoModel())\r\n\r\n```\r\n\r\nIn the example provided above, the ``log_model`` operation creates three entries in the OBS storage ``obs://mlflow-test/$RUN_ID/artifacts/model_test/``, the MLmodel file\r\nand the conda.yaml file associated with the model.\r\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Plugin that provides Huawei Cloud OBS Artifact Store functionality for MLflow",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://gitee.com/HuaweiCloudDeveloper/huaweicloud-mlflow-plugins"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a0290f52a94c2742c2c52deb26e1a5c2485d1d20b26b826167a393252eef3183",
"md5": "2f06b015cf40336324c25f20d6f18361",
"sha256": "2cf28c5e50e79d380fcde622ec08812c5126d6b59ea14caa256504ec7124c94d"
},
"downloads": -1,
"filename": "huaweicloudstoreplugin-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2f06b015cf40336324c25f20d6f18361",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9309,
"upload_time": "2023-11-13T06:35:54",
"upload_time_iso_8601": "2023-11-13T06:35:54.333943Z",
"url": "https://files.pythonhosted.org/packages/a0/29/0f52a94c2742c2c52deb26e1a5c2485d1d20b26b826167a393252eef3183/huaweicloudstoreplugin-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "06f1bf5f44cd6f5cbc6e9261b61d45879fef0386b9fb0fdd3cee7ab79479a248",
"md5": "f1371110836358c194950fa5d9be0a0d",
"sha256": "af9a1bc91512864c730b8de1606da99e4bdf3161e2c452c1ab44ddb09ded314b"
},
"downloads": -1,
"filename": "huaweicloudstoreplugin-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "f1371110836358c194950fa5d9be0a0d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9754,
"upload_time": "2023-11-13T06:35:55",
"upload_time_iso_8601": "2023-11-13T06:35:55.883575Z",
"url": "https://files.pythonhosted.org/packages/06/f1/bf5f44cd6f5cbc6e9261b61d45879fef0386b9fb0fdd3cee7ab79479a248/huaweicloudstoreplugin-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-13 06:35:55",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "huaweicloudstoreplugin"
}
JSON