<p align="center">
<img src="https://github.com/star-whale/starwhale/raw/main/docs/static/img/starwhale.png" width="600" style="max-width: 600px;">
</p>
<p align="center">
<a href=https://github.com/ambv/black>
<img src="https://img.shields.io/badge/code%20style-black-000000.svg">
</a>
<a href="https://starwhale.slack.com">
<img src="https://img.shields.io/static/v1.svg?label=chat&message=on%20slack&color=27b1ff&style=flat">
</a>
<a href="https://pypi.org/project/starwhale/">
<img src="https://img.shields.io/pypi/v/starwhale?style=flat">
</a>
<a href="https://github.com/star-whale/starwhale/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/star-whale/starwhale?style=flat">
</a>
<a href="https://pypi.org/project/starwhale/">
<img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/starwhale">
</a>
<a href="https://github.com/star-whale/starwhale/actions/workflows/client.yaml">
<img src="https://github.com/star-whale/starwhale/actions/workflows/client.yaml/badge.svg">
</a>
<a href="https://github.com/star-whale/starwhale/actions/workflows/server-ut-report.yml">
<img src="https://github.com/star-whale/starwhale/actions/workflows/server-ut-report.yml/badge.svg">
</a>
<a href="https://github.com/star-whale/starwhale/actions/workflows/console.yml">
<img src="https://github.com/star-whale/starwhale/actions/workflows/console.yml/badge.svg">
</a>
<a href='https://app.codecov.io/gh/star-whale/starwhale'>
<img alt="Codecov" src="https://img.shields.io/codecov/c/github/star-whale/starwhale?flag=controller&label=Java%20Cov">
</a>
<a href="https://app.codecov.io/gh/star-whale/starwhale">
<img alt="Codecov" src="https://img.shields.io/codecov/c/github/star-whale/starwhale?flag=standalone&label=Python%20cov">
</a>
<a href='https://artifacthub.io/packages/helm/starwhale/starwhale'>
<img src='https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/starwhale' alt='Artifact Hub'/>
</a>
<a href="https://github.com/star-whale/starwhale/actions/workflows/e2e-test.yml">
<img src='https://github.com/star-whale/starwhale/actions/workflows/e2e-test.yml/badge.svg' alt='Starwhale E2E Test'>
</a>
</p>
## What is Starwhale
Starwhale is an MLOps platform. It provides **Instance**, **Project**, **Runtime**, **Model**, and **Dataset**.
- **Instance**: Each installation of Starwhale is called an instance.
- π» **Standalone Instance**: The simplest form that requires only the Starwhale Client(`swcli`). `swcli` is written by pure python3.
- π **On-Premises Instance**: Cloud form, we call it **private cloud instance**. Kubernetes and BareMetal both meet the basic environmental requirements.
- βοΈ **Cloud Hosted Instance**: Cloud form, we call it **public cloud instance**. Starwhale team maintains the web service.
**Starwhale tries to keep concepts consistent across different types of instances. In this way, people can easily exchange data and migrate between them.**
- **Project**: The basic unit for organizing different resources.
- **ML Basic Elements**: The Machine Learning/Deep Learning running environments or artifacts. Starwhale empowers the ML/DL essential elements with packaging, versioning, reproducibility, and shareability.
- π **Runtime**: Software dependencies description to "run" a model, which includes python libraries, native libraries, native binaries, etc.
- π **Model**: The standard model format used in model delivery.
- π« **Dataset**: A unified description of how the data and labels are stored and organized. Starwhale datasets can be loaded efficiently.
- **Running Fundamentals**: Starwhale uses **Job**, **Step**, and **Task** to execute ML/DL actions like model trainingοΌ evaluation, and serving. Starwhale's **Controller-Agents** structure scales out easily.
- π₯ **Job**: A set of programs to do specific work. Each job consists of one or more steps.
- π΅ **Step**: Represents distinct stages of the work. Each step consists of one or more tasks.
- π₯ **Task**: Operation entity. Tasks are in some specific steps.
- **Scenarios**: Starwhale provides the best practice and out-of-the-box for different ML/DL scenarios.
- π **Model Training(TBD)**: Use Starwhale Python SDK to record experiment meta, metric, log, and artifact.
- π₯οΈ **Model Evaluation**: `PipelineHandler` and some report decorators can give you complete, helpful, and user-friendly evaluation reports with only a few lines of codes.
- π« **Model Serving(TBD)**: Starwhale Model can be deployed as a web service or stream service in production with deployment capability, observability, and scalability. Data scientists do not need to write ML/DL irrelevant codes.
## MNIST Quick Tour for the standalone instance
### Use Notebook
- You can try it in [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/mnist/notebook.ipynb)
- Or run [example/mnist/notebook.ipynb](example/mnist/notebook.ipynb) locally using [vscode](https://code.visualstudio.com/) or [jupyterlab](https://github.com/jupyterlab/jupyterlab)
### Use your own python env
- π° **STEP1**: Installing Starwhale
```bash
python3 -m pip install starwhale
```
- π΅ **STEP2**: Downloading the MNIST example
```bash
git clone https://github.com/star-whale/starwhale.git
```
If [git-lfs](https://git-lfs.github.com/) has not been previously installed in the local environment(the command is `git lfs install`), you need to download the trained model file.
```bash
wget https://media.githubusercontent.com/media/star-whale/starwhale/main/example/mnist/models/mnist_cnn.pt -O example/mnist/models/mnist_cnn.pt
```
- β **STEP3**: Building a runtime
```bash
cd example/runtime/pytorch
swcli runtime build .
swcli runtime list
swcli runtime info pytorch/version/latest
```
- π **STEP4**: Building a model
- Enter `example/mnist` directory:
```bash
cd ../../mnist
```
- Write some code with Starwhale Python SDK. Complete code is [here](https://github.com/star-whale/starwhale/blob/main/example/mnist/mnist/evaluator.py).
```python
import typing as t
import torch
from starwhale import Image, PipelineHandler, PPLResultIterator, multi_classification
class MNISTInference(PipelineHandler):
def __init__(self) -> None:
super().__init__()
self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
self.model = self._load_model(self.device)
def ppl(self, img: Image, **kw: t.Any) -> t.Tuple[t.List[int], t.List[float]]:
data_tensor = self._pre(img)
output = self.model(data_tensor)
return self._post(output)
@multi_classification(
confusion_matrix_normalize="all",
show_hamming_loss=True,
show_cohen_kappa_score=True,
show_roc_auc=True,
all_labels=[i for i in range(0, 10)],
)
def cmp(
self, ppl_result: PPLResultIterator
) -> t.Tuple[t.List[int], t.List[int], t.List[t.List[float]]]:
result, label, pr = [], [], []
for _data in ppl_result:
label.append(_data["annotations"]["label"])
result.extend(_data["result"][0])
pr.extend(_data["result"][1])
return label, result, pr
def _pre(self, input:bytes):
"""write some mnist preprocessing code"""
def _post(self, input:bytes):
"""write some mnist post-processing code"""
def _load_model():
"""load your pre trained model"""
```
- Define `model.yaml`.
```yaml
name: mnist
model:
- models/mnist_cnn.pt
config:
- config/hyperparam.json
run:
handler: mnist.evaluator:MNISTInference
```
- Run one command to build the model.
```bash
swcli model build .
swcli model info mnist/version/latest
```
- πΊ **STEP5**: Building a dataset
- Download MNIST RAW data files.
```bash
mkdir -p data && cd data
wget http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
wget http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
gzip -d *.gz
cd ..
ls -lah data/*
```
- Write some code with Starwhale Python SDK. Full code is [here](https://github.com/star-whale/starwhale/blob/main/example/mnist/mnist/dataset.py).
```python
import struct
import typing as t
from pathlib import Path
from starwhale import BuildExecutor
class DatasetProcessExecutor(SWDSBinBuildExecutor):
def iter_item(self) -> t.Generator[t.Tuple[t.Any, t.Any], None, None]:
root_dir = Path(__file__).parent.parent / "data"
with (root_dir / "t10k-images-idx3-ubyte").open("rb") as data_file, (
root_dir / "t10k-labels-idx1-ubyte"
).open("rb") as label_file:
_, data_number, height, width = struct.unpack(">IIII", data_file.read(16))
_, label_number = struct.unpack(">II", label_file.read(8))
print(
f">data({data_file.name}) split data:{data_number}, label:{label_number} group"
)
image_size = height * width
for i in range(0, min(data_number, label_number)):
_data = data_file.read(image_size)
_label = struct.unpack(">B", label_file.read(1))[0]
yield GrayscaleImage(
_data,
display_name=f"{i}",
shape=(height, width, 1),
), {"label": _label}
```
- Define `dataset.yaml`.
```yaml
name: mnist
handler: mnist.dataset:DatasetProcessExecutor
attr:
alignment_size: 1k
volume_size: 4M
data_mime_type: "x/grayscale"
```
- Run one command to build the dataset.
```bash
swcli dataset build .
swcli dataset info mnist/version/latest
```
Starwhale also supports build dataset with pure python sdk. You can try it in [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/datasets/sdk/dataset-sdk.ipynb).
- π **STEP6**: Running an evaluation job
```bash
swcli -vvv eval run --model mnist/version/latest --dataset mnist/version/latest --runtime pytorch/version/latest
swcli eval list
swcli eval info ${version}
```
π Now, you have completed the fundamental steps for Starwhale standalone.
Let's go ahead and finish the tutorial on the on-premises instance.
## MNIST Quick Tour for on-premises instance
- π° **STEP1**: Install minikube and helm
- [Minikube](https://minikube.sigs.k8s.io/docs/start/) 1.25+
- [Helm](https://helm.sh/docs/intro/install/) 3.2.0+
- π΅ **STEP2**: Start minikube
```bash
minikube start
```
> For users in the mainland of China, please add these startup parametersοΌ`--image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers`. If there is no kubectl bin in your machine, you may use `minikube kubectl` or `alias kubectl="minikube kubectl --"` alias command.
- π΅ **STEP3**: Installing Starwhale
```bash
helm repo add starwhale https://star-whale.github.io/charts
helm repo update
helm install --devel my-starwhale starwhale/starwhale -n starwhale --create-namespace --set minikube.enabled=true
```
After the installation is successful, the following prompt message appears:
```bash
NAME: my-starwhale
LAST DEPLOYED: Thu Jun 23 14:48:02 2022
NAMESPACE: starwhale
STATUS: deployed
REVISION: 1
NOTES:
******************************************
Chart Name: starwhale
Chart Version: 0.3.0
App Version: 0.3.0
...
Port Forward Visit:
- starwhale controller:
- run: kubectl port-forward --namespace starwhale svc/my-starwhale-controller 8082:8082
- visit: http://localhost:8082
- minio admin:
- run: kubectl port-forward --namespace starwhale svc/my-starwhale-minio 9001:9001
- visit: http://localhost:9001
- mysql:
- run: kubectl port-forward --namespace starwhale svc/my-starwhale-mysql 3306:3306
- visit: mysql -h 127.0.0.1 -P 3306 -ustarwhale -pstarwhale
******************************************
Login Info:
- starwhale: u:starwhale, p:abcd1234
- minio admin: u:minioadmin, p:minioadmin
*_* Enjoy using Starwhale. *_*
```
Then keep checking the minikube service status until all pods are running.
```bash
kubectl get pods -n starwhale
```
| NAME | READY | STATUS | RESTARTS | AGE |
|:-----|-------|--------|----------|-----|
|my-starwhale-controller-7d864558bc-vxvb8|1/1|Running|0|1m
|my-starwhale-minio-7d45db75f6-7wq9b|1/1|Running|0|2m
|my-starwhale-mysql-0|1/1|Running|0|2m
Make the Starwhale controller accessible locally with the following command:
```bash
kubectl port-forward --namespace starwhale svc/my-starwhale-controller 8082:8082
```
- β **STEP4**: Upload the artifacts to the cloud instance
> **pre-prepared artifacts**
> Before starting this tutorial, the following three artifacts should already exist on your machineοΌ
>
> - a starwhale model named mnist
> - a starwhale dataset named mnist
> - a starwhale runtime named pytorch
>
> The above three artifacts are what we built on our machine using starwhale.
1. Use swcli to operate the remote server
First, log in to the server:
```bash
swcli instance login --username starwhale --password abcd1234 --alias dev http://localhost:8082
```
2. Start copying the model, dataset, and runtime that we constructed earlier:
```bash
swcli model copy mnist/version/latest dev/project/starwhale
swcli dataset copy mnist/version/latest dev/project/starwhale
swcli runtime copy pytorch/version/latest dev/project/starwhale
```
- π **STEP5**: Use the web UI to run an evaluation
1. Log in Starwhale instance: let's use the username(starwhale) and password(abcd1234) to open the server web UI(<http://localhost:8082/>).
2. Then, we will see the project named 'project_for_mnist' that we created earlier with swcli. Click the project name, you will see the model, runtime, and dataset uploaded in the previous step.
<details>
<summary>Show the uploaded artifacts screenshots</summary>
</details>
3. Create and view an evaluation job
<details>
<summary>Show create job screenshot</summary>
</details>
**Congratulations! You have completed the evaluation process for a model.**
## Documentation, Community, and Support
- Visit [Starwhale HomePage](https://starwhale.ai).
- More information in the [official documentation](https://doc.starwhale.ai).
- For general questions and support, join the [Slack](https://starwhale.slack.com/).
- For bug reports and feature requests, please use [Github Issue](https://github.com/star-whale/starwhale/issues).
- To get community updates, follow [@starwhaleai](https://twitter.com/starwhaleai) on Twitter.
- For Starwhale artifacts, please visit:
- Python Package on [Pypi](https://pypi.org/project/starwhale/).
- Helm Charts on [Artifacthub](https://artifacthub.io/packages/helm/starwhale/starwhale).
- Docker Images on [Docker Hub](https://hub.docker.com/u/starwhaleai) and [ghcr.io](https://github.com/orgs/star-whale/packages).
- Additionally, you can always find us at *developer@starwhale.ai*.
## Contributing
πΌπ**PRs are always welcomed** ππΊ. See [Contribution to Starwhale](https://doc.starwhale.ai/docs/community/contribute) for more details.
## License
Starwhale is licensed under the [Apache License 2.0](https://github.com/star-whale/starwhale/blob/main/LICENSE).
Raw data
{
"_id": null,
"home_page": "https://github.com/star-whale/starwhale",
"name": "starwhale",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7, <3.12",
"maintainer_email": "",
"keywords": "MLOps,AI,Starwhale,Model Evaluation",
"author": "Starwhale Team",
"author_email": "developer@starwhale.ai",
"download_url": "https://files.pythonhosted.org/packages/58/87/0407d2d0b36cead4f728240553c9ebe4aaef4b8e97a3f4b9a982026d1ff4/starwhale-0.3.6.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://github.com/star-whale/starwhale/raw/main/docs/static/img/starwhale.png\" width=\"600\" style=\"max-width: 600px;\">\n</p>\n\n<p align=\"center\">\n\n<a href=https://github.com/ambv/black>\n <img src=\"https://img.shields.io/badge/code%20style-black-000000.svg\">\n</a>\n\n<a href=\"https://starwhale.slack.com\">\n <img src=\"https://img.shields.io/static/v1.svg?label=chat&message=on%20slack&color=27b1ff&style=flat\">\n</a>\n\n<a href=\"https://pypi.org/project/starwhale/\">\n <img src=\"https://img.shields.io/pypi/v/starwhale?style=flat\">\n</a>\n\n<a href=\"https://github.com/star-whale/starwhale/blob/main/LICENSE\">\n <img src=\"https://img.shields.io/github/license/star-whale/starwhale?style=flat\">\n</a>\n\n<a href=\"https://pypi.org/project/starwhale/\">\n <img alt=\"PyPI - Python Version\" src=\"https://img.shields.io/pypi/pyversions/starwhale\">\n</a>\n\n<a href=\"https://github.com/star-whale/starwhale/actions/workflows/client.yaml\">\n <img src=\"https://github.com/star-whale/starwhale/actions/workflows/client.yaml/badge.svg\">\n</a>\n\n<a href=\"https://github.com/star-whale/starwhale/actions/workflows/server-ut-report.yml\">\n <img src=\"https://github.com/star-whale/starwhale/actions/workflows/server-ut-report.yml/badge.svg\">\n</a>\n\n<a href=\"https://github.com/star-whale/starwhale/actions/workflows/console.yml\">\n <img src=\"https://github.com/star-whale/starwhale/actions/workflows/console.yml/badge.svg\">\n</a>\n\n<a href='https://app.codecov.io/gh/star-whale/starwhale'>\n <img alt=\"Codecov\" src=\"https://img.shields.io/codecov/c/github/star-whale/starwhale?flag=controller&label=Java%20Cov\">\n</a>\n\n<a href=\"https://app.codecov.io/gh/star-whale/starwhale\">\n <img alt=\"Codecov\" src=\"https://img.shields.io/codecov/c/github/star-whale/starwhale?flag=standalone&label=Python%20cov\">\n</a>\n\n<a href='https://artifacthub.io/packages/helm/starwhale/starwhale'>\n <img src='https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/starwhale' alt='Artifact Hub'/>\n</a>\n\n<a href=\"https://github.com/star-whale/starwhale/actions/workflows/e2e-test.yml\">\n <img src='https://github.com/star-whale/starwhale/actions/workflows/e2e-test.yml/badge.svg' alt='Starwhale E2E Test'>\n</a>\n\n</p>\n\n## What is Starwhale\n\nStarwhale is an MLOps platform. It provides **Instance**, **Project**, **Runtime**, **Model**, and **Dataset**.\n\n- **Instance**: Each installation of Starwhale is called an instance.\n - \ud83d\udc7b **Standalone Instance**: The simplest form that requires only the Starwhale Client(`swcli`). `swcli` is written by pure python3.\n - \ud83c\udf8d **On-Premises Instance**: Cloud form, we call it **private cloud instance**. Kubernetes and BareMetal both meet the basic environmental requirements.\n - \u2601\ufe0f **Cloud Hosted Instance**: Cloud form, we call it **public cloud instance**. Starwhale team maintains the web service.\n\n **Starwhale tries to keep concepts consistent across different types of instances. In this way, people can easily exchange data and migrate between them.**\n\n- **Project**: The basic unit for organizing different resources.\n\n- **ML Basic Elements**: The Machine Learning/Deep Learning running environments or artifacts. Starwhale empowers the ML/DL essential elements with packaging, versioning, reproducibility, and shareability.\n - \ud83d\udc0c **Runtime**: Software dependencies description to \"run\" a model, which includes python libraries, native libraries, native binaries, etc.\n - \ud83d\udc07 **Model**: The standard model format used in model delivery.\n - \ud83d\udc2b **Dataset**: A unified description of how the data and labels are stored and organized. Starwhale datasets can be loaded efficiently.\n\n- **Running Fundamentals**: Starwhale uses **Job**, **Step**, and **Task** to execute ML/DL actions like model training\uff0c evaluation, and serving. Starwhale's **Controller-Agents** structure scales out easily.\n - \ud83e\udd55 **Job**: A set of programs to do specific work. Each job consists of one or more steps.\n - \ud83c\udf35 **Step**: Represents distinct stages of the work. Each step consists of one or more tasks.\n - \ud83e\udd51 **Task**: Operation entity. Tasks are in some specific steps.\n\n- **Scenarios**: Starwhale provides the best practice and out-of-the-box for different ML/DL scenarios.\n - \ud83d\ude9d **Model Training(TBD)**: Use Starwhale Python SDK to record experiment meta, metric, log, and artifact.\n - \ud83d\udee5\ufe0f **Model Evaluation**: `PipelineHandler` and some report decorators can give you complete, helpful, and user-friendly evaluation reports with only a few lines of codes.\n - \ud83d\udeeb **Model Serving(TBD)**: Starwhale Model can be deployed as a web service or stream service in production with deployment capability, observability, and scalability. Data scientists do not need to write ML/DL irrelevant codes.\n\n## MNIST Quick Tour for the standalone instance\n\n### Use Notebook\n\n- You can try it in [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/mnist/notebook.ipynb)\n- Or run [example/mnist/notebook.ipynb](example/mnist/notebook.ipynb) locally using [vscode](https://code.visualstudio.com/) or [jupyterlab](https://github.com/jupyterlab/jupyterlab)\n\n### Use your own python env\n\n\n\n- \ud83c\udf70 **STEP1**: Installing Starwhale\n\n ```bash\n python3 -m pip install starwhale\n ```\n\n- \ud83c\udf75 **STEP2**: Downloading the MNIST example\n\n ```bash\n git clone https://github.com/star-whale/starwhale.git\n ```\n\n If [git-lfs](https://git-lfs.github.com/) has not been previously installed in the local environment(the command is `git lfs install`), you need to download the trained model file.\n\n ```bash\n wget https://media.githubusercontent.com/media/star-whale/starwhale/main/example/mnist/models/mnist_cnn.pt -O example/mnist/models/mnist_cnn.pt\n ```\n\n- \u2615 **STEP3**: Building a runtime\n\n ```bash\n cd example/runtime/pytorch\n swcli runtime build .\n swcli runtime list\n swcli runtime info pytorch/version/latest\n ```\n\n- \ud83c\udf5e **STEP4**: Building a model\n\n - Enter `example/mnist` directory:\n\n ```bash\n cd ../../mnist\n ```\n\n - Write some code with Starwhale Python SDK. Complete code is [here](https://github.com/star-whale/starwhale/blob/main/example/mnist/mnist/evaluator.py).\n\n ```python\n import typing as t\n import torch\n from starwhale import Image, PipelineHandler, PPLResultIterator, multi_classification\n\n class MNISTInference(PipelineHandler):\n def __init__(self) -> None:\n super().__init__()\n self.device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n self.model = self._load_model(self.device)\n\n def ppl(self, img: Image, **kw: t.Any) -> t.Tuple[t.List[int], t.List[float]]:\n data_tensor = self._pre(img)\n output = self.model(data_tensor)\n return self._post(output)\n\n @multi_classification(\n confusion_matrix_normalize=\"all\",\n show_hamming_loss=True,\n show_cohen_kappa_score=True,\n show_roc_auc=True,\n all_labels=[i for i in range(0, 10)],\n )\n def cmp(\n self, ppl_result: PPLResultIterator\n ) -> t.Tuple[t.List[int], t.List[int], t.List[t.List[float]]]:\n result, label, pr = [], [], []\n for _data in ppl_result:\n label.append(_data[\"annotations\"][\"label\"])\n result.extend(_data[\"result\"][0])\n pr.extend(_data[\"result\"][1])\n return label, result, pr\n\n def _pre(self, input:bytes):\n \"\"\"write some mnist preprocessing code\"\"\"\n\n def _post(self, input:bytes):\n \"\"\"write some mnist post-processing code\"\"\"\n\n def _load_model():\n \"\"\"load your pre trained model\"\"\"\n ```\n\n - Define `model.yaml`.\n\n ```yaml\n name: mnist\n model:\n - models/mnist_cnn.pt\n config:\n - config/hyperparam.json\n run:\n handler: mnist.evaluator:MNISTInference\n ```\n\n - Run one command to build the model.\n\n ```bash\n swcli model build .\n swcli model info mnist/version/latest\n ```\n\n- \ud83c\udf7a **STEP5**: Building a dataset\n\n - Download MNIST RAW data files.\n\n ```bash\n mkdir -p data && cd data\n wget http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n wget http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n gzip -d *.gz\n cd ..\n ls -lah data/*\n ```\n\n - Write some code with Starwhale Python SDK. Full code is [here](https://github.com/star-whale/starwhale/blob/main/example/mnist/mnist/dataset.py).\n\n ```python\n import struct\n import typing as t\n from pathlib import Path\n from starwhale import BuildExecutor\n\n class DatasetProcessExecutor(SWDSBinBuildExecutor):\n def iter_item(self) -> t.Generator[t.Tuple[t.Any, t.Any], None, None]:\n root_dir = Path(__file__).parent.parent / \"data\"\n\n with (root_dir / \"t10k-images-idx3-ubyte\").open(\"rb\") as data_file, (\n root_dir / \"t10k-labels-idx1-ubyte\"\n ).open(\"rb\") as label_file:\n _, data_number, height, width = struct.unpack(\">IIII\", data_file.read(16))\n _, label_number = struct.unpack(\">II\", label_file.read(8))\n print(\n f\">data({data_file.name}) split data:{data_number}, label:{label_number} group\"\n )\n image_size = height * width\n\n for i in range(0, min(data_number, label_number)):\n _data = data_file.read(image_size)\n _label = struct.unpack(\">B\", label_file.read(1))[0]\n yield GrayscaleImage(\n _data,\n display_name=f\"{i}\",\n shape=(height, width, 1),\n ), {\"label\": _label}\n ```\n\n - Define `dataset.yaml`.\n\n ```yaml\n name: mnist\n handler: mnist.dataset:DatasetProcessExecutor\n attr:\n alignment_size: 1k\n volume_size: 4M\n data_mime_type: \"x/grayscale\"\n ```\n\n - Run one command to build the dataset.\n\n ```bash\n swcli dataset build .\n swcli dataset info mnist/version/latest\n ```\n\n Starwhale also supports build dataset with pure python sdk. You can try it in [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/datasets/sdk/dataset-sdk.ipynb).\n\n- \ud83c\udf56 **STEP6**: Running an evaluation job\n\n ```bash\n swcli -vvv eval run --model mnist/version/latest --dataset mnist/version/latest --runtime pytorch/version/latest\n swcli eval list\n swcli eval info ${version}\n ```\n\n\ud83d\udc4f Now, you have completed the fundamental steps for Starwhale standalone.\n\nLet's go ahead and finish the tutorial on the on-premises instance.\n\n## MNIST Quick Tour for on-premises instance\n\n\n\n- \ud83c\udf70 **STEP1**: Install minikube and helm\n\n - [Minikube](https://minikube.sigs.k8s.io/docs/start/) 1.25+\n - [Helm](https://helm.sh/docs/intro/install/) 3.2.0+\n\n- \ud83c\udf75 **STEP2**: Start minikube\n\n ```bash\n minikube start\n ```\n\n > For users in the mainland of China, please add these startup parameters\uff1a`--image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers`. If there is no kubectl bin in your machine, you may use `minikube kubectl` or `alias kubectl=\"minikube kubectl --\"` alias command.\n\n- \ud83c\udf75 **STEP3**: Installing Starwhale\n\n ```bash\n helm repo add starwhale https://star-whale.github.io/charts\n helm repo update\n helm install --devel my-starwhale starwhale/starwhale -n starwhale --create-namespace --set minikube.enabled=true\n ```\n\n After the installation is successful, the following prompt message appears:\n\n ```bash\n NAME: my-starwhale\n LAST DEPLOYED: Thu Jun 23 14:48:02 2022\n NAMESPACE: starwhale\n STATUS: deployed\n REVISION: 1\n NOTES:\n ******************************************\n Chart Name: starwhale\n Chart Version: 0.3.0\n App Version: 0.3.0\n ...\n\n Port Forward Visit:\n - starwhale controller:\n - run: kubectl port-forward --namespace starwhale svc/my-starwhale-controller 8082:8082\n - visit: http://localhost:8082\n - minio admin:\n - run: kubectl port-forward --namespace starwhale svc/my-starwhale-minio 9001:9001\n - visit: http://localhost:9001\n - mysql:\n - run: kubectl port-forward --namespace starwhale svc/my-starwhale-mysql 3306:3306\n - visit: mysql -h 127.0.0.1 -P 3306 -ustarwhale -pstarwhale\n\n ******************************************\n Login Info:\n - starwhale: u:starwhale, p:abcd1234\n - minio admin: u:minioadmin, p:minioadmin\n\n *_* Enjoy using Starwhale. *_*\n ```\n\n Then keep checking the minikube service status until all pods are running.\n\n ```bash\n kubectl get pods -n starwhale\n ```\n\n | NAME | READY | STATUS | RESTARTS | AGE |\n |:-----|-------|--------|----------|-----|\n |my-starwhale-controller-7d864558bc-vxvb8|1/1|Running|0|1m\n |my-starwhale-minio-7d45db75f6-7wq9b|1/1|Running|0|2m\n |my-starwhale-mysql-0|1/1|Running|0|2m\n\n Make the Starwhale controller accessible locally with the following command:\n\n ```bash\n kubectl port-forward --namespace starwhale svc/my-starwhale-controller 8082:8082\n ```\n\n- \u2615 **STEP4**: Upload the artifacts to the cloud instance\n\n > **pre-prepared artifacts**\n > Before starting this tutorial, the following three artifacts should already exist on your machine\uff1a\n >\n > - a starwhale model named mnist\n > - a starwhale dataset named mnist\n > - a starwhale runtime named pytorch\n >\n > The above three artifacts are what we built on our machine using starwhale.\n\n 1. Use swcli to operate the remote server\n First, log in to the server:\n\n ```bash\n swcli instance login --username starwhale --password abcd1234 --alias dev http://localhost:8082\n ```\n\n 2. Start copying the model, dataset, and runtime that we constructed earlier:\n\n ```bash\n swcli model copy mnist/version/latest dev/project/starwhale\n swcli dataset copy mnist/version/latest dev/project/starwhale\n swcli runtime copy pytorch/version/latest dev/project/starwhale\n ```\n\n- \ud83c\udf5e **STEP5**: Use the web UI to run an evaluation\n\n 1. Log in Starwhale instance: let's use the username(starwhale) and password(abcd1234) to open the server web UI(<http://localhost:8082/>).\n\n 2. Then, we will see the project named 'project_for_mnist' that we created earlier with swcli. Click the project name, you will see the model, runtime, and dataset uploaded in the previous step.\n <details>\n <summary>Show the uploaded artifacts screenshots</summary>\n\n \n </details>\n 3. Create and view an evaluation job\n <details>\n <summary>Show create job screenshot</summary>\n\n \n </details>\n\n**Congratulations! You have completed the evaluation process for a model.**\n\n## Documentation, Community, and Support\n\n- Visit [Starwhale HomePage](https://starwhale.ai).\n- More information in the [official documentation](https://doc.starwhale.ai).\n- For general questions and support, join the [Slack](https://starwhale.slack.com/).\n- For bug reports and feature requests, please use [Github Issue](https://github.com/star-whale/starwhale/issues).\n- To get community updates, follow [@starwhaleai](https://twitter.com/starwhaleai) on Twitter.\n- For Starwhale artifacts, please visit:\n\n - Python Package on [Pypi](https://pypi.org/project/starwhale/).\n - Helm Charts on [Artifacthub](https://artifacthub.io/packages/helm/starwhale/starwhale).\n - Docker Images on [Docker Hub](https://hub.docker.com/u/starwhaleai) and [ghcr.io](https://github.com/orgs/star-whale/packages).\n\n- Additionally, you can always find us at *developer@starwhale.ai*.\n\n## Contributing\n\n\ud83c\udf3c\ud83d\udc4f**PRs are always welcomed** \ud83d\udc4d\ud83c\udf7a. See [Contribution to Starwhale](https://doc.starwhale.ai/docs/community/contribute) for more details.\n\n## License\n\nStarwhale is licensed under the [Apache License 2.0](https://github.com/star-whale/starwhale/blob/main/LICENSE).\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "An MLOps Platform for Model Evaluation",
"version": "0.3.6",
"split_keywords": [
"mlops",
"ai",
"starwhale",
"model evaluation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "205b578aa3904b9fd612219b9ec8f7479037daaa97fd1228150277faff897199",
"md5": "4cfec53a879dfdae4ed62fbe7e9cb370",
"sha256": "298bc3cafc738439be3d14ecf74c86c50cdeae731595118c90c24572947c4c20"
},
"downloads": -1,
"filename": "starwhale-0.3.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4cfec53a879dfdae4ed62fbe7e9cb370",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7, <3.12",
"size": 210201,
"upload_time": "2023-01-30T08:18:58",
"upload_time_iso_8601": "2023-01-30T08:18:58.449481Z",
"url": "https://files.pythonhosted.org/packages/20/5b/578aa3904b9fd612219b9ec8f7479037daaa97fd1228150277faff897199/starwhale-0.3.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "58870407d2d0b36cead4f728240553c9ebe4aaef4b8e97a3f4b9a982026d1ff4",
"md5": "b44be3d0ec822c42416857c9a306f457",
"sha256": "601a01a883bbb422aace96bf542e74bc4f42377368075044afef965de351edad"
},
"downloads": -1,
"filename": "starwhale-0.3.6.tar.gz",
"has_sig": false,
"md5_digest": "b44be3d0ec822c42416857c9a306f457",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7, <3.12",
"size": 171928,
"upload_time": "2023-01-30T08:19:00",
"upload_time_iso_8601": "2023-01-30T08:19:00.547533Z",
"url": "https://files.pythonhosted.org/packages/58/87/0407d2d0b36cead4f728240553c9ebe4aaef4b8e97a3f4b9a982026d1ff4/starwhale-0.3.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-30 08:19:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "star-whale",
"github_project": "starwhale",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "starwhale"
}
JSON
