# Marqtune Python Client
## Overview
The Marqtune Python Client is a Python library that provides a convenient interface for interacting with the Marqtune API. This library allows users to perform various operations, such as training models, preparing data, evaluating models, managing tasks, and working with data management endpoints.
## Installation
To use the Marqtune Python Client, you need to install it first. You can install it using pip:
```bash
pip install marqtune
```
## Getting Started
### Initializing the Client
To get started, create an instance of the Client class by providing the URL to the Marqtune API and an API key:
```python
from marqtune import Client
from marqtune.enums import DatasetType
url = "https://marqtune.marqo.ai"
api_key = "your_api_key"
marqtune_client = Client(url=url, api_key=api_key)
```
### Create Training Dataset
A training dataset is used to train machine learning models. Here is how you can create a training dataset:
```python
# Define the data schema for the dataset
data_schema = {
"my-text-1": "text",
"my-text-2": "text",
"my-image": "image_pointer",
"my-score": "score"
}
# Create a training dataset
dataset = marqtune_client.create_dataset(
dataset_name="training_dataset",
file_path="path/to/your/dataset/file.csv",
dataset_type=DatasetType.TRAINING,
data_schema=data_schema,
wait_for_completion=True
)
print(f"Training dataset created with ID: {dataset.describe()['datasetId']}")
```
Please refer to the documentation for more details on the format of dataset.
### Training a Model
With dataset ready to be used, a model can be trained based on a base model and dataset_id. The base model can be an
open_clip model or a Marqtuned model.
```python
marqtune_client.train_model(
dataset_id="dataset_id",
model_name="test_model",
base_model="ViT-B-32",
base_checkpoint="laion400m_e31",
model_type=ModelType.OPEN_CLIP,
max_training_time=600,
instance_type=InstanceType.BASIC,
hyperparameters={"parameter1": "value1", "parameter2": "value2"},
wait_for_completion=True
)
```
### Downloading a Model
After training a model, you can download it using the download_model method. You need to provide the model_id and the checkpoint number you want to download. If the checkpoint is not specified, the latest checkpoint will be downloaded.
```python
marqtune_client.model("model_id").download()
```
### Evaluating a Model
After you trained a model you can then evaluate it's performance and compare it to base model. There are 2 steps in evaluating a model:
* Create an evaluation dataset
* Evaluate the model
#### Create an Evaluation Dataset
An evaluation dataset is used for verification and evaluation of the trained model. Here is how you can create an evaluation dataset:
```python
# Define the data schema for the dataset
data_schema = {
"my_query": "text",
"my_text": "text",
"image": "image_pointer",
"score": "score"
}
# Create an evaluation dataset
dataset = marqtune_client.create_dataset(
dataset_name="evaluation_dataset",
file_path="path/to/your/dataset/file.csv",
dataset_type=DatasetType.EVALUATION,
data_schema=data_schema,
query_column="my_query",
result_columns=["my_text", "image"],
wait_for_completion=True
)
evaluation_dataset_id = dataset.describe()['datasetId']
print(f"Evaluation dataset created with ID: {evaluation_dataset_id}")
```
#### Evaluate the Model
Once the evaluation dataset is ready, you can evaluate the model using the evaluate method. You need to provide the model_id, dataset_id, checkpoint, model_type, and hyperparameters. If wait_for_completion is set to True, the method will wait for the evaluation task to complete before returning.
```python
marqtune_client.evaluate(
model="model_id",
dataset_id="evaluation_dataset_id",
checkpoint="epoch_4",
model_type=ModelType.MARQTUNED,
hyperparameters={"parameter1": "value1", "parameter2": "value2"},
wait_for_completion=True
)
```
Please refer to the documentation for more details on hyperparameters.
### Task Management
The Marqtune API provides several methods to manage tasks, such as creating datasets, training models, and evaluating models. Below are the methods available for task management:
#### Dataset Methods
* describe(): Describe the dataset.
* logs(): Get the logs for the dataset.
* download_logs(): Download the logs for the dataset.
* delete(): Delete the dataset, terminating any running operations.
#### Model Methods
* describe(): Describe the model.
* logs(): Get the logs for the model.
* download(): Download the model.
* download_logs(): Download the logs for the model.
* delete(): Delete the model, terminating any running operations.
#### Evaluation Methods
* describe(): Describe the evaluation.
* logs(): Get the logs for the evaluation.
* download_logs(): Download the logs for the evaluation.
* delete(): Delete the evaluation, terminating any running operations
### Bucket Management
Manage your system data by uploading input-data, downloading models and deleting objects when not needed.
## Documentation
For detailed information about each method and its parameters, refer to the docstrings in the source code.
Raw data
{
"_id": null,
"home_page": null,
"name": "marqtune",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": null,
"keywords": "search python marqo opensearch tensor neural semantic vector embedding",
"author": "marqo org",
"author_email": "org@marqo.io",
"download_url": "https://files.pythonhosted.org/packages/a8/a7/6e68a2fb3b37d8b750a64c890a4ddda5bc15355c3dad10d08ebb17380df8/marqtune-0.2.2.tar.gz",
"platform": null,
"description": "# Marqtune Python Client\n## Overview\nThe Marqtune Python Client is a Python library that provides a convenient interface for interacting with the Marqtune API. This library allows users to perform various operations, such as training models, preparing data, evaluating models, managing tasks, and working with data management endpoints.\n\n## Installation\nTo use the Marqtune Python Client, you need to install it first. You can install it using pip:\n\n```bash\npip install marqtune\n```\n\n## Getting Started\n### Initializing the Client\nTo get started, create an instance of the Client class by providing the URL to the Marqtune API and an API key:\n\n```python\nfrom marqtune import Client\nfrom marqtune.enums import DatasetType\n\nurl = \"https://marqtune.marqo.ai\"\napi_key = \"your_api_key\"\n\nmarqtune_client = Client(url=url, api_key=api_key)\n```\n\n### Create Training Dataset\nA training dataset is used to train machine learning models. Here is how you can create a training dataset:\n```python\n# Define the data schema for the dataset\ndata_schema = {\n \"my-text-1\": \"text\",\n \"my-text-2\": \"text\",\n \"my-image\": \"image_pointer\",\n \"my-score\": \"score\"\n}\n\n# Create a training dataset\ndataset = marqtune_client.create_dataset(\n dataset_name=\"training_dataset\",\n file_path=\"path/to/your/dataset/file.csv\",\n dataset_type=DatasetType.TRAINING,\n data_schema=data_schema,\n wait_for_completion=True\n)\n\nprint(f\"Training dataset created with ID: {dataset.describe()['datasetId']}\")\n```\n\nPlease refer to the documentation for more details on the format of dataset.\n\n### Training a Model\nWith dataset ready to be used, a model can be trained based on a base model and dataset_id. The base model can be an \nopen_clip model or a Marqtuned model. \n\n```python\nmarqtune_client.train_model(\n dataset_id=\"dataset_id\",\n model_name=\"test_model\",\n base_model=\"ViT-B-32\",\n base_checkpoint=\"laion400m_e31\",\n model_type=ModelType.OPEN_CLIP,\n max_training_time=600,\n instance_type=InstanceType.BASIC,\n hyperparameters={\"parameter1\": \"value1\", \"parameter2\": \"value2\"},\n wait_for_completion=True\n )\n```\n\n### Downloading a Model\nAfter training a model, you can download it using the download_model method. You need to provide the model_id and the checkpoint number you want to download. If the checkpoint is not specified, the latest checkpoint will be downloaded.\n```python\nmarqtune_client.model(\"model_id\").download()\n```\n\n### Evaluating a Model\nAfter you trained a model you can then evaluate it's performance and compare it to base model. There are 2 steps in evaluating a model:\n* Create an evaluation dataset\n* Evaluate the model\n\n#### Create an Evaluation Dataset\nAn evaluation dataset is used for verification and evaluation of the trained model. Here is how you can create an evaluation dataset:\n```python\n# Define the data schema for the dataset\ndata_schema = {\n \"my_query\": \"text\",\n \"my_text\": \"text\",\n \"image\": \"image_pointer\",\n \"score\": \"score\"\n}\n\n# Create an evaluation dataset\ndataset = marqtune_client.create_dataset(\n dataset_name=\"evaluation_dataset\",\n file_path=\"path/to/your/dataset/file.csv\",\n dataset_type=DatasetType.EVALUATION,\n data_schema=data_schema,\n query_column=\"my_query\",\n result_columns=[\"my_text\", \"image\"],\n wait_for_completion=True\n)\n\nevaluation_dataset_id = dataset.describe()['datasetId']\nprint(f\"Evaluation dataset created with ID: {evaluation_dataset_id}\")\n```\n#### Evaluate the Model\nOnce the evaluation dataset is ready, you can evaluate the model using the evaluate method. You need to provide the model_id, dataset_id, checkpoint, model_type, and hyperparameters. If wait_for_completion is set to True, the method will wait for the evaluation task to complete before returning.\n```python\nmarqtune_client.evaluate(\n model=\"model_id\",\n dataset_id=\"evaluation_dataset_id\",\n checkpoint=\"epoch_4\",\n model_type=ModelType.MARQTUNED,\n hyperparameters={\"parameter1\": \"value1\", \"parameter2\": \"value2\"},\n wait_for_completion=True\n )\n```\nPlease refer to the documentation for more details on hyperparameters.\n\n### Task Management\nThe Marqtune API provides several methods to manage tasks, such as creating datasets, training models, and evaluating models. Below are the methods available for task management:\n\n#### Dataset Methods\n* describe(): Describe the dataset.\n* logs(): Get the logs for the dataset.\n* download_logs(): Download the logs for the dataset.\n* delete(): Delete the dataset, terminating any running operations.\n\n#### Model Methods\n* describe(): Describe the model.\n* logs(): Get the logs for the model.\n* download(): Download the model.\n* download_logs(): Download the logs for the model.\n* delete(): Delete the model, terminating any running operations.\n\n#### Evaluation Methods\n* describe(): Describe the evaluation.\n* logs(): Get the logs for the evaluation.\n* download_logs(): Download the logs for the evaluation.\n* delete(): Delete the evaluation, terminating any running operations\n\n### Bucket Management\nManage your system data by uploading input-data, downloading models and deleting objects when not needed.\n\n## Documentation\nFor detailed information about each method and its parameters, refer to the docstrings in the source code.\n",
"bugtrack_url": null,
"license": null,
"summary": "Client for marqtune",
"version": "0.2.2",
"project_urls": null,
"split_keywords": [
"search",
"python",
"marqo",
"opensearch",
"tensor",
"neural",
"semantic",
"vector",
"embedding"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d4d684d34017dd8c03c1e6d30da27a7c17a5151c1ab140424f1a51fc363272fa",
"md5": "0023701039abf6d42de07d34950b548c",
"sha256": "ea53ef9f3abb25f48708708e9ba8567a394b2d6f0d0c3adf839f6ffe7050f65f"
},
"downloads": -1,
"filename": "marqtune-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0023701039abf6d42de07d34950b548c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3",
"size": 10294,
"upload_time": "2024-09-12T04:51:32",
"upload_time_iso_8601": "2024-09-12T04:51:32.331651Z",
"url": "https://files.pythonhosted.org/packages/d4/d6/84d34017dd8c03c1e6d30da27a7c17a5151c1ab140424f1a51fc363272fa/marqtune-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a8a76e68a2fb3b37d8b750a64c890a4ddda5bc15355c3dad10d08ebb17380df8",
"md5": "f64a2bdb3c53f461912e424e250b9c14",
"sha256": "c48333122cc58bb5f96c8586a38ac6c7a7bd10fe067b19dd541a6db93af56cb0"
},
"downloads": -1,
"filename": "marqtune-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "f64a2bdb3c53f461912e424e250b9c14",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 11014,
"upload_time": "2024-09-12T04:51:33",
"upload_time_iso_8601": "2024-09-12T04:51:33.649110Z",
"url": "https://files.pythonhosted.org/packages/a8/a7/6e68a2fb3b37d8b750a64c890a4ddda5bc15355c3dad10d08ebb17380df8/marqtune-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-12 04:51:33",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "marqtune"
}
JSON