# LabelGenius
A lightweight research toolkit for **multimodal classification** using **GPT** (text/image/both) and **CLIP** (zero-shot or fine-tuned). Built for quick, reproducible experiments and paper appendices.
---
## Quick Links
- **[Try the Colab Demo](https://colab.research.google.com/drive/1BvVWMQ20i7kkyAYjmz__PDT9OTXGtY3H?usp=sharing)** — Run the full tutorial in your browser
- **Case Studies** — Complete replication scripts in the `Case_analysis/` folder
- **[GitHub Repository](https://github.com/mediaccs/LabelGenius)**
---
## What Can You Do?
LabelGenius helps you:
- **Label with GPT** — Classify text, images, or multimodal data using zero-shot or few-shot prompting
- **Classify with CLIP** — Run zero-shot classification or fine-tune a lightweight model
- **Evaluate results** — Get accuracy, F1 scores, and confusion matrices automatically
- **Estimate costs** — Calculate API expenses before running large-scale experiments
---
## Installation
**From PyPI (recommended):**
```bash
pip install labelgenius
```
**Pinned version:**
```bash
pip install labelgenius==0.1.3
```
**From source:**
```bash
git clone https://github.com/mediaccs/LabelGenius
cd labelgenius
pip install -e .
```
**Requirements:** Python 3.9+, dependencies auto-installed from `pyproject.toml`. To manually install dependencies, see `pyproject.toml` for the full list. For GPT features, get an API key from [OpenAI](https://platform.openai.com/api-keys).
---
## Core Functions
### GPT Classification
| Function | Purpose |
|----------|---------|
| `classification_GPT` | Text classification (zero-shot or few-shot) |
| `generate_GPT_finetune_jsonl` | Prepare training data for fine-tuning |
| `finetune_GPT` | Fine-tune a GPT model |
### CLIP Classification (if data cannot be shared with business)
| Function | Purpose |
|----------|---------|
| `classification_CLIP_0_shot` | Zero-shot classification with CLIP |
| `classification_CLIP_finetuned` | Use a fine-tuned CLIP model |
| `finetune_CLIP` | Fine-tune CLIP on your dataset |
### Utilities
| Function | Purpose |
|----------|---------|
| `auto_verification` | Calculate classification metrics |
| `price_estimation` | Estimate OpenAI API costs |
---
## Quick Start
### Import the Library
```python
from labelgenius import (
classification_GPT,
classification_CLIP_0_shot,
finetune_CLIP,
classification_CLIP_finetuned,
auto_verification,
price_estimation,
)
```
### Prepare Your Data
- **Text only:** CSV/Excel/JSON with your text columns
- **Images only:** Directory of images (with optional ID mapping table)
- **Multimodal:** Table with `image_id` column + images at `image_dir/<image_id>.jpg`
---
## Using GPT Models
LabelGenius supports both **GPT-4.1** and **GPT-5** with different control mechanisms:
### GPT-4.1 (Temperature-Based)
Control randomness with `temperature` (0 = deterministic, 2 = creative):
```python
df = classification_GPT(
text_path="data/headlines.xlsx",
category=["sports", "politics", "tech"],
prompt="Classify this headline into one category.",
column_4_labeling=["headline"],
model="gpt-4.1",
temperature=0.7,
mode="text",
output_column_name="predicted_category"
)
```
### GPT-5 (Reasoning and Non-Reasoning)
GPT-5 supports both reasoning and non-reasoning modes controlled by `reasoning_effort`:
```python
df = classification_GPT(
text_path="data/headlines.xlsx",
category=["sports", "politics", "tech"],
prompt="Classify this headline into one category.",
column_4_labeling=["headline"],
model="gpt-5",
reasoning_effort="medium", # minimal | low | medium | high
mode="text",
output_column_name="predicted_category"
)
```
**Reasoning Modes:**
- `"minimal"` — Non-reasoning mode (fast, standard responses)
- `"low"` / `"medium"` / `"high"` — Reasoning modes (increasing reasoning depth)
---
## Function Reference
### `classification_GPT`
Classify text, images, or multimodal data using GPT models.
**Parameters:**
- `text_path` — Path to your data file (CSV/Excel/JSON)
- `category` — List of possible category labels
- `prompt` — Instruction prompt for classification
- `column_4_labeling` — Column(s) to use as input
- `model` — Model name (e.g., "gpt-4.1", "gpt-5")
- `temperature` — Randomness control for GPT-4.1 (0-2)
- `reasoning_effort` — Reasoning depth for GPT-5 ("minimal", "low", "medium", "high")
- `mode` — Input type: "text", "image", or "both"
- `output_column_name` — Name for prediction column
- `num_themes` — Number of themes for multi-label tasks (default: 1)
- `num_votes` — Number of runs for majority voting (default: 1)
**Example (Single-Label Text):**
```python
df = classification_GPT(
text_path="data/articles.csv",
category=["positive", "negative", "neutral"],
prompt="Classify the sentiment of this article.",
column_4_labeling=["title", "content"],
model="gpt-4.1",
temperature=0.5,
mode="text",
output_column_name="sentiment"
)
```
**Example (Multi-Label with Majority Voting):**
```python
prompt = """Label these 5 topics with 0 or 1. Return up to two 1s.
Answer format: [0,0,0,0,0]"""
df = classification_GPT(
text_path="data/posts.xlsx",
category=["0", "1"],
prompt=prompt,
column_4_labeling=["post_text"],
model="gpt-5",
reasoning_effort="low",
mode="text",
output_column_name="topic_labels",
num_themes=5,
num_votes=3 # Run 3 times and use majority vote
)
```
**Note:** You can control the number of labels by adjusting the prompt. For example:
- "Return exactly one 1" — Single label
- "Return up to two 1s" — Up to 2 labels
- "Return up to three 1s" — Up to 3 labels
- "Return any number of 1s" — Unrestricted multi-label
**Example (Multimodal):**
```python
df = classification_GPT(
text_path="data/products.csv",
category=["electronics", "clothing", "food"],
prompt="Classify this product based on text and image.",
column_4_labeling=["product_name", "description"],
model="gpt-4.1",
temperature=0.3,
mode="both", # Uses both text and images
image_dir="data/product_images",
output_column_name="category"
)
```
---
### `generate_GPT_finetune_jsonl`
Prepare training data in JSONL format for GPT fine-tuning.
**Parameters:**
- `df` — DataFrame containing your training data
- `output_path` — Where to save the JSONL file
- `system_prompt` — Instruction prompt for the model
- `input_col` — Column(s) to use as input
- `label_col` — Column(s) containing true labels
**Example:**
```python
generate_GPT_finetune_jsonl(
df=training_data,
output_path="finetune_data.jsonl",
system_prompt="Classify the sentiment of this review.",
input_col=["review_text"],
label_col=["sentiment"]
)
```
---
### `finetune_GPT`
Launch a GPT fine-tuning job using OpenAI's API.
**Parameters:**
- `training_file_path` — Path to your JSONL training file
- `model` — Base model to fine-tune (must be a snapshot model, i.e., model name ending with a specific date, e.g., "gpt-4o-mini-2024-07-18")
- `hyperparameters` — Dict with batch_size, learning_rate_multiplier, etc.
**Returns:** Fine-tuned model identifier
**Example:**
```python
model_id = finetune_GPT(
training_file_path="finetune_data.jsonl",
model="gpt-4o-mini-2024-07-18",
hyperparameters={
"batch_size": 8,
"learning_rate_multiplier": 0.01,
"n_epochs": 3
}
)
print(f"Fine-tuned model: {model_id}")
```
**Note:** GPT-5 fine-tuning availability varies. Use GPT-4 snapshots if needed.
**Using the fine-tuned model:**
After fine-tuning completes, you'll receive a model ID via email or in the Python output. Use this ID with `classification_GPT` by replacing the `model` parameter:
```python
# Use your fine-tuned model for classification
df = classification_GPT(
text_path="data/test.csv",
category=["positive", "negative", "neutral"],
prompt="Classify the sentiment of this review.",
column_4_labeling=["review_text"],
model="ft:gpt-4o-mini-2024-07-18:your-org:model-name:abc123", # Your fine-tuned model ID
temperature=0.5,
mode="text",
output_column_name="sentiment"
)
```
---
### `classification_CLIP_0_shot`
Perform zero-shot classification using CLIP without training.
**Parameters:**
- `text_path` — Path to your data file
- `img_dir` — Directory containing images (optional)
- `mode` — Input type: "text", "image", or "both"
- `prompt` — List of category names/descriptions for zero-shot
- `text_column` — Column(s) to use for text input
- `predict_column` — Name for prediction column
**Example (Text Only):**
```python
df = classification_CLIP_0_shot(
text_path="data/articles.csv",
mode="text",
prompt=["sports news", "political news", "technology news"],
text_column=["headline", "summary"],
predict_column="clip_category"
)
```
**Example (Image Only):**
```python
df = classification_CLIP_0_shot(
text_path="data/image_ids.csv",
img_dir="data/images",
mode="image",
prompt=["a photo of a cat", "a photo of a dog", "a photo of a bird"],
predict_column="animal_type"
)
```
**Example (Multimodal):**
```python
df = classification_CLIP_0_shot(
text_path="data/posts.csv",
img_dir="data/post_images",
mode="both",
prompt=["advertisement", "personal photo", "news image"],
text_column=["caption"],
predict_column="image_category"
)
```
---
### `finetune_CLIP`
Fine-tune CLIP on your labeled dataset with a small classification head.
**Parameters:**
- `mode` — Input type: "text", "image", or "both"
- `text_path` — Path to training data
- `text_column` — Column(s) for text input (if using text)
- `img_dir` — Image directory (if using images)
- `true_label` — Column containing true labels
- `model_name` — Filename to save the trained model
- `num_epochs` — Number of training epochs
- `batch_size` — Training batch size
- `learning_rate` — Learning rate for optimizer
**Returns:** Best validation accuracy achieved
**Example:**
```python
best_acc = finetune_CLIP(
mode="both",
text_path="data/train.csv",
text_column=["headline", "description"],
img_dir="data/train_images",
true_label="category_id",
model_name="my_clip_model.pth",
num_epochs=10,
batch_size=16,
learning_rate=1e-5
)
print(f"Best validation accuracy: {best_acc:.2%}")
```
---
### `classification_CLIP_finetuned`
Use your fine-tuned CLIP model to classify new data.
**Parameters:**
- `mode` — Input type: "text", "image", or "both"
- `text_path` — Path to test data
- `img_dir` — Image directory (if using images)
- `model_name` — Path to your trained model file
- `text_column` — Column(s) for text input
- `predict_column` — Name for prediction column
- `num_classes` — Number of categories in your task
**Example:**
```python
df = classification_CLIP_finetuned(
mode="both",
text_path="data/test.csv",
img_dir="data/test_images",
model_name="my_clip_model.pth",
text_column=["headline", "description"],
predict_column="predicted_category",
num_classes=24
)
```
---
### `auto_verification`
Calculate classification metrics including accuracy, F1 scores, and confusion matrix.
**Parameters:**
- `df` — DataFrame with predictions and true labels
- `predicted_cols` — List of prediction column names
- `true_cols` — List of true label column names
- `category` — List of possible categories
**Example:**
```python
auto_verification(
df=results,
predicted_cols=["gpt_pred", "clip_pred"],
true_cols=["true_label", "true_label"],
category=["0", "1"]
)
```
**Output:** Prints accuracy, F1 scores, and displays confusion matrix visualization.
---
### `price_estimation`
Estimate the cost of OpenAI API calls.
**Parameters:**
- `response` — OpenAI API response object
- `num_rows` — Number of data rows processed
- `input_cost_per_million` — Input token cost (in USD per 1M tokens)
- `output_cost_per_million` — Output token cost (in USD per 1M tokens)
- `num_votes` — Number of API calls per row (for majority voting)
**Returns:** Estimated total cost in USD
**Example:**
```python
cost = price_estimation(
response=api_response,
num_rows=1000,
input_cost_per_million=5.0,
output_cost_per_million=15.0,
num_votes=3
)
print(f"Estimated total cost: ${cost:.2f}")
```
---
## Important Notes
**Demo Scale:**
The Colab demo uses ~20 samples per task for speed. Expect high variance in small-sample results.
**Training Data Requirements:**
Fine-tuning typically requires hundreds of examples per class. Small datasets (~20 examples) may lead to overfitting or majority-class predictions.
**Image Requirements:**
For multimodal classification, ensure your `image_id` column matches filenames in `image_dir`.
**Model Availability:**
GPT-5 fine-tuning may not be available for all accounts. Use GPT-4 snapshots as alternatives.
---
## Contributing
We welcome contributions! Feel free to open an issue for bugs or feature requests, or submit a pull request to improve the codebase and documentation.
---
## License
LabelGenius is released under the MIT License.
Raw data
{
"_id": null,
"home_page": null,
"name": "labelgenius",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "gpt, clip, classification, multimodal, llm, openai, vision, research",
"author": null,
"author_email": "\"Jiacheng Huang, M.S.\" <huan1660@umn.edu>, \"Zhongxing Zhang, M.S.\" <zhan8889@umn.edu>, \"Chris Chao Su, Ph.D.\" <suchao@bu.edu>",
"download_url": "https://files.pythonhosted.org/packages/a4/69/d767088e1b293a51100f7a48a2b18089f392157317b31cc1c696e025d0c6/labelgenius-0.1.3.tar.gz",
"platform": null,
"description": "# LabelGenius\n\nA lightweight research toolkit for **multimodal classification** using **GPT** (text/image/both) and **CLIP** (zero-shot or fine-tuned). Built for quick, reproducible experiments and paper appendices.\n\n---\n\n## Quick Links\n\n- **[Try the Colab Demo](https://colab.research.google.com/drive/1BvVWMQ20i7kkyAYjmz__PDT9OTXGtY3H?usp=sharing)** \u2014 Run the full tutorial in your browser\n- **Case Studies** \u2014 Complete replication scripts in the `Case_analysis/` folder\n- **[GitHub Repository](https://github.com/mediaccs/LabelGenius)**\n\n---\n\n## What Can You Do?\n\nLabelGenius helps you:\n- **Label with GPT** \u2014 Classify text, images, or multimodal data using zero-shot or few-shot prompting\n- **Classify with CLIP** \u2014 Run zero-shot classification or fine-tune a lightweight model\n- **Evaluate results** \u2014 Get accuracy, F1 scores, and confusion matrices automatically\n- **Estimate costs** \u2014 Calculate API expenses before running large-scale experiments\n\n---\n\n## Installation\n\n**From PyPI (recommended):**\n```bash\npip install labelgenius\n```\n\n**Pinned version:**\n```bash\npip install labelgenius==0.1.3\n```\n\n**From source:**\n```bash\ngit clone https://github.com/mediaccs/LabelGenius\ncd labelgenius\npip install -e .\n```\n\n**Requirements:** Python 3.9+, dependencies auto-installed from `pyproject.toml`. To manually install dependencies, see `pyproject.toml` for the full list. For GPT features, get an API key from [OpenAI](https://platform.openai.com/api-keys).\n\n---\n\n## Core Functions\n\n### GPT Classification\n| Function | Purpose |\n|----------|---------|\n| `classification_GPT` | Text classification (zero-shot or few-shot) |\n| `generate_GPT_finetune_jsonl` | Prepare training data for fine-tuning |\n| `finetune_GPT` | Fine-tune a GPT model |\n\n### CLIP Classification (if data cannot be shared with business)\n| Function | Purpose |\n|----------|---------|\n| `classification_CLIP_0_shot` | Zero-shot classification with CLIP |\n| `classification_CLIP_finetuned` | Use a fine-tuned CLIP model |\n| `finetune_CLIP` | Fine-tune CLIP on your dataset |\n\n### Utilities\n| Function | Purpose |\n|----------|---------|\n| `auto_verification` | Calculate classification metrics |\n| `price_estimation` | Estimate OpenAI API costs |\n\n---\n\n## Quick Start\n\n### Import the Library\n```python\nfrom labelgenius import (\n classification_GPT,\n classification_CLIP_0_shot,\n finetune_CLIP,\n classification_CLIP_finetuned,\n auto_verification,\n price_estimation,\n)\n```\n\n### Prepare Your Data\n- **Text only:** CSV/Excel/JSON with your text columns\n- **Images only:** Directory of images (with optional ID mapping table)\n- **Multimodal:** Table with `image_id` column + images at `image_dir/<image_id>.jpg`\n\n---\n\n## Using GPT Models\n\nLabelGenius supports both **GPT-4.1** and **GPT-5** with different control mechanisms:\n\n### GPT-4.1 (Temperature-Based)\nControl randomness with `temperature` (0 = deterministic, 2 = creative):\n\n```python\ndf = classification_GPT(\n text_path=\"data/headlines.xlsx\",\n category=[\"sports\", \"politics\", \"tech\"],\n prompt=\"Classify this headline into one category.\",\n column_4_labeling=[\"headline\"],\n model=\"gpt-4.1\",\n temperature=0.7,\n mode=\"text\",\n output_column_name=\"predicted_category\"\n)\n```\n\n### GPT-5 (Reasoning and Non-Reasoning)\nGPT-5 supports both reasoning and non-reasoning modes controlled by `reasoning_effort`:\n\n```python\ndf = classification_GPT(\n text_path=\"data/headlines.xlsx\",\n category=[\"sports\", \"politics\", \"tech\"],\n prompt=\"Classify this headline into one category.\",\n column_4_labeling=[\"headline\"],\n model=\"gpt-5\",\n reasoning_effort=\"medium\", # minimal | low | medium | high\n mode=\"text\",\n output_column_name=\"predicted_category\"\n)\n```\n\n**Reasoning Modes:**\n- `\"minimal\"` \u2014 Non-reasoning mode (fast, standard responses)\n- `\"low\"` / `\"medium\"` / `\"high\"` \u2014 Reasoning modes (increasing reasoning depth)\n\n---\n\n## Function Reference\n\n### `classification_GPT`\n\nClassify text, images, or multimodal data using GPT models.\n\n**Parameters:**\n- `text_path` \u2014 Path to your data file (CSV/Excel/JSON)\n- `category` \u2014 List of possible category labels\n- `prompt` \u2014 Instruction prompt for classification\n- `column_4_labeling` \u2014 Column(s) to use as input\n- `model` \u2014 Model name (e.g., \"gpt-4.1\", \"gpt-5\")\n- `temperature` \u2014 Randomness control for GPT-4.1 (0-2)\n- `reasoning_effort` \u2014 Reasoning depth for GPT-5 (\"minimal\", \"low\", \"medium\", \"high\")\n- `mode` \u2014 Input type: \"text\", \"image\", or \"both\"\n- `output_column_name` \u2014 Name for prediction column\n- `num_themes` \u2014 Number of themes for multi-label tasks (default: 1)\n- `num_votes` \u2014 Number of runs for majority voting (default: 1)\n\n**Example (Single-Label Text):**\n```python\ndf = classification_GPT(\n text_path=\"data/articles.csv\",\n category=[\"positive\", \"negative\", \"neutral\"],\n prompt=\"Classify the sentiment of this article.\",\n column_4_labeling=[\"title\", \"content\"],\n model=\"gpt-4.1\",\n temperature=0.5,\n mode=\"text\",\n output_column_name=\"sentiment\"\n)\n```\n\n**Example (Multi-Label with Majority Voting):**\n```python\nprompt = \"\"\"Label these 5 topics with 0 or 1. Return up to two 1s.\nAnswer format: [0,0,0,0,0]\"\"\"\n\ndf = classification_GPT(\n text_path=\"data/posts.xlsx\",\n category=[\"0\", \"1\"],\n prompt=prompt,\n column_4_labeling=[\"post_text\"],\n model=\"gpt-5\",\n reasoning_effort=\"low\",\n mode=\"text\",\n output_column_name=\"topic_labels\",\n num_themes=5,\n num_votes=3 # Run 3 times and use majority vote\n)\n```\n\n**Note:** You can control the number of labels by adjusting the prompt. For example:\n- \"Return exactly one 1\" \u2014 Single label\n- \"Return up to two 1s\" \u2014 Up to 2 labels\n- \"Return up to three 1s\" \u2014 Up to 3 labels\n- \"Return any number of 1s\" \u2014 Unrestricted multi-label\n\n**Example (Multimodal):**\n```python\ndf = classification_GPT(\n text_path=\"data/products.csv\",\n category=[\"electronics\", \"clothing\", \"food\"],\n prompt=\"Classify this product based on text and image.\",\n column_4_labeling=[\"product_name\", \"description\"],\n model=\"gpt-4.1\",\n temperature=0.3,\n mode=\"both\", # Uses both text and images\n image_dir=\"data/product_images\",\n output_column_name=\"category\"\n)\n```\n\n---\n\n### `generate_GPT_finetune_jsonl`\n\nPrepare training data in JSONL format for GPT fine-tuning.\n\n**Parameters:**\n- `df` \u2014 DataFrame containing your training data\n- `output_path` \u2014 Where to save the JSONL file\n- `system_prompt` \u2014 Instruction prompt for the model\n- `input_col` \u2014 Column(s) to use as input\n- `label_col` \u2014 Column(s) containing true labels\n\n**Example:**\n```python\ngenerate_GPT_finetune_jsonl(\n df=training_data,\n output_path=\"finetune_data.jsonl\",\n system_prompt=\"Classify the sentiment of this review.\",\n input_col=[\"review_text\"],\n label_col=[\"sentiment\"]\n)\n```\n\n---\n\n### `finetune_GPT`\n\nLaunch a GPT fine-tuning job using OpenAI's API.\n\n**Parameters:**\n- `training_file_path` \u2014 Path to your JSONL training file\n- `model` \u2014 Base model to fine-tune (must be a snapshot model, i.e., model name ending with a specific date, e.g., \"gpt-4o-mini-2024-07-18\")\n- `hyperparameters` \u2014 Dict with batch_size, learning_rate_multiplier, etc.\n\n**Returns:** Fine-tuned model identifier\n\n**Example:**\n```python\nmodel_id = finetune_GPT(\n training_file_path=\"finetune_data.jsonl\",\n model=\"gpt-4o-mini-2024-07-18\",\n hyperparameters={\n \"batch_size\": 8,\n \"learning_rate_multiplier\": 0.01,\n \"n_epochs\": 3\n }\n)\nprint(f\"Fine-tuned model: {model_id}\")\n```\n\n**Note:** GPT-5 fine-tuning availability varies. Use GPT-4 snapshots if needed.\n\n**Using the fine-tuned model:**\n\nAfter fine-tuning completes, you'll receive a model ID via email or in the Python output. Use this ID with `classification_GPT` by replacing the `model` parameter:\n\n```python\n# Use your fine-tuned model for classification\ndf = classification_GPT(\n text_path=\"data/test.csv\",\n category=[\"positive\", \"negative\", \"neutral\"],\n prompt=\"Classify the sentiment of this review.\",\n column_4_labeling=[\"review_text\"],\n model=\"ft:gpt-4o-mini-2024-07-18:your-org:model-name:abc123\", # Your fine-tuned model ID\n temperature=0.5,\n mode=\"text\",\n output_column_name=\"sentiment\"\n)\n```\n\n---\n\n### `classification_CLIP_0_shot`\n\nPerform zero-shot classification using CLIP without training.\n\n**Parameters:**\n- `text_path` \u2014 Path to your data file\n- `img_dir` \u2014 Directory containing images (optional)\n- `mode` \u2014 Input type: \"text\", \"image\", or \"both\"\n- `prompt` \u2014 List of category names/descriptions for zero-shot\n- `text_column` \u2014 Column(s) to use for text input\n- `predict_column` \u2014 Name for prediction column\n\n**Example (Text Only):**\n```python\ndf = classification_CLIP_0_shot(\n text_path=\"data/articles.csv\",\n mode=\"text\",\n prompt=[\"sports news\", \"political news\", \"technology news\"],\n text_column=[\"headline\", \"summary\"],\n predict_column=\"clip_category\"\n)\n```\n\n**Example (Image Only):**\n```python\ndf = classification_CLIP_0_shot(\n text_path=\"data/image_ids.csv\",\n img_dir=\"data/images\",\n mode=\"image\",\n prompt=[\"a photo of a cat\", \"a photo of a dog\", \"a photo of a bird\"],\n predict_column=\"animal_type\"\n)\n```\n\n**Example (Multimodal):**\n```python\ndf = classification_CLIP_0_shot(\n text_path=\"data/posts.csv\",\n img_dir=\"data/post_images\",\n mode=\"both\",\n prompt=[\"advertisement\", \"personal photo\", \"news image\"],\n text_column=[\"caption\"],\n predict_column=\"image_category\"\n)\n```\n\n---\n\n### `finetune_CLIP`\n\nFine-tune CLIP on your labeled dataset with a small classification head.\n\n**Parameters:**\n- `mode` \u2014 Input type: \"text\", \"image\", or \"both\"\n- `text_path` \u2014 Path to training data\n- `text_column` \u2014 Column(s) for text input (if using text)\n- `img_dir` \u2014 Image directory (if using images)\n- `true_label` \u2014 Column containing true labels\n- `model_name` \u2014 Filename to save the trained model\n- `num_epochs` \u2014 Number of training epochs\n- `batch_size` \u2014 Training batch size\n- `learning_rate` \u2014 Learning rate for optimizer\n\n**Returns:** Best validation accuracy achieved\n\n**Example:**\n```python\nbest_acc = finetune_CLIP(\n mode=\"both\",\n text_path=\"data/train.csv\",\n text_column=[\"headline\", \"description\"],\n img_dir=\"data/train_images\",\n true_label=\"category_id\",\n model_name=\"my_clip_model.pth\",\n num_epochs=10,\n batch_size=16,\n learning_rate=1e-5\n)\nprint(f\"Best validation accuracy: {best_acc:.2%}\")\n```\n\n---\n\n### `classification_CLIP_finetuned`\n\nUse your fine-tuned CLIP model to classify new data.\n\n**Parameters:**\n- `mode` \u2014 Input type: \"text\", \"image\", or \"both\"\n- `text_path` \u2014 Path to test data\n- `img_dir` \u2014 Image directory (if using images)\n- `model_name` \u2014 Path to your trained model file\n- `text_column` \u2014 Column(s) for text input\n- `predict_column` \u2014 Name for prediction column\n- `num_classes` \u2014 Number of categories in your task\n\n**Example:**\n```python\ndf = classification_CLIP_finetuned(\n mode=\"both\",\n text_path=\"data/test.csv\",\n img_dir=\"data/test_images\",\n model_name=\"my_clip_model.pth\",\n text_column=[\"headline\", \"description\"],\n predict_column=\"predicted_category\",\n num_classes=24\n)\n```\n\n---\n\n### `auto_verification`\n\nCalculate classification metrics including accuracy, F1 scores, and confusion matrix.\n\n**Parameters:**\n- `df` \u2014 DataFrame with predictions and true labels\n- `predicted_cols` \u2014 List of prediction column names\n- `true_cols` \u2014 List of true label column names\n- `category` \u2014 List of possible categories\n\n**Example:**\n```python\nauto_verification(\n df=results,\n predicted_cols=[\"gpt_pred\", \"clip_pred\"],\n true_cols=[\"true_label\", \"true_label\"],\n category=[\"0\", \"1\"]\n)\n```\n\n**Output:** Prints accuracy, F1 scores, and displays confusion matrix visualization.\n\n---\n\n### `price_estimation`\n\nEstimate the cost of OpenAI API calls.\n\n**Parameters:**\n- `response` \u2014 OpenAI API response object\n- `num_rows` \u2014 Number of data rows processed\n- `input_cost_per_million` \u2014 Input token cost (in USD per 1M tokens)\n- `output_cost_per_million` \u2014 Output token cost (in USD per 1M tokens)\n- `num_votes` \u2014 Number of API calls per row (for majority voting)\n\n**Returns:** Estimated total cost in USD\n\n**Example:**\n```python\ncost = price_estimation(\n response=api_response,\n num_rows=1000,\n input_cost_per_million=5.0,\n output_cost_per_million=15.0,\n num_votes=3\n)\nprint(f\"Estimated total cost: ${cost:.2f}\")\n```\n\n---\n\n## Important Notes\n\n**Demo Scale:** \nThe Colab demo uses ~20 samples per task for speed. Expect high variance in small-sample results.\n\n**Training Data Requirements:** \nFine-tuning typically requires hundreds of examples per class. Small datasets (~20 examples) may lead to overfitting or majority-class predictions.\n\n**Image Requirements:** \nFor multimodal classification, ensure your `image_id` column matches filenames in `image_dir`.\n\n**Model Availability:** \nGPT-5 fine-tuning may not be available for all accounts. Use GPT-4 snapshots as alternatives.\n\n\n---\n\n## Contributing\n\nWe welcome contributions! Feel free to open an issue for bugs or feature requests, or submit a pull request to improve the codebase and documentation.\n\n---\n\n## License\n\nLabelGenius is released under the MIT License.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Flexible GPT + CLIP classification toolkit for multimodal labeling",
"version": "0.1.3",
"project_urls": {
"Homepage": "https://github.com/mediaccs/LabelGenius",
"Issues": "https://github.com/mediaccs/LabelGenius/issues",
"Repository": "https://github.com/mediaccs/LabelGenius"
},
"split_keywords": [
"gpt",
" clip",
" classification",
" multimodal",
" llm",
" openai",
" vision",
" research"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "3a7c53e5f5eced22dd6c44d9d486e7a15da4623b063b3645fb753ef6f40c7a17",
"md5": "e8f7246901c3eb7f525d45f6421c1fde",
"sha256": "f737b1074dbbd2ab2c0931c372ead94e3c9b84521e0173c3316b8c4c1b8ed79a"
},
"downloads": -1,
"filename": "labelgenius-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e8f7246901c3eb7f525d45f6421c1fde",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 18655,
"upload_time": "2025-10-09T03:06:52",
"upload_time_iso_8601": "2025-10-09T03:06:52.658350Z",
"url": "https://files.pythonhosted.org/packages/3a/7c/53e5f5eced22dd6c44d9d486e7a15da4623b063b3645fb753ef6f40c7a17/labelgenius-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a469d767088e1b293a51100f7a48a2b18089f392157317b31cc1c696e025d0c6",
"md5": "0dab69a3ea64627d0c328387b8ce756d",
"sha256": "1772cb65d6a4467228ee1ad518cdc17c340402513926a1e5d2f64b255cfd19f3"
},
"downloads": -1,
"filename": "labelgenius-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "0dab69a3ea64627d0c328387b8ce756d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 22854,
"upload_time": "2025-10-09T03:06:54",
"upload_time_iso_8601": "2025-10-09T03:06:54.515173Z",
"url": "https://files.pythonhosted.org/packages/a4/69/d767088e1b293a51100f7a48a2b18089f392157317b31cc1c696e025d0c6/labelgenius-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-09 03:06:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mediaccs",
"github_project": "LabelGenius",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "labelgenius"
}
JSON
