# quotientai
[](https://pypi.org/project/quotientai)
## Overview
`quotientai` is an SDK and CLI built to manage artifacts (prompts, datasets), and run evaluations on [Quotient](https://quotientai.co).
## Installation
```console
pip install quotientai
```
## Usage
Create an API key on Quotient and set it as an environment variable called `QUOTIENT_API_KEY`. Then follow the examples below or see our [docs](https://docs.quotientai.co) for a more comprehensive walkthrough.
### Examples
**Create a prompt:**
```python
from quotientai import QuotientAI
quotient = QuotientAI()
new_prompt = quotient.prompts.create(
name="customer-support-inquiry"
system_prompt="You are a helpful assistant.",
user_prompt="How can I assist you today?"
)
print(new_prompt)
```
**Create a dataset:**
```python
from quotientai import QuotientAI
quotient = QuotientAI()
new_dataset = quotient.datasets.create(
name="my-sample-dataset"
description="My first dataset",
rows=[
{"input": "Sample input", "expected": "Sample output"},
{"input": "Another input", "expected": "Another output"}
]
)
print(new_dataset)
```
**Create a log with hallucination detection:**
Log an event with hallucination detection. This will create a log event in Quotient and perform hallucination detection on the model output, input, and documents. This is a fire and forget operation, so it will not block the execution of your code.
Additional examples can be found in the [examples](examples) directory.
```python
from quotientai import QuotientAI
quotient = QuotientAI()
quotient_logger = quotient.logger.init(
# Required
app_name="my-app",
environment="dev",
# dynamic labels for slicing/dicing analytics e.g. by customer, feature, etc
tags={"model": "gpt-4o", "feature": "customer-support"},
hallucination_detection=True,
inconsistency_detection=True,
)
quotient_logger.log(
model_input="Sample input",
model_output="Sample output",
# Documents from your retriever used to generate the model output
documents=[{"page_content": "Sample document"}],
# optional additional context to help with hallucination detection, e.g. rules, constraints, etc
contexts=["Sample context"],
)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "quotientai",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "quotient, evaluation, llms, machine learning, ai",
"author": "Freddie Vargus",
"author_email": "freddie@quotientai.co",
"download_url": "https://files.pythonhosted.org/packages/b2/82/c85c3117abed53ec6dec0bd8138a9ab48fe906eb98bb94976bede38b7b8b/quotientai-0.1.5.tar.gz",
"platform": null,
"description": "# quotientai\n[](https://pypi.org/project/quotientai)\n\n## Overview\n\n`quotientai` is an SDK and CLI built to manage artifacts (prompts, datasets), and run evaluations on [Quotient](https://quotientai.co).\n\n## Installation\n\n```console\npip install quotientai\n```\n\n## Usage\n\nCreate an API key on Quotient and set it as an environment variable called `QUOTIENT_API_KEY`. Then follow the examples below or see our [docs](https://docs.quotientai.co) for a more comprehensive walkthrough.\n\n### Examples\n\n**Create a prompt:**\n\n```python\nfrom quotientai import QuotientAI\n\nquotient = QuotientAI()\n\nnew_prompt = quotient.prompts.create(\n name=\"customer-support-inquiry\"\n system_prompt=\"You are a helpful assistant.\",\n user_prompt=\"How can I assist you today?\"\n)\n\nprint(new_prompt)\n```\n\n**Create a dataset:**\n\n```python\nfrom quotientai import QuotientAI\n\nquotient = QuotientAI()\n\nnew_dataset = quotient.datasets.create(\n name=\"my-sample-dataset\"\n description=\"My first dataset\",\n rows=[\n {\"input\": \"Sample input\", \"expected\": \"Sample output\"},\n {\"input\": \"Another input\", \"expected\": \"Another output\"}\n ]\n)\n\nprint(new_dataset)\n```\n\n**Create a log with hallucination detection:**\nLog an event with hallucination detection. This will create a log event in Quotient and perform hallucination detection on the model output, input, and documents. This is a fire and forget operation, so it will not block the execution of your code.\n\nAdditional examples can be found in the [examples](examples) directory.\n\n```python\nfrom quotientai import QuotientAI\n\nquotient = QuotientAI()\nquotient_logger = quotient.logger.init(\n # Required\n app_name=\"my-app\",\n environment=\"dev\",\n # dynamic labels for slicing/dicing analytics e.g. by customer, feature, etc\n tags={\"model\": \"gpt-4o\", \"feature\": \"customer-support\"},\n hallucination_detection=True,\n inconsistency_detection=True,\n)\n\nquotient_logger.log(\n model_input=\"Sample input\",\n model_output=\"Sample output\",\n # Documents from your retriever used to generate the model output\n documents=[{\"page_content\": \"Sample document\"}], \n # optional additional context to help with hallucination detection, e.g. rules, constraints, etc\n contexts=[\"Sample context\"], \n)\n```\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "CLI for evaluating large language models with Quotient",
"version": "0.1.5",
"project_urls": null,
"split_keywords": [
"quotient",
" evaluation",
" llms",
" machine learning",
" ai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "82c0caded129ddef465f15943476faa269987bf344d16187e146f1695641c48f",
"md5": "8a1cca7d025ca97b47c0e865f640ff38",
"sha256": "99f6a355af3647331d590dd95718f841f5116784cc7235008bccfc7829f6ce5c"
},
"downloads": -1,
"filename": "quotientai-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8a1cca7d025ca97b47c0e865f640ff38",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 22467,
"upload_time": "2025-02-10T23:35:23",
"upload_time_iso_8601": "2025-02-10T23:35:23.508922Z",
"url": "https://files.pythonhosted.org/packages/82/c0/caded129ddef465f15943476faa269987bf344d16187e146f1695641c48f/quotientai-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b282c85c3117abed53ec6dec0bd8138a9ab48fe906eb98bb94976bede38b7b8b",
"md5": "03677a4b5034b2065dbf933e626e2850",
"sha256": "b05bc96bd098962ca72cacd2eb7382db51fceeaed04575456cf765520ed50047"
},
"downloads": -1,
"filename": "quotientai-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "03677a4b5034b2065dbf933e626e2850",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 18199,
"upload_time": "2025-02-10T23:35:25",
"upload_time_iso_8601": "2025-02-10T23:35:25.425355Z",
"url": "https://files.pythonhosted.org/packages/b2/82/c85c3117abed53ec6dec0bd8138a9ab48fe906eb98bb94976bede38b7b8b/quotientai-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-10 23:35:25",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "quotientai"
}
JSON