# arize-phoenix-evals
<p align="center">
<a href="https://pypi.org/project/arize-phoenix-evals/">
<img src="https://img.shields.io/pypi/v/arize-phoenix-evals" alt="PyPI Version">
</a>
<a href="https://arize-phoenix.readthedocs.io/projects/evals/en/latest/index.html">
<img src="https://img.shields.io/badge/docs-blue?logo=readthedocs&logoColor=white" alt="Documentation">
</a>
</p>
Phoenix Evals provides **lightweight, composable building blocks** for writing and running evaluations on LLM applications, including tools to determine relevance, toxicity, hallucination detection, and much more.
## Features
- **Works with your preferred model SDKs** via adapters (OpenAI, LiteLLM, LangChain)
- **Powerful input mapping and binding** for working with complex data structures
- **Several pre-built metrics** for common evaluation tasks like hallucination detection
- **Evaluators are natively instrumented** via OpenTelemetry tracing for observability and dataset curation
- **Blazing fast performance** - achieve up to 20x speedup with built-in concurrency and batching
- **Tons of convenience features** to improve the developer experience!
## Installation
Install Phoenix Evals 2.0 using pip:
```shell
pip install 'arize-phoenix-evals>=2.0.0' openai
```
## Quick Start
```python
from phoenix.evals import create_classifier
from phoenix.evals.llm import LLM
# Create an LLM instance
llm = LLM(provider="openai", model="gpt-4o")
# Create an evaluator
evaluator = create_classifier(
name="helpfulness",
prompt_template="Rate the response to the user query as helpful or not:\n\nQuery: {input}\nResponse: {output}",
llm=llm,
choices={"helpful": 1.0, "not_helpful": 0.0},
)
# Simple evaluation
scores = evaluator.evaluate({"input": "How do I reset?", "output": "Go to settings > reset."})
scores[0].pretty_print()
# With input mapping for nested data
scores = evaluator.evaluate(
{"data": {"query": "How do I reset?", "response": "Go to settings > reset."}},
input_mapping={"input": "data.query", "output": "data.response"}
)
scores[0].pretty_print()
```
## Evaluating Dataframes
```python
import pandas as pd
from phoenix.evals import create_classifier, evaluate_dataframe
from phoenix.evals.llm import LLM
# Create an LLM instance
llm = LLM(provider="openai", model="gpt-4o")
# Create multiple evaluators
relevance_evaluator = create_classifier(
name="relevance",
prompt_template="Is the response relevant to the query?\n\nQuery: {input}\nResponse: {output}",
llm=llm,
choices={"relevant": 1.0, "irrelevant": 0.0},
)
helpfulness_evaluator = create_classifier(
name="helpfulness",
prompt_template="Is the response helpful?\n\nQuery: {input}\nResponse: {output}",
llm=llm,
choices={"helpful": 1.0, "not_helpful": 0.0},
)
# Prepare your dataframe
df = pd.DataFrame([
{"input": "How do I reset my password?", "output": "Go to settings > account > reset password."},
{"input": "What's the weather like?", "output": "I can help you with password resets."},
])
# Evaluate the dataframe
results_df = evaluate_dataframe(
dataframe=df,
evaluators=[relevance_evaluator, helpfulness_evaluator],
)
print(results_df.head())
```
## Documentation
- **[Full Documentation](https://arize-phoenix.readthedocs.io/projects/evals/en/latest/index.html)** - Complete API reference and guides
- **[Phoenix Docs](https://arize.com/docs/phoenix)** - Detailed use-cases and examples
- **[OpenInference](https://github.com/Arize-ai/openinference)** - Auto-instrumentation libraries for frameworks
## Community
Join our community to connect with thousands of AI builders:
- π Join our [Slack community](https://arize-ai.slack.com/join/shared_invite/zt-11t1vbu4x-xkBIHmOREQnYnYDH1GDfCg).
- π Read the [Phoenix documentation](https://arize.com/docs/phoenix).
- π‘ Ask questions and provide feedback in the _#phoenix-support_ channel.
- π Leave a star on our [GitHub](https://github.com/Arize-ai/phoenix).
- π Report bugs with [GitHub Issues](https://github.com/Arize-ai/phoenix/issues).
- π Follow us on [π](https://twitter.com/ArizePhoenix).
- πΊοΈ Check out our [roadmap](https://github.com/orgs/Arize-ai/projects/45) to see where we're heading next.
Raw data
{
"_id": null,
"home_page": null,
"name": "arize-phoenix-evals",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.14,>=3.8",
"maintainer_email": null,
"keywords": "Explainability, Monitoring, Observability",
"author": null,
"author_email": "Arize AI <phoenix-devs@arize.com>",
"download_url": "https://files.pythonhosted.org/packages/08/51/de7bd0611ee4e9a229a5eca3e7458a3682c159c8ccefe332a54adb25d3ac/arize_phoenix_evals-2.5.0.tar.gz",
"platform": null,
"description": "# arize-phoenix-evals\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/arize-phoenix-evals/\">\n <img src=\"https://img.shields.io/pypi/v/arize-phoenix-evals\" alt=\"PyPI Version\">\n </a>\n <a href=\"https://arize-phoenix.readthedocs.io/projects/evals/en/latest/index.html\">\n <img src=\"https://img.shields.io/badge/docs-blue?logo=readthedocs&logoColor=white\" alt=\"Documentation\">\n </a>\n</p>\n\nPhoenix Evals provides **lightweight, composable building blocks** for writing and running evaluations on LLM applications, including tools to determine relevance, toxicity, hallucination detection, and much more.\n\n## Features\n\n- **Works with your preferred model SDKs** via adapters (OpenAI, LiteLLM, LangChain)\n- **Powerful input mapping and binding** for working with complex data structures\n- **Several pre-built metrics** for common evaluation tasks like hallucination detection\n- **Evaluators are natively instrumented** via OpenTelemetry tracing for observability and dataset curation\n- **Blazing fast performance** - achieve up to 20x speedup with built-in concurrency and batching\n- **Tons of convenience features** to improve the developer experience!\n\n## Installation\n\nInstall Phoenix Evals 2.0 using pip:\n\n```shell\npip install 'arize-phoenix-evals>=2.0.0' openai\n```\n\n## Quick Start\n\n```python\nfrom phoenix.evals import create_classifier\nfrom phoenix.evals.llm import LLM\n\n# Create an LLM instance\nllm = LLM(provider=\"openai\", model=\"gpt-4o\")\n\n# Create an evaluator\nevaluator = create_classifier(\n name=\"helpfulness\",\n prompt_template=\"Rate the response to the user query as helpful or not:\\n\\nQuery: {input}\\nResponse: {output}\",\n llm=llm,\n choices={\"helpful\": 1.0, \"not_helpful\": 0.0},\n)\n\n# Simple evaluation\nscores = evaluator.evaluate({\"input\": \"How do I reset?\", \"output\": \"Go to settings > reset.\"})\nscores[0].pretty_print()\n\n# With input mapping for nested data\nscores = evaluator.evaluate(\n {\"data\": {\"query\": \"How do I reset?\", \"response\": \"Go to settings > reset.\"}},\n input_mapping={\"input\": \"data.query\", \"output\": \"data.response\"}\n)\nscores[0].pretty_print()\n```\n\n## Evaluating Dataframes\n\n```python\nimport pandas as pd\nfrom phoenix.evals import create_classifier, evaluate_dataframe\nfrom phoenix.evals.llm import LLM\n\n# Create an LLM instance\nllm = LLM(provider=\"openai\", model=\"gpt-4o\")\n\n# Create multiple evaluators\nrelevance_evaluator = create_classifier(\n name=\"relevance\",\n prompt_template=\"Is the response relevant to the query?\\n\\nQuery: {input}\\nResponse: {output}\",\n llm=llm,\n choices={\"relevant\": 1.0, \"irrelevant\": 0.0},\n)\n\nhelpfulness_evaluator = create_classifier(\n name=\"helpfulness\",\n prompt_template=\"Is the response helpful?\\n\\nQuery: {input}\\nResponse: {output}\",\n llm=llm,\n choices={\"helpful\": 1.0, \"not_helpful\": 0.0},\n)\n\n# Prepare your dataframe\ndf = pd.DataFrame([\n {\"input\": \"How do I reset my password?\", \"output\": \"Go to settings > account > reset password.\"},\n {\"input\": \"What's the weather like?\", \"output\": \"I can help you with password resets.\"},\n])\n\n# Evaluate the dataframe\nresults_df = evaluate_dataframe(\n dataframe=df,\n evaluators=[relevance_evaluator, helpfulness_evaluator],\n)\n\nprint(results_df.head())\n```\n\n## Documentation\n\n- **[Full Documentation](https://arize-phoenix.readthedocs.io/projects/evals/en/latest/index.html)** - Complete API reference and guides\n- **[Phoenix Docs](https://arize.com/docs/phoenix)** - Detailed use-cases and examples\n- **[OpenInference](https://github.com/Arize-ai/openinference)** - Auto-instrumentation libraries for frameworks\n\n## Community\n\nJoin our community to connect with thousands of AI builders:\n\n- \ud83c\udf0d Join our [Slack community](https://arize-ai.slack.com/join/shared_invite/zt-11t1vbu4x-xkBIHmOREQnYnYDH1GDfCg).\n- \ud83d\udcda Read the [Phoenix documentation](https://arize.com/docs/phoenix).\n- \ud83d\udca1 Ask questions and provide feedback in the _#phoenix-support_ channel.\n- \ud83c\udf1f Leave a star on our [GitHub](https://github.com/Arize-ai/phoenix).\n- \ud83d\udc1e Report bugs with [GitHub Issues](https://github.com/Arize-ai/phoenix/issues).\n- \ud835\udd4f Follow us on [\ud835\udd4f](https://twitter.com/ArizePhoenix).\n- \ud83d\uddfa\ufe0f Check out our [roadmap](https://github.com/orgs/Arize-ai/projects/45) to see where we're heading next.\n",
"bugtrack_url": null,
"license": "Elastic-2.0",
"summary": "LLM Evaluations",
"version": "2.5.0",
"project_urls": {
"Documentation": "https://arize.com/docs/phoenix/",
"Issues": "https://github.com/Arize-ai/phoenix/issues",
"Source": "https://github.com/Arize-ai/phoenix"
},
"split_keywords": [
"explainability",
" monitoring",
" observability"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f0e75126a9c1c9577699d47f852802e1016e0636bff907c0f57f4f23d59607ea",
"md5": "01054e50e697ae1e95e758b8e5b4a96b",
"sha256": "ddea69bff74dc5f0417cb13c05920a42bfdc7b36572e96290acdc2875cf6832b"
},
"downloads": -1,
"filename": "arize_phoenix_evals-2.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "01054e50e697ae1e95e758b8e5b4a96b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.14,>=3.8",
"size": 134718,
"upload_time": "2025-10-07T15:27:45",
"upload_time_iso_8601": "2025-10-07T15:27:45.212560Z",
"url": "https://files.pythonhosted.org/packages/f0/e7/5126a9c1c9577699d47f852802e1016e0636bff907c0f57f4f23d59607ea/arize_phoenix_evals-2.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0851de7bd0611ee4e9a229a5eca3e7458a3682c159c8ccefe332a54adb25d3ac",
"md5": "84a0b4e75ac1cdbba0de4c22e83f61fc",
"sha256": "6f7995ee29e401a9df6ead675ea9d5dd005ccd5ffe209924f265e9b80b1cdd81"
},
"downloads": -1,
"filename": "arize_phoenix_evals-2.5.0.tar.gz",
"has_sig": false,
"md5_digest": "84a0b4e75ac1cdbba0de4c22e83f61fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.14,>=3.8",
"size": 98136,
"upload_time": "2025-10-07T15:27:46",
"upload_time_iso_8601": "2025-10-07T15:27:46.612387Z",
"url": "https://files.pythonhosted.org/packages/08/51/de7bd0611ee4e9a229a5eca3e7458a3682c159c8ccefe332a54adb25d3ac/arize_phoenix_evals-2.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-07 15:27:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Arize-ai",
"github_project": "phoenix",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "arize-phoenix-evals"
}
JSON
