| Name | rhesis-sdk JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | None |
| Summary | SDK for testing and validating LLM applications |
| upload_time | 2025-10-30 16:50:22 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
ai
llm
machine-learning
testing
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Rhesis SDK 🧠
<meta name="google-site-verification" content="muyrLNdeOT9KjYaOnfpOmGi8K5xPe8o7r_ov3kEGdXA" />
<p align="center">
<a href="https://github.com/rhesis-ai/rhesis/blob/main/LICENSE">
<img src="https://img.shields.io/badge/license-MIT-blue" alt="License">
</a>
<a href="https://pypi.org/project/rhesis-sdk/">
<img src="https://img.shields.io/pypi/v/rhesis-sdk" alt="PyPI Version">
</a>
<a href="https://pypi.org/project/rhesis-sdk/">
<img src="https://img.shields.io/pypi/pyversions/rhesis-sdk" alt="Python Versions">
</a>
<a href="https://discord.rhesis.ai">
<img src="https://img.shields.io/discord/1340989671601209408?color=7289da&label=Discord&logo=discord&logoColor=white" alt="Discord">
</a>
<a href="https://www.linkedin.com/company/rhesis-ai">
<img src="https://img.shields.io/badge/LinkedIn-Rhesis_AI-blue?logo=linkedin" alt="LinkedIn">
</a>
<a href="https://huggingface.co/rhesis">
<img src="https://img.shields.io/badge/🤗-Rhesis-yellow" alt="Hugging Face">
</a>
<a href="https://docs.rhesis.ai">
<img src="https://img.shields.io/badge/docs-rhesis.ai-blue" alt="Documentation">
</a>
</p>
> Your team defines expectations, Rhesis generates and executes thousands of test scenarios. So that you know what you ship.
The Rhesis SDK empowers developers to programmatically access curated test sets and generate comprehensive test scenarios for Gen AI applications. Transform domain expertise into automated testing: access thousands of test scenarios, generate custom validation suites, and integrate seamlessly into your workflow to keep your Gen AI robust, reliable & compliant.
<img src="https://cdn.prod.website-files.com/68c3e3b148a4fd9bcf76eb6a/68d66fa1ff10c81d4e4e4d0f_Frame%201000004352.png"
loading="lazy"
width="1392"
sizes="(max-width: 479px) 100vw, (max-width: 767px) 95vw, (max-width: 991px) 94vw, 95vw"
alt="Rhesis Platform Results"
srcset="https://cdn.prod.website-files.com/68c3e3b148a4fd9bcf76eb6a/68d66fa1ff10c81d4e4e4d0f_Frame%201000004352.png 2939w"
class="uui-layout41_lightbox-image-01-2">
## 📑 Table of Contents
- [Features](#-features)
- [Installation](#-installation)
- [Getting Started](#-getting-started)
- [Obtain an API Key](#1-obtain-an-api-key-)
- [Configure the SDK](#2-configure-the-sdk-%EF%B8%8F)
- [Quick Start](#-quick-start)
- [Working with Test Sets](#working-with-test-sets-)
- [Generating Custom Test Sets](#generating-custom-test-sets-%EF%B8%8F)
- [About Rhesis AI](#-about-rhesis-ai)
- [Community](#-community-)
- [Hugging Face](#-hugging-face)
- [Support](#-support)
- [License](#-license)
## ✨ Features
The Rhesis SDK provides programmatic access to the Rhesis testing platform:
- **Access Test Sets**: Browse and load curated test sets across multiple domains and use cases
- **Generate Test Scenarios**: Create custom test sets from prompts, requirements, or domain knowledge
- **Seamless Integration**: Integrate testing into your CI/CD pipeline and development workflow
- **Comprehensive Coverage**: Scale your testing from dozens to thousands of scenarios
- **Open Source**: MIT-licensed with full transparency and community-driven development
## 🚀 Installation
Install the Rhesis SDK using pip:
```bash
pip install rhesis-sdk
```
## 🐍 Python Requirements
Rhesis SDK requires **Python 3.10** or newer. For development, we recommend using [pyenv](https://github.com/pyenv/pyenv) to manage Python versions:
```bash
# Install pyenv
curl https://pyenv.run | bash
# Install build dependencies (Ubuntu/Debian)
sudo apt update && sudo apt install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \
libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl
# Install Python 3.10
pyenv install 3.10.17
# Set local Python version for SDK
cd sdk
pyenv local 3.10.17
# Create a virtual environment with UV
uv venv
```
## 🏁 Getting Started
### 1. Obtain an API Key 🔑
1. Visit [https://app.rhesis.ai](https://app.rhesis.ai)
2. Sign up for a Rhesis account
3. Navigate to your account settings
4. Generate a new API key
Your API key will be in the format `rh-XXXXXXXXXXXXXXXXXXXX`. Keep this key secure and never share it publicly.
> **Note:** On the Rhesis App, you can also create test sets for your own use cases and access them via the SDK. You only need to connect your GitHub account to create a test set.
### 2. Configure the SDK ⚙️
You can configure the Rhesis SDK either through environment variables or direct configuration:
#### Using Environment Variables
```bash
export RHESIS_API_KEY="your-api-key"
export RHESIS_BASE_URL="https://api.rhesis.ai" # optional
```
#### Direct Configuration
```python
import rhesis
# Set configuration directly
rhesis.api_key = "rh-XXXXXXXXXXXXXXXXXXXX"
rhesis.base_url = "https://api.rhesis.ai" # optional
```
## ⚡ Quick Start
Before you start, you can configure the Rhesis SDK either through environment variables or direct configuration, as described above.
### Working with Test Sets 📋
```python
from rhesis.sdk.entities import TestSet
from rhesis.sdk.synthesizers import PromptSynthesizer
# List all test sets
for test_set in TestSet().all():
print(test_set)
# Load a specific test set
test_set = TestSet(id="agent-or-industry-fraud-harmful")
test_set.load()
# Download test set data
test_set.download()
# Generate a new test set
prompt_synthesizer = PromptSynthesizer(
prompt="Generate tests for an insurance chatbot that can answer questions about the company's policies."
)
test_set = prompt_synthesizer.generate(num_tests=100)
```
For more detailed examples, check out our [example notebooks](https://github.com/rhesis-ai/rhesis/tree/main/examples).
### Generating Custom Test Sets 🛠️
If none of the existing test sets fit your needs, you can generate your own.
You can check out [app.rhesis.ai](http://app.rhesis.ai). There you can define requirements, scenarios and personas, and even import your existing GitHub repository.
## 🧪 About Rhesis AI
Rhesis is an open-source testing platform that transforms how Gen AI teams validate their applications. Through collaborative test management, domain expertise becomes comprehensive automated testing: legal defines requirements, marketing sets expectations, engineers build quality, and everyone knows exactly how the Gen AI application performs before users do.
**Key capabilities:**
- **Collaborative Test Management**: Your entire team contributes requirements without writing code
- **Automated Test Generation**: Generate thousands of test scenarios from team expertise
- **Comprehensive Coverage**: Scale from dozens of manual tests to thousands of automated scenarios
- **Edge Case Discovery**: Find potential failures before your users do
- **Compliance Validation**: Ensure systems meet regulatory and ethical standards
Made in Potsdam, Germany 🇩🇪
Visit [rhesis.ai](https://rhesis.ai) to learn more about our platform and services.
## 👥 Community 💬
Join our [Discord server](https://discord.rhesis.ai) to connect with other users and developers.
## 🤗 Hugging Face
You can also find us on [Hugging Face](https://huggingface.co/rhesis). There, you can find our test sets across multiple use cases.
## 🆘 Support
For questions, issues, or feature requests:
- **Documentation**: [docs.rhesis.ai](https://docs.rhesis.ai)
- **Discord Community**: [discord.rhesis.ai](https://discord.rhesis.ai)
- **GitHub Discussions**: [Community discussions](https://github.com/rhesis-ai/rhesis/discussions)
- **Email**: hello@rhesis.ai
- **Issues**: [Report bugs or request features](https://github.com/rhesis-ai/rhesis/issues)
## 📝 License
The Rhesis SDK is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
The SDK is completely open-source and freely available for use, modification, and distribution.
---
**Made with ❤️ in Potsdam, Germany 🇩🇪**
Learn more at [rhesis.ai](https://rhesis.ai)
Raw data
{
"_id": null,
"home_page": null,
"name": "rhesis-sdk",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "ai, llm, machine-learning, testing",
"author": null,
"author_email": "Engineering Team <engineering@rhesis.ai>, Harry Cruz <harry@rhesis.ai>, Nicolai Bohn <nicolai@rhesis.ai>",
"download_url": "https://files.pythonhosted.org/packages/12/79/5110b0189604d5696c05a71fef81119d729d46d969e2d2e0b0c0bee6a471/rhesis_sdk-0.4.0.tar.gz",
"platform": null,
"description": "# Rhesis SDK \ud83e\udde0\n\n<meta name=\"google-site-verification\" content=\"muyrLNdeOT9KjYaOnfpOmGi8K5xPe8o7r_ov3kEGdXA\" />\n\n<p align=\"center\">\n <a href=\"https://github.com/rhesis-ai/rhesis/blob/main/LICENSE\">\n <img src=\"https://img.shields.io/badge/license-MIT-blue\" alt=\"License\">\n </a>\n <a href=\"https://pypi.org/project/rhesis-sdk/\">\n <img src=\"https://img.shields.io/pypi/v/rhesis-sdk\" alt=\"PyPI Version\">\n </a>\n <a href=\"https://pypi.org/project/rhesis-sdk/\">\n <img src=\"https://img.shields.io/pypi/pyversions/rhesis-sdk\" alt=\"Python Versions\">\n </a>\n <a href=\"https://discord.rhesis.ai\">\n <img src=\"https://img.shields.io/discord/1340989671601209408?color=7289da&label=Discord&logo=discord&logoColor=white\" alt=\"Discord\">\n </a>\n <a href=\"https://www.linkedin.com/company/rhesis-ai\">\n <img src=\"https://img.shields.io/badge/LinkedIn-Rhesis_AI-blue?logo=linkedin\" alt=\"LinkedIn\">\n </a>\n <a href=\"https://huggingface.co/rhesis\">\n <img src=\"https://img.shields.io/badge/\ud83e\udd17-Rhesis-yellow\" alt=\"Hugging Face\">\n </a>\n <a href=\"https://docs.rhesis.ai\">\n <img src=\"https://img.shields.io/badge/docs-rhesis.ai-blue\" alt=\"Documentation\">\n </a>\n</p>\n\n> Your team defines expectations, Rhesis generates and executes thousands of test scenarios. So that you know what you ship.\n\nThe Rhesis SDK empowers developers to programmatically access curated test sets and generate comprehensive test scenarios for Gen AI applications. Transform domain expertise into automated testing: access thousands of test scenarios, generate custom validation suites, and integrate seamlessly into your workflow to keep your Gen AI robust, reliable & compliant.\n\n<img src=\"https://cdn.prod.website-files.com/68c3e3b148a4fd9bcf76eb6a/68d66fa1ff10c81d4e4e4d0f_Frame%201000004352.png\" \n loading=\"lazy\" \n width=\"1392\" \n sizes=\"(max-width: 479px) 100vw, (max-width: 767px) 95vw, (max-width: 991px) 94vw, 95vw\" \n alt=\"Rhesis Platform Results\" \n srcset=\"https://cdn.prod.website-files.com/68c3e3b148a4fd9bcf76eb6a/68d66fa1ff10c81d4e4e4d0f_Frame%201000004352.png 2939w\" \n class=\"uui-layout41_lightbox-image-01-2\">\n\n## \ud83d\udcd1 Table of Contents\n\n- [Features](#-features)\n- [Installation](#-installation)\n- [Getting Started](#-getting-started)\n - [Obtain an API Key](#1-obtain-an-api-key-)\n - [Configure the SDK](#2-configure-the-sdk-%EF%B8%8F)\n- [Quick Start](#-quick-start)\n - [Working with Test Sets](#working-with-test-sets-)\n - [Generating Custom Test Sets](#generating-custom-test-sets-%EF%B8%8F)\n- [About Rhesis AI](#-about-rhesis-ai)\n- [Community](#-community-)\n- [Hugging Face](#-hugging-face)\n- [Support](#-support)\n- [License](#-license)\n\n## \u2728 Features\n\nThe Rhesis SDK provides programmatic access to the Rhesis testing platform:\n\n- **Access Test Sets**: Browse and load curated test sets across multiple domains and use cases\n- **Generate Test Scenarios**: Create custom test sets from prompts, requirements, or domain knowledge\n- **Seamless Integration**: Integrate testing into your CI/CD pipeline and development workflow\n- **Comprehensive Coverage**: Scale your testing from dozens to thousands of scenarios\n- **Open Source**: MIT-licensed with full transparency and community-driven development\n\n## \ud83d\ude80 Installation\n\nInstall the Rhesis SDK using pip:\n\n```bash\npip install rhesis-sdk\n```\n\n## \ud83d\udc0d Python Requirements\n\nRhesis SDK requires **Python 3.10** or newer. For development, we recommend using [pyenv](https://github.com/pyenv/pyenv) to manage Python versions:\n\n```bash\n# Install pyenv\ncurl https://pyenv.run | bash\n\n# Install build dependencies (Ubuntu/Debian)\nsudo apt update && sudo apt install -y make build-essential libssl-dev zlib1g-dev \\\nlibbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \\\nlibncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl\n\n# Install Python 3.10\npyenv install 3.10.17\n\n# Set local Python version for SDK\ncd sdk\npyenv local 3.10.17\n\n# Create a virtual environment with UV\nuv venv\n```\n\n## \ud83c\udfc1 Getting Started\n\n### 1. Obtain an API Key \ud83d\udd11\n\n1. Visit [https://app.rhesis.ai](https://app.rhesis.ai)\n2. Sign up for a Rhesis account\n3. Navigate to your account settings\n4. Generate a new API key\n\nYour API key will be in the format `rh-XXXXXXXXXXXXXXXXXXXX`. Keep this key secure and never share it publicly.\n\n> **Note:** On the Rhesis App, you can also create test sets for your own use cases and access them via the SDK. You only need to connect your GitHub account to create a test set.\n\n### 2. Configure the SDK \u2699\ufe0f\n\nYou can configure the Rhesis SDK either through environment variables or direct configuration:\n\n#### Using Environment Variables\n\n```bash\nexport RHESIS_API_KEY=\"your-api-key\"\nexport RHESIS_BASE_URL=\"https://api.rhesis.ai\" # optional\n```\n\n#### Direct Configuration\n\n```python\nimport rhesis\n\n# Set configuration directly\nrhesis.api_key = \"rh-XXXXXXXXXXXXXXXXXXXX\"\nrhesis.base_url = \"https://api.rhesis.ai\" # optional\n```\n\n## \u26a1 Quick Start\n\nBefore you start, you can configure the Rhesis SDK either through environment variables or direct configuration, as described above.\n\n### Working with Test Sets \ud83d\udccb\n\n```python\nfrom rhesis.sdk.entities import TestSet\nfrom rhesis.sdk.synthesizers import PromptSynthesizer\n\n# List all test sets\nfor test_set in TestSet().all():\n print(test_set)\n\n# Load a specific test set\ntest_set = TestSet(id=\"agent-or-industry-fraud-harmful\")\ntest_set.load()\n\n# Download test set data\ntest_set.download()\n\n# Generate a new test set\nprompt_synthesizer = PromptSynthesizer(\n prompt=\"Generate tests for an insurance chatbot that can answer questions about the company's policies.\"\n)\ntest_set = prompt_synthesizer.generate(num_tests=100)\n```\n\nFor more detailed examples, check out our [example notebooks](https://github.com/rhesis-ai/rhesis/tree/main/examples).\n\n### Generating Custom Test Sets \ud83d\udee0\ufe0f\n\nIf none of the existing test sets fit your needs, you can generate your own.\n\nYou can check out [app.rhesis.ai](http://app.rhesis.ai). There you can define requirements, scenarios and personas, and even import your existing GitHub repository.\n\n## \ud83e\uddea About Rhesis AI\n\nRhesis is an open-source testing platform that transforms how Gen AI teams validate their applications. Through collaborative test management, domain expertise becomes comprehensive automated testing: legal defines requirements, marketing sets expectations, engineers build quality, and everyone knows exactly how the Gen AI application performs before users do.\n\n**Key capabilities:**\n- **Collaborative Test Management**: Your entire team contributes requirements without writing code\n- **Automated Test Generation**: Generate thousands of test scenarios from team expertise\n- **Comprehensive Coverage**: Scale from dozens of manual tests to thousands of automated scenarios\n- **Edge Case Discovery**: Find potential failures before your users do\n- **Compliance Validation**: Ensure systems meet regulatory and ethical standards\n\nMade in Potsdam, Germany \ud83c\udde9\ud83c\uddea\n\nVisit [rhesis.ai](https://rhesis.ai) to learn more about our platform and services.\n\n## \ud83d\udc65 Community \ud83d\udcac\n\nJoin our [Discord server](https://discord.rhesis.ai) to connect with other users and developers.\n\n## \ud83e\udd17 Hugging Face\n\nYou can also find us on [Hugging Face](https://huggingface.co/rhesis). There, you can find our test sets across multiple use cases.\n\n## \ud83c\udd98 Support\n\nFor questions, issues, or feature requests:\n- **Documentation**: [docs.rhesis.ai](https://docs.rhesis.ai)\n- **Discord Community**: [discord.rhesis.ai](https://discord.rhesis.ai)\n- **GitHub Discussions**: [Community discussions](https://github.com/rhesis-ai/rhesis/discussions)\n- **Email**: hello@rhesis.ai\n- **Issues**: [Report bugs or request features](https://github.com/rhesis-ai/rhesis/issues)\n\n## \ud83d\udcdd License\n\nThe Rhesis SDK is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\nThe SDK is completely open-source and freely available for use, modification, and distribution.\n\n---\n\n**Made with \u2764\ufe0f in Potsdam, Germany \ud83c\udde9\ud83c\uddea**\n\nLearn more at [rhesis.ai](https://rhesis.ai)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "SDK for testing and validating LLM applications",
"version": "0.4.0",
"project_urls": {
"Bug Tracker": "https://github.com/rhesis-ai/rhesis/issues",
"Documentation": "https://rhesis-sdk.readthedocs.io",
"Homepage": "https://rhesis.ai",
"Repository": "https://github.com/rhesis-ai/rhesis"
},
"split_keywords": [
"ai",
" llm",
" machine-learning",
" testing"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "d00318e98629e6aeef2915163d69f08423df02bfc61c723eb1b7ba83dc8e3ee6",
"md5": "429ef7888605dc59ba0a2996378aac93",
"sha256": "183ee917944d3c134d8b8e3881e715154cd83bc4da46f42d6be94daea5ef169c"
},
"downloads": -1,
"filename": "rhesis_sdk-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "429ef7888605dc59ba0a2996378aac93",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 119520,
"upload_time": "2025-10-30T16:50:20",
"upload_time_iso_8601": "2025-10-30T16:50:20.434911Z",
"url": "https://files.pythonhosted.org/packages/d0/03/18e98629e6aeef2915163d69f08423df02bfc61c723eb1b7ba83dc8e3ee6/rhesis_sdk-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "12795110b0189604d5696c05a71fef81119d729d46d969e2d2e0b0c0bee6a471",
"md5": "467338f6a5bd310028cc221f384dd7cd",
"sha256": "7795fb0e8c16a6c3fc7d80c3e7ec57eeb237467e0488102c1bed723b75dcb264"
},
"downloads": -1,
"filename": "rhesis_sdk-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "467338f6a5bd310028cc221f384dd7cd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 333493,
"upload_time": "2025-10-30T16:50:22",
"upload_time_iso_8601": "2025-10-30T16:50:22.345691Z",
"url": "https://files.pythonhosted.org/packages/12/79/5110b0189604d5696c05a71fef81119d729d46d969e2d2e0b0c0bee6a471/rhesis_sdk-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-30 16:50:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rhesis-ai",
"github_project": "rhesis",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "rhesis-sdk"
}
