# Inspeqai Python SDK
- **Website:** [Inspeq.ai](https://www.inspeq.ai)
- **Inspeq App:** [Inspeq App](https://app.inspeq.ai)
- **Detailed Documentation:** [Inspeq Documentation](https://docs.inspeq.ai)
## Quickstart Guide
### Creating a Virtual Environment
To ensure a clean and isolated environment for your project, it’s recommended to create a virtual environment.
#### Linux/MacOS
1. Open a terminal.
2. Navigate to your project directory.
3. Create a virtual environment:
```bash
python3 -m venv venv
```
4. Activate the virtual environment:
```bash
source venv/bin/activate
```
#### Windows
1. Open a terminal.
2. Navigate to your project directory.
3. Create a virtual environment:
```bash
python -m venv venv
```
4. Activate the virtual environment:
```bash
venv\Scripts\activate
```
**Note:** Ensure your environment is activated every time you use the package.
### SDK Installation
Install the Inspeqai SDK using pip:
```sh
pip install inspeqai
```
### Obtain SDK API Keys
Get your API keys from [here](https://app.inspeq.ai/).
### Usage
Create a `main.py` file and use the following code snippet to get started:
```python
from inspeq.client import InspeqEval
# Initialization
API_KEY = "your_inspeq_sdk_key"
PROJECT_ID = "your_project_id"
from inspeq.client import InspeqEval
inspeq_eval = InspeqEval(inspeq_api_key=API_KEY, project_id=PROJECT_ID)
input_data = [{
"llm_input_query": "string",
"llm_input_context": "string",
"llm_output": "string"
}]
metrics_config = {
"response_tone_config": {
"threshold": 0.5,
"custom_labels": [
"Negative",
"Neutral",
"Positive"
],
"label_thresholds": [
0,
0.5,
0.7,
1
]
}
}
try:
results = inspeq_eval.evaluate_llm_task(
input_data=input_data,
task_name="your_task_name",
metrics_config=metrics_config,
metrics_list=["response_tone"]
)
print(results)
except ValueError as e:
print(e)
```
### All Metrics Provided by Inspeq SDK
Different metrics require different parameters. You can visit the [official documentation](https://docs.inspeq.ai/) for detailed information.
### Supported Features
**Metrics:**
- **Factual Consistency:** Ensures the precision and correctness of information in the generated text compared to the given context or anticipated factual knowledge.
- **Do Not Use Keywords:** Verifies that certain keywords are not present in the response.
- **Answer Relevance:** Assesses the alignment between the model's responses and the intended meaning of the input.
- **Word Limit Test:** Checks if the generated text adheres to specified word limits.
- **Response Tonality:** Analyzes the type of tone or overall sentiment highlighted in the response.
- **Conceptual Similarity:** Measures the semantic similarity or relatedness between the generated response and the provided context.
- **Coherence:** Evaluates the organization, structure, and ease of understanding of the generated text.
- **Readability:** Assesses if the generated text is appropriate for the target audience’s reading level.
- **Clarity:** Measures the clarity of the response in terms of language and structure.
- **Model Refusal:** Detects if the model responds with a refusal response when appropriate.
- **Data Leakage:** Identifies if the model response contains any personal information such as credit card numbers, phone numbers, emails, URLs, etc.
- **Creativity:** Evaluates the creativity of the generated content based on lexical diversity, contextual similarity, and hallucination score.
- **Diversity:** Measures the diversity of vocabulary used in the text.
- **Narrative Continuity:** Evaluates whether the generated response maintains coherence and logical flow with the preceding narrative.
---
### Additional Resources
For more detailed information and advanced usage, refer to the [Inspeq Documentation](https://docs.inspeq.ai/).
By following these instructions, you will be able to efficiently set up and utilize the Inspeqai Python SDK. If you have any questions or need further assistance, please refer to the official documentation or contact support.
---
This version includes detailed steps for setting up the environment, installing the SDK, and using it, as well as a comprehensive list of features supported by the Inspeq SDK. It also highlights the importance of the `project_id` parameter.
Raw data
{
"_id": null,
"home_page": null,
"name": "inspeqai",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Inspeq",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/20/be/24b13e6a56c18f5431518cc2b6030ca0cf611bb2d1084fa71cef24c0af04/inspeqai-1.0.23.tar.gz",
"platform": null,
"description": "# Inspeqai Python SDK\n\n- **Website:** [Inspeq.ai](https://www.inspeq.ai)\n- **Inspeq App:** [Inspeq App](https://app.inspeq.ai)\n- **Detailed Documentation:** [Inspeq Documentation](https://docs.inspeq.ai)\n\n## Quickstart Guide\n\n### Creating a Virtual Environment\n\nTo ensure a clean and isolated environment for your project, it\u2019s recommended to create a virtual environment.\n\n#### Linux/MacOS\n\n1. Open a terminal.\n2. Navigate to your project directory.\n3. Create a virtual environment:\n\n ```bash\n python3 -m venv venv\n ```\n\n4. Activate the virtual environment:\n\n ```bash\n source venv/bin/activate\n ```\n\n#### Windows\n\n1. Open a terminal.\n2. Navigate to your project directory.\n3. Create a virtual environment:\n\n ```bash\n python -m venv venv\n ```\n\n4. Activate the virtual environment:\n\n ```bash\n venv\\Scripts\\activate\n ```\n\n**Note:** Ensure your environment is activated every time you use the package.\n\n### SDK Installation\n\nInstall the Inspeqai SDK using pip:\n\n```sh\npip install inspeqai\n```\n\n### Obtain SDK API Keys\n\nGet your API keys from [here](https://app.inspeq.ai/).\n\n### Usage\n\nCreate a `main.py` file and use the following code snippet to get started:\n\n```python\nfrom inspeq.client import InspeqEval\n\n# Initialization\nAPI_KEY = \"your_inspeq_sdk_key\"\nPROJECT_ID = \"your_project_id\"\n\nfrom inspeq.client import InspeqEval\n\ninspeq_eval = InspeqEval(inspeq_api_key=API_KEY, project_id=PROJECT_ID)\n\ninput_data = [{\n \"llm_input_query\": \"string\", \n \"llm_input_context\": \"string\", \n \"llm_output\": \"string\" \n}]\n\nmetrics_config = {\n \"response_tone_config\": {\n \"threshold\": 0.5,\n \"custom_labels\": [\n \"Negative\",\n \"Neutral\",\n \"Positive\"\n ],\n \"label_thresholds\": [\n 0,\n 0.5,\n 0.7,\n 1\n ]\n }\n}\n\ntry:\n results = inspeq_eval.evaluate_llm_task(\n input_data=input_data,\n task_name=\"your_task_name\",\n metrics_config=metrics_config,\n metrics_list=[\"response_tone\"]\n )\n print(results)\nexcept ValueError as e:\n print(e)\n\n```\n\n### All Metrics Provided by Inspeq SDK\n\nDifferent metrics require different parameters. You can visit the [official documentation](https://docs.inspeq.ai/) for detailed information.\n\n### Supported Features\n\n**Metrics:**\n\n- **Factual Consistency:** Ensures the precision and correctness of information in the generated text compared to the given context or anticipated factual knowledge.\n- **Do Not Use Keywords:** Verifies that certain keywords are not present in the response.\n- **Answer Relevance:** Assesses the alignment between the model's responses and the intended meaning of the input.\n- **Word Limit Test:** Checks if the generated text adheres to specified word limits.\n- **Response Tonality:** Analyzes the type of tone or overall sentiment highlighted in the response.\n- **Conceptual Similarity:** Measures the semantic similarity or relatedness between the generated response and the provided context.\n- **Coherence:** Evaluates the organization, structure, and ease of understanding of the generated text.\n- **Readability:** Assesses if the generated text is appropriate for the target audience\u2019s reading level.\n- **Clarity:** Measures the clarity of the response in terms of language and structure.\n- **Model Refusal:** Detects if the model responds with a refusal response when appropriate.\n- **Data Leakage:** Identifies if the model response contains any personal information such as credit card numbers, phone numbers, emails, URLs, etc.\n- **Creativity:** Evaluates the creativity of the generated content based on lexical diversity, contextual similarity, and hallucination score.\n- **Diversity:** Measures the diversity of vocabulary used in the text.\n- **Narrative Continuity:** Evaluates whether the generated response maintains coherence and logical flow with the preceding narrative.\n\n---\n\n### Additional Resources\n\nFor more detailed information and advanced usage, refer to the [Inspeq Documentation](https://docs.inspeq.ai/).\n\nBy following these instructions, you will be able to efficiently set up and utilize the Inspeqai Python SDK. If you have any questions or need further assistance, please refer to the official documentation or contact support.\n\n---\n\nThis version includes detailed steps for setting up the environment, installing the SDK, and using it, as well as a comprehensive list of features supported by the Inspeq SDK. It also highlights the importance of the `project_id` parameter.\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Inspeq AI SDK",
"version": "1.0.23",
"project_urls": {
"Documentation": "https://docs.inspeq.ai",
"Source": "https://github.com/inspeq/inspeq-py-sdk"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "20be24b13e6a56c18f5431518cc2b6030ca0cf611bb2d1084fa71cef24c0af04",
"md5": "47e428c55b76e714a27c66cb6400f6d7",
"sha256": "c6ab5d15382c9bb01ceb25c678cb1dd0a1a9fa4a5bc43b67e70f8c999aa0a432"
},
"downloads": -1,
"filename": "inspeqai-1.0.23.tar.gz",
"has_sig": false,
"md5_digest": "47e428c55b76e714a27c66cb6400f6d7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 11299,
"upload_time": "2024-07-06T07:38:25",
"upload_time_iso_8601": "2024-07-06T07:38:25.040766Z",
"url": "https://files.pythonhosted.org/packages/20/be/24b13e6a56c18f5431518cc2b6030ca0cf611bb2d1084fa71cef24c0af04/inspeqai-1.0.23.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-06 07:38:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "inspeq",
"github_project": "inspeq-py-sdk",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "certifi",
"specs": [
[
"==",
"2023.11.17"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.3.2"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.6"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.31.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.2.0"
]
]
}
],
"lcname": "inspeqai"
}
JSON
