<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_dark.png">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_light.png">
<img alt="Hashnode logo" src="https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_light.png" height="130">
</picture>
</p>
<p align="center">
<a href="">
<img alt="GitHub" src="https://img.shields.io/github/license/Auto-Playground/ragrank">
</a>
<a href="https://pypi.org/project/ragrank/">
<img alt="Build" src="https://img.shields.io/pypi/pyversions/ragrank">
</a>
<a href="https://ragrank.readthedocs.io/latest/">
<img alt="GitHub" src="https://img.shields.io/readthedocs/ragrank">
</a>
<a href="https://pypi.org/project/ragrank/">
<img alt="GitHub" src="https://img.shields.io/github/v/release/Auto-Playground/Ragrank?color=orange">
</a>
<a href="https://github.com/Auto-Playground/Ragrank/actions">
<img alt="GitHub" src="https://img.shields.io/github/actions/workflow/status/Auto-Playground/ragrank/.github%2Fworkflows%2Ftests.yml">
</a>
</p>
<h4 align="center">
<p>
<a href="https://ragrank.readthedocs.io/latest/">Documentation</a> |
<a href="https://api-ragrank.readthedocs.io/">API reference</a> |
<a href="https://ragrank.readthedocs.io/latest/get_started/basic_evaluation.html">Quickstart</a> |
<a href="https://discord.gg/KzfVpds3">Join the Community</a>
<p>
</h4>
Welcome to Ragrank! This toolkit is designed to assist you in evaluating the performance of your Retrieval-Augmented Generation (RAG) applications. You will get proper metrics for evaluate RAG model. The product is still in `beta` stage.
## 🔥 Installation
Ragrank is available as a PyPi package. To install it, simply run:
```bash
pip install ragrank
```
If you prefer to install it from the source:
```bash
git clone https://github.com/Auto-Playground/ragrank.git && cd ragrank
poetry install
```
## 🚀 Quick Start
Set your `OPENAI_API_KEY` as an environment variable (you can also evaluate using your own custom model, refer [docs](https://ragrank.readthedocs.io/)):
```bash
export OPENAI_API_KEY="..."
```
Here's a quick example of how you can use Ragrank to evaluate the relevance of generated responses:
```python
from ragrank import evaluate
from ragrank.dataset import from_dict
from ragrank.metric import response_relevancy
# Define your dataset
data = from_dict({
"question": "What is the capital of France?",
"context": ["France is famous for its iconic landmarks such as the Eiffel Tower and its rich culinary tradition."],
"response": "The capital of France is Paris.",
})
# Evaluate the response relevance metric
result = evaluate(data, metrics=[response_relevancy])
# Display the evaluation results
result.to_dataframe()
```
For more information on how to use Ragrank and its various features, please refer to the [documentation](https://ragrank.readthedocs.io/). 📚
## License
This project is licensed under the [Apache License](https://github.com/Auto-Playground/Ragrank/blob/main/LICENSE). Feel free to use and modify it according to your needs.
## Feedback and Support
If you encounter any issues, have questions, or would like to provide feedback, please don't hesitate to open an issue on the GitHub repository. Your contributions and suggestions are highly appreciated!
Join our community on Discord to connect with other users, ask questions, and share your experiences with Ragrank. We're here to help you make the most out of your NLP projects! 💬
> Happy evaluating! 🙂
Raw data
{
"_id": null,
"home_page": "https://github.com/Auto-Playground/ragrank",
"name": "ragrank",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "rag, evaluation, llmops",
"author": "Izam Mohammed",
"author_email": "izamdeveloper1@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0a/64/e67d61dd4d84fb85468b35334048f89dc014ce32e6656137a94f3aaddb4b/ragrank-0.0.7.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <picture>\n <source media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_dark.png\">\n <source media=\"(prefers-color-scheme: light)\" srcset=\"https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_light.png\">\n <img alt=\"Hashnode logo\" src=\"https://raw.githubusercontent.com/Auto-Playground/Ragrank/main/docs/docs/_static/imgs/ragrank_light.png\" height=\"130\">\n </picture>\n\n</p>\n\n<p align=\"center\">\n <a href=\"\">\n <img alt=\"GitHub\" src=\"https://img.shields.io/github/license/Auto-Playground/ragrank\">\n </a>\n <a href=\"https://pypi.org/project/ragrank/\">\n <img alt=\"Build\" src=\"https://img.shields.io/pypi/pyversions/ragrank\">\n </a>\n <a href=\"https://ragrank.readthedocs.io/latest/\">\n <img alt=\"GitHub\" src=\"https://img.shields.io/readthedocs/ragrank\">\n </a>\n <a href=\"https://pypi.org/project/ragrank/\">\n <img alt=\"GitHub\" src=\"https://img.shields.io/github/v/release/Auto-Playground/Ragrank?color=orange\">\n </a>\n <a href=\"https://github.com/Auto-Playground/Ragrank/actions\">\n <img alt=\"GitHub\" src=\"https://img.shields.io/github/actions/workflow/status/Auto-Playground/ragrank/.github%2Fworkflows%2Ftests.yml\">\n </a>\n</p>\n\n<h4 align=\"center\">\n <p>\n <a href=\"https://ragrank.readthedocs.io/latest/\">Documentation</a> |\n <a href=\"https://api-ragrank.readthedocs.io/\">API reference</a> |\n <a href=\"https://ragrank.readthedocs.io/latest/get_started/basic_evaluation.html\">Quickstart</a> |\n <a href=\"https://discord.gg/KzfVpds3\">Join the Community</a>\n <p>\n</h4>\n\nWelcome to Ragrank! This toolkit is designed to assist you in evaluating the performance of your Retrieval-Augmented Generation (RAG) applications. You will get proper metrics for evaluate RAG model. The product is still in `beta` stage.\n\n## \ud83d\udd25 Installation\n\nRagrank is available as a PyPi package. To install it, simply run:\n\n```bash\npip install ragrank\n```\n\nIf you prefer to install it from the source:\n\n```bash\ngit clone https://github.com/Auto-Playground/ragrank.git && cd ragrank\npoetry install\n```\n\n## \ud83d\ude80 Quick Start\n\nSet your `OPENAI_API_KEY` as an environment variable (you can also evaluate using your own custom model, refer [docs](https://ragrank.readthedocs.io/)):\n```bash\nexport OPENAI_API_KEY=\"...\"\n```\n\nHere's a quick example of how you can use Ragrank to evaluate the relevance of generated responses:\n\n```python\nfrom ragrank import evaluate\nfrom ragrank.dataset import from_dict\nfrom ragrank.metric import response_relevancy\n\n# Define your dataset\ndata = from_dict({\n \"question\": \"What is the capital of France?\",\n \"context\": [\"France is famous for its iconic landmarks such as the Eiffel Tower and its rich culinary tradition.\"],\n \"response\": \"The capital of France is Paris.\",\n})\n\n# Evaluate the response relevance metric\nresult = evaluate(data, metrics=[response_relevancy])\n\n# Display the evaluation results\nresult.to_dataframe()\n```\n\nFor more information on how to use Ragrank and its various features, please refer to the [documentation](https://ragrank.readthedocs.io/). \ud83d\udcda\n\n## License\n\nThis project is licensed under the [Apache License](https://github.com/Auto-Playground/Ragrank/blob/main/LICENSE). Feel free to use and modify it according to your needs.\n\n## Feedback and Support\n\nIf you encounter any issues, have questions, or would like to provide feedback, please don't hesitate to open an issue on the GitHub repository. Your contributions and suggestions are highly appreciated!\n\nJoin our community on Discord to connect with other users, ask questions, and share your experiences with Ragrank. We're here to help you make the most out of your NLP projects! \ud83d\udcac\n\n> Happy evaluating! \ud83d\ude42",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "An evaluation library for RAG models",
"version": "0.0.7",
"project_urls": {
"Homepage": "https://github.com/Auto-Playground/ragrank",
"Repository": "https://github.com/Auto-Playground/ragrank"
},
"split_keywords": [
"rag",
" evaluation",
" llmops"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f21d3383abce2da75dac6b4ecd63114cbcfd9f95520fd2fe46b7d5a4ec5d4e86",
"md5": "cc2bf6a8656d13c3499adec2ae3e2f42",
"sha256": "f3650ac0db5056ecc56d3dbb1baa98a17ee81228e8822b12d1b99e953154a35d"
},
"downloads": -1,
"filename": "ragrank-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc2bf6a8656d13c3499adec2ae3e2f42",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 38898,
"upload_time": "2024-05-05T14:07:19",
"upload_time_iso_8601": "2024-05-05T14:07:19.290543Z",
"url": "https://files.pythonhosted.org/packages/f2/1d/3383abce2da75dac6b4ecd63114cbcfd9f95520fd2fe46b7d5a4ec5d4e86/ragrank-0.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0a64e67d61dd4d84fb85468b35334048f89dc014ce32e6656137a94f3aaddb4b",
"md5": "3ec7a90387486e9a6f0948ca19d23136",
"sha256": "6e4738fd17869e4256fdd22ada8a162e6b8a0783678940af4e04b806f3c79a82"
},
"downloads": -1,
"filename": "ragrank-0.0.7.tar.gz",
"has_sig": false,
"md5_digest": "3ec7a90387486e9a6f0948ca19d23136",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 25274,
"upload_time": "2024-05-05T14:07:20",
"upload_time_iso_8601": "2024-05-05T14:07:20.929382Z",
"url": "https://files.pythonhosted.org/packages/0a/64/e67d61dd4d84fb85468b35334048f89dc014ce32e6656137a94f3aaddb4b/ragrank-0.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-05 14:07:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Auto-Playground",
"github_project": "ragrank",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ragrank"
}
JSON
