| Name | optimizer-rag JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | A document compression and optimization library using Groq LLMs and LangChain |
| upload_time | 2025-07-28 21:21:26 |
| maintainer | None |
| docs_url | None |
| author | Mihir Kapile |
| requires_python | >=3.8 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Optimizer
Optimizer is a Python library for compressing large documents using LLMs like Groq. It provides functionality for summarizing document chunks based on a query, selecting the most relevant summaries, and managing token budgets effectively — ideal for RAG (Retrieval Augmented Generation) applications.
## Features
- Summarize document chunks using Groq models
- Select relevant summaries based on token limits
- Easily integrate with any RAG pipeline
## Installation
```bash
pip install optimizer-groq
```
## Requirements
Make sure the following packages are installed (handled automatically if installing via pip):
```text
langchain>=0.1.16
groq
tiktoken
numpy
scikit-learn
sentence-transformers
python-dotenv
```
## Usage
```python
from optimizer.compressor import compress_chunk
chunks = ["Paragraph 1...", "Paragraph 2...", "Paragraph 3..."]
query = "What are the benefits of renewable energy?"
token_budget = 512
selected = compress_chunk(chunks, query, token_budget)
print(selected)
```
You can also customize the model by injecting the model into the library functions like this.
```python
from openai import OpenAI
from optimizer.compressor import compress_chunk
client = OpenAI(api_key="your-openai-api-key")
chunks = ["Text A...", "Text B..."]
query = "Summarize risks."
token_budget = 400
summaries = compress_chunk(chunks, query, token_budget, client=client, model="gpt-4")
```
## Environment Setup
Set your Groq API key in a `.env` file:
```env
GROQ_API_KEY=your_groq_api_key_here
```
Raw data
{
"_id": null,
"home_page": null,
"name": "optimizer-rag",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Mihir Kapile",
"author_email": "mihirkapile@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/31/ff/b7cf62dbf17e63ae8cf323c6b7bb7ba290cfd8d1b822a2e40918fcf0deb7/optimizer_rag-0.1.1.tar.gz",
"platform": null,
"description": "# Optimizer\r\n\r\nOptimizer is a Python library for compressing large documents using LLMs like Groq. It provides functionality for summarizing document chunks based on a query, selecting the most relevant summaries, and managing token budgets effectively \u00e2\u20ac\u201d ideal for RAG (Retrieval Augmented Generation) applications.\r\n\r\n## Features\r\n\r\n- Summarize document chunks using Groq models\r\n- Select relevant summaries based on token limits\r\n- Easily integrate with any RAG pipeline\r\n\r\n## Installation\r\n\r\n```bash\r\npip install optimizer-groq\r\n```\r\n\r\n## Requirements\r\n\r\nMake sure the following packages are installed (handled automatically if installing via pip):\r\n\r\n```text\r\nlangchain>=0.1.16\r\ngroq\r\ntiktoken\r\nnumpy\r\nscikit-learn\r\nsentence-transformers\r\npython-dotenv\r\n```\r\n\r\n## Usage\r\n\r\n```python\r\nfrom optimizer.compressor import compress_chunk\r\n\r\nchunks = [\"Paragraph 1...\", \"Paragraph 2...\", \"Paragraph 3...\"]\r\nquery = \"What are the benefits of renewable energy?\"\r\ntoken_budget = 512\r\n\r\nselected = compress_chunk(chunks, query, token_budget)\r\nprint(selected)\r\n```\r\n\r\nYou can also customize the model by injecting the model into the library functions like this.\r\n\r\n```python\r\nfrom openai import OpenAI\r\nfrom optimizer.compressor import compress_chunk\r\n\r\nclient = OpenAI(api_key=\"your-openai-api-key\")\r\n\r\nchunks = [\"Text A...\", \"Text B...\"]\r\nquery = \"Summarize risks.\"\r\ntoken_budget = 400\r\n\r\nsummaries = compress_chunk(chunks, query, token_budget, client=client, model=\"gpt-4\")\r\n```\r\n\r\n\r\n## Environment Setup\r\n\r\nSet your Groq API key in a `.env` file:\r\n\r\n```env\r\nGROQ_API_KEY=your_groq_api_key_here\r\n```\r\n",
"bugtrack_url": null,
"license": null,
"summary": "A document compression and optimization library using Groq LLMs and LangChain",
"version": "0.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "00d1584442dfc38374c60464b70b5076ee20a697cb91c5e4e2939c4794993ce7",
"md5": "d6ed66fdc9a60998bd014773b205b35d",
"sha256": "7cc21eb804ee505ede9e5a59e8f06513373d269ea2951c43c025cba0276614b2"
},
"downloads": -1,
"filename": "optimizer_rag-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d6ed66fdc9a60998bd014773b205b35d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 5429,
"upload_time": "2025-07-28T21:21:25",
"upload_time_iso_8601": "2025-07-28T21:21:25.501598Z",
"url": "https://files.pythonhosted.org/packages/00/d1/584442dfc38374c60464b70b5076ee20a697cb91c5e4e2939c4794993ce7/optimizer_rag-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "31ffb7cf62dbf17e63ae8cf323c6b7bb7ba290cfd8d1b822a2e40918fcf0deb7",
"md5": "68296f87c49df9ac6608f0e1b1cc62c6",
"sha256": "c8dd068e762c69abe3a95aef4d7bf6bc625ed147d2e78f86de4d931f0e3abab5"
},
"downloads": -1,
"filename": "optimizer_rag-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "68296f87c49df9ac6608f0e1b1cc62c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 4216,
"upload_time": "2025-07-28T21:21:26",
"upload_time_iso_8601": "2025-07-28T21:21:26.609994Z",
"url": "https://files.pythonhosted.org/packages/31/ff/b7cf62dbf17e63ae8cf323c6b7bb7ba290cfd8d1b822a2e40918fcf0deb7/optimizer_rag-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-28 21:21:26",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "optimizer-rag"
}
