| Name | llama-index-postprocessor-nvidia-rerank JSON |
| Version |
0.3.3
JSON |
| download |
| home_page | None |
| Summary | llama-index postprocessor nvidia_rerank integration |
| upload_time | 2024-10-25 15:58:45 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.8.1 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# NVIDIA NIMs
The `llama-index-postprocessor-nvidia-rerank` package contains LlamaIndex integrations building applications with models on
NVIDIA NIM inference microservice. NIM supports models across domains like chat, embedding, and re-ranking models
from the community as well as NVIDIA. These models are optimized by NVIDIA to deliver the best performance on NVIDIA
accelerated infrastructure and deployed as a NIM, an easy-to-use, prebuilt containers that deploy anywhere using a single
command on NVIDIA accelerated infrastructure.
NVIDIA hosted deployments of NIMs are available to test on the [NVIDIA API catalog](https://build.nvidia.com/). After testing,
NIMs can be exported from NVIDIA’s API catalog using the NVIDIA AI Enterprise license and run on-premises or in the cloud,
giving enterprises ownership and full control of their IP and AI application.
NIMs are packaged as container images on a per model basis and are distributed as NGC container images through the NVIDIA NGC Catalog.
At their core, NIMs provide easy, consistent, and familiar APIs for running inference on an AI model.
# LlamaIndex Postprocessor Integration: Nvidia_Rerank
Below is an example on how to use some common functionality surrounding text-generative and embedding models
## Installation
```shell
pip install --upgrade llama-index llama-index-core llama-index-postprocessor-nvidia-rerank
```
## Setup
**To get started:**
1. Create a free account with [NVIDIA](https://build.nvidia.com/), which hosts NVIDIA AI Foundation models.
2. Select the `Retrieval` tab, then select your model of choice.
3. Under `Input` select the `Python` tab, and click `Get API Key`. Then click `Generate Key`.
4. Copy and save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.
```python
import getpass
import os
if os.environ.get("NVIDIA_API_KEY", "").startswith("nvapi-"):
print("Valid NVIDIA_API_KEY already in environment. Delete to reset")
else:
nvapi_key = getpass.getpass("NVAPI Key (starts with nvapi-): ")
assert nvapi_key.startswith(
"nvapi-"
), f"{nvapi_key[:5]}... is not a valid key"
os.environ["NVIDIA_API_KEY"] = nvapi_key
```
## Working with NVIDIA API Catalog
```python
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
rerank = NVIDIARerank()
```
## Working with NVIDIA NIMs
```python
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
# connect to an reranking NIM running at localhost:1976
rerank = NVIDIARerank(base_url="http://localhost:1976/v1")
```
## Supported models
Querying `available_models` will still give you all of the other models offered by your API credentials.
```python
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
rerank.available_models
```
**To find out more about a specific model, please navigate to the NVIDIA NIM section of ai.nvidia.com [as linked here](https://docs.api.nvidia.com/nim/).**
## Reranking
Below is an example:
```python
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
from llama_index.core import Document
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.node_parser import SentenceSplitter, SimpleFileNodeParser
# load documents
documents = SimpleDirectoryReader("/path_to_your_data_folder").load_data()
# use API Catalog's reranker model
my_key = os.environ["NVIDIA_API_KEY"]
rerank = NVIDIARerank()
# parse nodes
parser = SentenceSplitter(separator="\n", chunk_size=200, chunk_overlap=0)
nodes = parser.get_nodes_from_documents(documents)
# rerank
rerank.postprocess_nodes(nodes, query_str=query)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-postprocessor-nvidia-rerank",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/06/ee/10e0e8d7b2791839f65649f674628f61a6bc8ca44294e07744d923fb6fca/llama_index_postprocessor_nvidia_rerank-0.3.3.tar.gz",
"platform": null,
"description": "# NVIDIA NIMs\n\nThe `llama-index-postprocessor-nvidia-rerank` package contains LlamaIndex integrations building applications with models on\nNVIDIA NIM inference microservice. NIM supports models across domains like chat, embedding, and re-ranking models\nfrom the community as well as NVIDIA. These models are optimized by NVIDIA to deliver the best performance on NVIDIA\naccelerated infrastructure and deployed as a NIM, an easy-to-use, prebuilt containers that deploy anywhere using a single\ncommand on NVIDIA accelerated infrastructure.\n\nNVIDIA hosted deployments of NIMs are available to test on the [NVIDIA API catalog](https://build.nvidia.com/). After testing,\nNIMs can be exported from NVIDIA\u2019s API catalog using the NVIDIA AI Enterprise license and run on-premises or in the cloud,\ngiving enterprises ownership and full control of their IP and AI application.\n\nNIMs are packaged as container images on a per model basis and are distributed as NGC container images through the NVIDIA NGC Catalog.\nAt their core, NIMs provide easy, consistent, and familiar APIs for running inference on an AI model.\n\n# LlamaIndex Postprocessor Integration: Nvidia_Rerank\n\nBelow is an example on how to use some common functionality surrounding text-generative and embedding models\n\n## Installation\n\n```shell\npip install --upgrade llama-index llama-index-core llama-index-postprocessor-nvidia-rerank\n```\n\n## Setup\n\n**To get started:**\n\n1. Create a free account with [NVIDIA](https://build.nvidia.com/), which hosts NVIDIA AI Foundation models.\n\n2. Select the `Retrieval` tab, then select your model of choice.\n\n3. Under `Input` select the `Python` tab, and click `Get API Key`. Then click `Generate Key`.\n\n4. Copy and save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.\n\n```python\nimport getpass\nimport os\n\nif os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n print(\"Valid NVIDIA_API_KEY already in environment. Delete to reset\")\nelse:\n nvapi_key = getpass.getpass(\"NVAPI Key (starts with nvapi-): \")\n assert nvapi_key.startswith(\n \"nvapi-\"\n ), f\"{nvapi_key[:5]}... is not a valid key\"\n os.environ[\"NVIDIA_API_KEY\"] = nvapi_key\n```\n\n## Working with NVIDIA API Catalog\n\n```python\nfrom llama_index.postprocessor.nvidia_rerank import NVIDIARerank\n\nrerank = NVIDIARerank()\n```\n\n## Working with NVIDIA NIMs\n\n```python\nfrom llama_index.postprocessor.nvidia_rerank import NVIDIARerank\n\n# connect to an reranking NIM running at localhost:1976\nrerank = NVIDIARerank(base_url=\"http://localhost:1976/v1\")\n```\n\n## Supported models\n\nQuerying `available_models` will still give you all of the other models offered by your API credentials.\n\n```python\nfrom llama_index.postprocessor.nvidia_rerank import NVIDIARerank\n\nrerank.available_models\n```\n\n**To find out more about a specific model, please navigate to the NVIDIA NIM section of ai.nvidia.com [as linked here](https://docs.api.nvidia.com/nim/).**\n\n## Reranking\n\nBelow is an example:\n\n```python\nfrom llama_index.postprocessor.nvidia_rerank import NVIDIARerank\n\nfrom llama_index.core import Document\nfrom llama_index.core import VectorStoreIndex, SimpleDirectoryReader\nfrom llama_index.core.node_parser import SentenceSplitter, SimpleFileNodeParser\n\n\n# load documents\ndocuments = SimpleDirectoryReader(\"/path_to_your_data_folder\").load_data()\n\n# use API Catalog's reranker model\nmy_key = os.environ[\"NVIDIA_API_KEY\"]\nrerank = NVIDIARerank()\n\n# parse nodes\nparser = SentenceSplitter(separator=\"\\n\", chunk_size=200, chunk_overlap=0)\nnodes = parser.get_nodes_from_documents(documents)\n# rerank\nrerank.postprocess_nodes(nodes, query_str=query)\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index postprocessor nvidia_rerank integration",
"version": "0.3.3",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "79d702bcd6c96eb299558519e7a90cf8b3734b6e54d6e06ea93ec5f9990d671d",
"md5": "b4ab913dfb47685aa1cb8a1ab53d8b12",
"sha256": "f0eb1b53884990da5866c9b0fe6fb00f2769297ba151b79dacf2895483fd5890"
},
"downloads": -1,
"filename": "llama_index_postprocessor_nvidia_rerank-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b4ab913dfb47685aa1cb8a1ab53d8b12",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 7137,
"upload_time": "2024-10-25T15:58:43",
"upload_time_iso_8601": "2024-10-25T15:58:43.508491Z",
"url": "https://files.pythonhosted.org/packages/79/d7/02bcd6c96eb299558519e7a90cf8b3734b6e54d6e06ea93ec5f9990d671d/llama_index_postprocessor_nvidia_rerank-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "06ee10e0e8d7b2791839f65649f674628f61a6bc8ca44294e07744d923fb6fca",
"md5": "cf031505a13d44729cc0e0d62e4208e5",
"sha256": "62a2fa1e2f889ba9484ce0f778a9c26c00b81e113886b9b4188d7c3f0b3ae4af"
},
"downloads": -1,
"filename": "llama_index_postprocessor_nvidia_rerank-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "cf031505a13d44729cc0e0d62e4208e5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 6672,
"upload_time": "2024-10-25T15:58:45",
"upload_time_iso_8601": "2024-10-25T15:58:45.607485Z",
"url": "https://files.pythonhosted.org/packages/06/ee/10e0e8d7b2791839f65649f674628f61a6bc8ca44294e07744d923fb6fca/llama_index_postprocessor_nvidia_rerank-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-25 15:58:45",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-postprocessor-nvidia-rerank"
}
