| Name | llama-index-vector-stores-nile JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | llama-index vector_stores nile integration |
| upload_time | 2024-10-14 01:54:11 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.8.1 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Nile Vector Store (PostgreSQL)
This integration makes it possible to use [Nile - Postgres re-engineered for multi-tenant applications](https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/)
as a vector store in LlamaIndex.
## What is Nile?
Nile is a Postgres database that enables all database operations per tenant including auto-scaling, branching, and backups, with full customer isolation.
Multi-tenant RAG applications are increasingly popular, since they provide security and privacy while using large language models.
However, managing the underlying Postgres database is not straightforward. DB-per-tenant is expensive and complex to manage, while shared-DB has security and privacy concerns, and also limits the scalability and performance of the RAG application. Nile re-engineered Postgres to deliver the best of all worlds - the isolation of DB-per-tenant, at the cost, efficiency and developer experience of a shared-DB.
Storing millions of vectors in a shared-DB can be slow and require significant resources to index and query. But if you store 1000 tenants in Nile's virtual tenant databases, each with 1000 vectors, this can be quite manageable. Especially since you can place larger tenants on their own compute, while smaller tenants can efficiently share compute resources and auto-scale as needed.
## Getting Started with Nile
Start by signing up for [Nile](https://console.thenile.dev/?utm_campaign=partnerlaunch&utm_source=llamaindex&utm_medium=docs). Once you've signed up for Nile, you'll be promoted to create your first database. Go ahead and do so. You'll be redirected to the "Query Editor" page of your new database.
From there, click on "Home" (top icon on the left menu), click on "generate credentials" and copy the resulting connection string. You will need it in a sec.
## Quickstart
Install the integration with:
```bash
pip install llama-index-vector-stores-nile
```
Use the connection string you generated earlier (at the "Getting started" step) to create a tenant-aware vector store.
:fire: NileVectorStore supports both tenant-aware vector stores, that isolates the documents for each tenant and a regular store which is typically used for shared data that all tenants can access. Below, we'll demonstrate the tenant-aware vector store.
```python
# Replace with your connection string.
NILE_SERVICE_URL = "postgresql://nile:password@db.thenile.dev:5432/nile"
vector_store = NileVectorStore(
service_url=NILEDB_SERVICE_URL,
table_name="documents",
tenant_aware=True,
num_dimensions=1536,
)
```
Create an index from documents:
```python
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
documents_nexiv + documents_modamart,
storage_context=storage_context,
show_progress=True,
)
```
Or from existing embeddings:
```python
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)
```
and query each tenant's data with guaranteed isolation:
```python
query_engine = index.as_query_engine(
vector_store_kwargs={
"tenant_id": str(tenant_id_modamart),
},
)
response = query_engine.query("What action items do we need to follow up on?")
print(response)
```
See resources below for more information and examples.
## Additional Resources
- [Example iPython / Jupyter notebook for Nile and LlamaIndex](https://docs.llamaindex.ai/en/stable/examples/vector_stores/NileVectorStore/)
- [Nile's generative AI and vector embeddings docs](https://www.thenile.dev/docs/ai-embeddings)
- [Nile's LlamaIndex documentation](https://www.thenile.dev/docs/partners/llama)
- [Nile's pgvector primer](https://www.thenile.dev/docs/ai-embeddings/pg_vector)
- [Few things you didn't know about pgvector](https://www.thenile.dev/blog/pgvector_myth_debunking)
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-vector-stores-nile",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/d9/46/c097212b329b3fdc095c334cc4389f0d5c8a5bb3c2f53a352faa147c3dd3/llama_index_vector_stores_nile-0.1.1.tar.gz",
"platform": null,
"description": "# Nile Vector Store (PostgreSQL)\n\nThis integration makes it possible to use [Nile - Postgres re-engineered for multi-tenant applications](https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/)\nas a vector store in LlamaIndex.\n\n## What is Nile?\n\nNile is a Postgres database that enables all database operations per tenant including auto-scaling, branching, and backups, with full customer isolation.\n\nMulti-tenant RAG applications are increasingly popular, since they provide security and privacy while using large language models.\n\nHowever, managing the underlying Postgres database is not straightforward. DB-per-tenant is expensive and complex to manage, while shared-DB has security and privacy concerns, and also limits the scalability and performance of the RAG application. Nile re-engineered Postgres to deliver the best of all worlds - the isolation of DB-per-tenant, at the cost, efficiency and developer experience of a shared-DB.\n\nStoring millions of vectors in a shared-DB can be slow and require significant resources to index and query. But if you store 1000 tenants in Nile's virtual tenant databases, each with 1000 vectors, this can be quite manageable. Especially since you can place larger tenants on their own compute, while smaller tenants can efficiently share compute resources and auto-scale as needed.\n\n## Getting Started with Nile\n\nStart by signing up for [Nile](https://console.thenile.dev/?utm_campaign=partnerlaunch&utm_source=llamaindex&utm_medium=docs). Once you've signed up for Nile, you'll be promoted to create your first database. Go ahead and do so. You'll be redirected to the \"Query Editor\" page of your new database.\n\nFrom there, click on \"Home\" (top icon on the left menu), click on \"generate credentials\" and copy the resulting connection string. You will need it in a sec.\n\n## Quickstart\n\nInstall the integration with:\n\n```bash\npip install llama-index-vector-stores-nile\n```\n\nUse the connection string you generated earlier (at the \"Getting started\" step) to create a tenant-aware vector store.\n\n:fire: NileVectorStore supports both tenant-aware vector stores, that isolates the documents for each tenant and a regular store which is typically used for shared data that all tenants can access. Below, we'll demonstrate the tenant-aware vector store.\n\n```python\n# Replace with your connection string.\nNILE_SERVICE_URL = \"postgresql://nile:password@db.thenile.dev:5432/nile\"\n\nvector_store = NileVectorStore(\n service_url=NILEDB_SERVICE_URL,\n table_name=\"documents\",\n tenant_aware=True,\n num_dimensions=1536,\n)\n```\n\nCreate an index from documents:\n\n```python\nstorage_context = StorageContext.from_defaults(vector_store=vector_store)\nindex = VectorStoreIndex.from_documents(\n documents_nexiv + documents_modamart,\n storage_context=storage_context,\n show_progress=True,\n)\n```\n\nOr from existing embeddings:\n\n```python\nindex = VectorStoreIndex.from_vector_store(vector_store=vector_store)\n```\n\nand query each tenant's data with guaranteed isolation:\n\n```python\nquery_engine = index.as_query_engine(\n vector_store_kwargs={\n \"tenant_id\": str(tenant_id_modamart),\n },\n)\nresponse = query_engine.query(\"What action items do we need to follow up on?\")\n\nprint(response)\n```\n\nSee resources below for more information and examples.\n\n## Additional Resources\n\n- [Example iPython / Jupyter notebook for Nile and LlamaIndex](https://docs.llamaindex.ai/en/stable/examples/vector_stores/NileVectorStore/)\n- [Nile's generative AI and vector embeddings docs](https://www.thenile.dev/docs/ai-embeddings)\n- [Nile's LlamaIndex documentation](https://www.thenile.dev/docs/partners/llama)\n- [Nile's pgvector primer](https://www.thenile.dev/docs/ai-embeddings/pg_vector)\n- [Few things you didn't know about pgvector](https://www.thenile.dev/blog/pgvector_myth_debunking)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index vector_stores nile integration",
"version": "0.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8d036835f95341f9605ac39b0f7853711eea38ea1d4f1c9fad7feb256f651262",
"md5": "42dd25038b3fc80dba4129ab1827a5d3",
"sha256": "eaedb654c090b35b7936ded48a2933a3a0b5d97b77db4c2080f2cbf797908e63"
},
"downloads": -1,
"filename": "llama_index_vector_stores_nile-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "42dd25038b3fc80dba4129ab1827a5d3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 7724,
"upload_time": "2024-10-14T01:54:10",
"upload_time_iso_8601": "2024-10-14T01:54:10.181573Z",
"url": "https://files.pythonhosted.org/packages/8d/03/6835f95341f9605ac39b0f7853711eea38ea1d4f1c9fad7feb256f651262/llama_index_vector_stores_nile-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d946c097212b329b3fdc095c334cc4389f0d5c8a5bb3c2f53a352faa147c3dd3",
"md5": "137b5595b2c2e02a86d2878fb9d51463",
"sha256": "98e558812206a9e40304ee332ca1a02944921b0870711d3e3e25009d50c74d65"
},
"downloads": -1,
"filename": "llama_index_vector_stores_nile-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "137b5595b2c2e02a86d2878fb9d51463",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 8718,
"upload_time": "2024-10-14T01:54:11",
"upload_time_iso_8601": "2024-10-14T01:54:11.871472Z",
"url": "https://files.pythonhosted.org/packages/d9/46/c097212b329b3fdc095c334cc4389f0d5c8a5bb3c2f53a352faa147c3dd3/llama_index_vector_stores_nile-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-14 01:54:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-vector-stores-nile"
}
