# VecLite
**Local‑first SQLite + vectors for agentic RAG — zero infra.**
VecLite isn't a traditional ORM adapted for search — it's **purpose‑built for retrieval**. It combines relational filters, BM25, vectors, hybrid fusion, and optional reranking in one lightweight library that runs entirely on your machine.
**Why VecLite exists:**
- ⚡ **Local‑first, zero infra** — SQLite + vectors; no services to run
- 🧰 **No plumbing** — embeddings + indices managed automatically
- 🧩 **Relational + views** — enrich chunks without denormalization
- 🔎 **Three search modes** — BM25, vector, hybrid (+ optional rerank modifier)
- ✅ **Atomic embeddings** — batch once; no rows without vectors on failure
- 🔗 **Simple filters** — eq/in_/between/JSON contains/ilike/regex
- 📦 **One folder** — your entire RAG + DB stack (sqlite.db + vectors/)
- 🗂️ **No metadata limits** — normal SQL columns/JSON, any shape
- 🧱 **Vector + non‑vector tables** — mix FTS‑only and regular tables alongside vectors
---
## Comparison
| Capability | VecLite (SQLite + Vectors) | Typical Managed Vector DBs |
| --- | --- | --- |
| Setup | Local‑first, zero infra | Hosted service / cluster |
| Data model | Relational tables + views + JSON | Vector‑centric, document/NoSQL |
| Non‑vector queries | Yes (filters, ranges, regex, FTS) | Limited (metadata filters only) |
| Joins / views | Yes (via SQL views) | No |
| Keyword search | Yes (BM25 via FTS5) | Rare / not native |
| Vector search | Yes (cosine) | Yes |
| Hybrid search | Yes (vector + BM25) | Varies; often custom |
| Rerank | Optional modifier | External / varies |
| CRUD | Full insert/update/upsert/delete | Insert/update; deletes vary |
| Metadata limits | No (normal SQL columns/JSON) | Often constrained (field count/size) |
| Record size limits | No hard per‑row payload limit | Common (~40KB metadata payloads) |
| Non‑vector tables | Yes (regular/FTS‑only tables) | No (vector‑centric indexes) |
| Consistency | Atomic batch embeddings | Varies by service |
| Storage | One folder: sqlite.db + vectors/ | Remote indexes/storage |
| Best for | Local RAG, agents, notebooks | Production scale, multi‑tenant APIs |
VecLite is not a production, multi‑tenant vector service — it’s a compact, relational RAG database you can fully own and query normally.
Note: Many managed vector services impose strict metadata payload limits (e.g., ~40KB per record) and center the data model around vector indexes; VecLite lets you store any shape/size of metadata and create tables without vectors at all.
> **Use VecLite for:** Local‑first RAG, agentic retrieval, Jupyter notebooks, desktop/edge apps
> **Not for:** High‑traffic multi‑tenant servers; drop‑in vector store adapters (integrate via custom retrievers instead)
---
## Quick Start
Build a semantic search system in 5 lines:
```bash
pip install veclite[voyage] # Includes Voyage AI embeddings
export VOYAGE_API_KEY="your-key" # Get from https://www.voyageai.com
```
```python
from veclite import Client, Schema
from veclite.schema import Table, Integer, Text
# 1. Define schema - embeddings happen automatically
class Document(Table):
__tablename__ = "documents"
id = Integer(primary_key=True)
content = Text(vector=True, fts=True) # vector + keyword search
# 2. Create database (nested folder)
schema = Schema()
schema.add_table(Document)
client = Client.create(schema, "rag_db") # creates ./rag_db/{sqlite.db, vectors/}
# 3. Insert - embeddings generated automatically
client.table("documents").insert([
{"content": "Python is a programming language"},
{"content": "Machine learning uses neural networks"},
{"content": "The Solar System has 8 planets"},
]).execute()
# 4. Search by meaning (finds ML doc, not Python)
results = client.table("documents").vector_search(
query="AI and deep learning",
topk=5
).execute()
```
**That's it.** No embedding pipelines, no vector databases, no infrastructure.
---
## Atomic Batch Embeddings (Consistency)
Ensure all‑or‑nothing inserts with batched embeddings:
```python
# Async example
async with async_client.batch_embeddings():
await async_client.table("documents").insert([...]).execute()
await async_client.table("documents").insert([...]).execute()
# If any embedding fails → rollback SQLite; no vectors written
```
- Default is atomic: one SQLite transaction; embeddings generated; vectors written; then COMMIT.
- Non‑atomic option: `async with db.batch_embeddings(atomic=False): ...` batches for efficiency and writes failures to an outbox for later retry via `flush_vector_outbox()`.
---
## Search Modes (+ optional rerank modifier)
VecLite provides the complete retrieval stack - perfect for agentic RAG systems that need different search strategies:
### 1. 🔍 **Keyword Search (BM25)**
Traditional full-text search with production-grade BM25 ranking:
```python
results = client.table("docs").keyword_search(
query="machine learning transformers",
topk=10
).execute()
```
**Use when:** Exact term matching matters (product codes, names, technical terms)
### 2. 🎯 **Vector Search (Semantic)**
Find by meaning, not just keywords:
```python
results = client.table("docs").vector_search(
query="AI tutorials for beginners", # Matches "ML guides for newcomers"
topk=10
).execute()
```
**Use when:** Semantic similarity matters more than exact terms
### 3. 🚀 **Hybrid Search (Best of Both)**
Combines keyword + vector with Reciprocal Rank Fusion:
```python
results = client.table("docs").hybrid_search(
query="transformer architecture",
alpha=0.7, # 70% semantic, 30% keyword
topk=10
).execute()
```
**Use when:** You want both precision (keywords) and recall (semantics)
**Perfect for:** General-purpose RAG retrieval
### 🎖️ Rerank Modifier (optional)
Post-retrieval modifier to refine candidates:
```python
from veclite.embeddings import VoyageClient
# Get candidates with hybrid search
candidates = client.table("docs") \
.hybrid_search("quantum computing", topk=100) \
.execute()
# Rerank top 100 → best 10
embedder = VoyageClient()
reranked = embedder.rerank(
query="quantum computing applications",
documents=[doc["content"] for doc in candidates.data],
top_k=10
)
```
**Use when:** Quality > speed (2-stage retrieval)
---
## Perfect for Agentic RAG
VecLite's modular design makes it ideal for agentic systems where the AI chooses retrieval strategies:
```python
class RAGAgent:
def retrieve(self, query: str, strategy: str = "auto"):
if strategy == "auto":
# Agent decides based on query type
if self._is_technical_query(query):
return self.keyword_search(query)
else:
return self.hybrid_search(query)
elif strategy == "keyword":
return self.db.table("docs").keyword_search(query, topk=10)
elif strategy == "semantic":
return self.db.table("docs").vector_search(query, topk=10)
elif strategy == "hybrid":
return self.db.table("docs").hybrid_search(query, alpha=0.7, topk=10)
elif strategy == "deep":
# Two-stage: hybrid → rerank
candidates = self.db.table("docs").hybrid_search(query, topk=100)
return self.embedder.rerank(query, candidates.data, top_k=10)
```
**Agents can:**
- Choose search strategies dynamically
- Combine multiple retrieval modes
- Filter by metadata before/after search
- Iteratively refine with different strategies
---
## Recipe: SEC Filings (Relational + FTS + Vectors)
Keep filings in one DB, pages as FTS‑only, and chunks with vectors. Let an agent both retrieve semantically and read exact page ranges.
```python
from veclite import Client, Schema
from veclite.schema import Table, Integer, Text, Boolean
class Filings(Table):
__tablename__ = "filings"
id = Integer(primary_key=True)
ticker = Text(index=True)
form_type = Text(index=True)
filing_date = Text(index=True)
class FilingPages(Table):
__tablename__ = "filing_pages"
id = Integer(primary_key=True)
filing_id = Integer(index=True)
page_number = Integer(index=True)
content = Text(fts=True) # FTS only
class FilingChunks(Table):
__tablename__ = "filing_chunks"
id = Integer(primary_key=True)
filing_id = Integer(index=True)
page = Integer(index=True)
content = Text(vector=True, fts=True) # vectors + FTS
has_table = Boolean(default=False)
schema = Schema()
schema.add_table(Filings)
schema.add_table(FilingPages)
schema.add_table(FilingChunks)
client = Client.create(schema, "sec_db") # ./sec_db/{sqlite.db, vectors/}
# Hybrid retrieval on chunks within a filing
q = "Apple risk factors and competitive challenges"
hits = client.table("filing_chunks").hybrid_search(q, topk=10, alpha=0.7) \
.eq("filing_id", 12345).execute()
# Read a page window around best hit
best = hits.data[0]
page = best["page"]
filing_id = best["filing_id"]
pages = client.table("filing_pages").select("*") \
.eq("filing_id", filing_id) \
.between("page_number", page - 1, page + 1) \
.order("page_number") \
.execute()
```
This three‑table pattern (filings, filing_pages, filing_chunks) gives agents precision (page ranges) and recall (semantic + keyword).
## Automatic Embeddings
No manual embedding pipeline needed - VecLite handles everything:
```python
# Mark field for auto-embeddings
class Paper(Table):
__tablename__ = "papers"
id = Integer(primary_key=True)
title = Text()
abstract = Text(vector=True, fts=True) # Auto-embed on insert/update
year = Integer()
```
**What happens automatically:**
- ✅ Embeddings generated on insert/update
- ✅ Batching for efficiency
- ✅ LMDB caching (avoid re-embedding)
- ✅ Vector storage alongside SQLite
**Supported models:**
- `vector=True` → voyage-3.5-lite (512D, **default**)
- `vector=VectorConfig.voyage_3()` → voyage-3 (1024D)
- `vector=VectorConfig.voyage_large()` → voyage-3.5-large (1536D)
- `contextualized=True` → voyage-context-3 (contextualized retrieval, 512D default)
---
## Advanced Search Examples
### Filtered Search
```python
# Search within filtered subset
results = client.table("papers") \
.hybrid_search("climate impacts", alpha=0.6, topk=20) \
.eq("category", "science") \
.gt("year", 2020) \
.is_not_null("peer_reviewed") \
.execute()
```
### Multi-Field Search
```python
class Article(Table):
__tablename__ = "articles"
id = Integer(primary_key=True)
title = Text(vector=True, fts=True)
body = Text(vector=True, fts=True)
# Search specific field
results = client.table("articles").vector_search(
query="AI safety",
column="title", # Search titles only
topk=10
).execute()
```
### Contextualized Embeddings (Advanced RAG)
```python
# Better retrieval with document context
class Filing(Table):
__tablename__ = "filings"
id = Integer(primary_key=True)
content = Text(contextualized=True, contextualized_dim=512, fts=True)
# Each chunk embedded with awareness of surrounding chunks
# → Higher quality retrieval for long documents
```
---
## Installation
```bash
# Core (SQLite + local vectors)
pip install veclite
# With Voyage AI embeddings (recommended)
pip install veclite[voyage]
# With embedding cache (LMDB)
pip install veclite[cache]
# Everything
pip install veclite[all]
```
**Requirements:**
- Python 3.9+
- SQLite 3.35+ (included with Python)
- NumPy
---
## Sync vs Async
Choose the right API for your use case:
**Sync** - Notebooks, scripts, simple applications:
```python
from veclite import Client
client = Client.create(schema, "db.db")
results = client.table("docs").hybrid_search("query", topk=10).execute()
```
**Async** - Web apps, concurrent workloads:
```python
from veclite import AsyncClient
client = AsyncClient.create(schema, "db.db")
results = await client.table("docs").hybrid_search("query", topk=10).execute()
```
---
## When to Use VecLite
### ✅ **Perfect For**
- **RAG Systems** - Complete standalone retrieval solution
- **Agentic RAG** - Agents that choose retrieval strategies dynamically
- **Semantic Search** - Find documents by meaning, not just keywords
- **Jupyter Notebooks** - Interactive development and analysis
- **Desktop Applications** - Local-first semantic search
- **Edge/IoT Devices** - On-device retrieval without external APIs
### ❌ **NOT For**
- **Production web servers** - Use Qdrant, Pinecone, Weaviate instead
- **Multi-tenant SaaS** - VecLite is single-tenant by design
- **High concurrency** - SQLite write limitations
---
## Documentation
📚 **[Full Documentation](https://veclite.readthedocs.io)**
- [Installation](https://veclite.readthedocs.io/installation.md)
- [Quickstart Guide](https://veclite.readthedocs.io/getting-started/quickstart.md)
- [Vector Search](https://veclite.readthedocs.io/search/vector.md)
- [Keyword Search](https://veclite.readthedocs.io/search/keyword.md)
- [Hybrid Search](https://veclite.readthedocs.io/search/hybrid.md)
- [Schema Definition](https://veclite.readthedocs.io/getting-started/schema.md)
- [API Reference](https://veclite.readthedocs.io/api/client.md)
---
## Testing
```bash
# Run all 120 tests
pytest tests/
# Run specific test suite
pytest tests/test_vector_search_sync.py
pytest tests/test_hybrid_search_async.py
```
---
## Contributing
Contributions welcome! VecLite is designed to be simple and focused on RAG use cases.
```bash
git clone https://github.com/lucasastorian/veclite.git
cd veclite
pip install -e ".[dev]"
pytest tests/
```
---
## License
MIT License - see [LICENSE](LICENSE) file for details.
---
## Acknowledgments
- Built on SQLite's FTS5 for BM25 keyword search
- Inspired by Supabase's fluent query API
- Optimized for RAG and local-first applications
- Voyage AI for state-of-the-art embeddings
---
**[View Docs](https://veclite.readthedocs.io)** | **[GitHub](https://github.com/lucasastorian/veclite)** | **[Issues](https://github.com/lucasastorian/veclite/issues)**
Raw data
{
"_id": null,
"home_page": null,
"name": "veclite",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Lucas Astorian <your.email@example.com>",
"keywords": "sqlite, orm, vector-database, embeddings, semantic-search, hybrid-search, rag, vector-search",
"author": null,
"author_email": "Lucas Astorian <your.email@example.com>",
"download_url": "https://files.pythonhosted.org/packages/c4/b8/9f429db8623041860e558ea451c8ccd03a4c1cc53aad0dc49f78f4569f88/veclite-0.1.0.tar.gz",
"platform": null,
"description": "# VecLite\n\n**Local\u2011first SQLite + vectors for agentic RAG \u2014 zero infra.**\n\n\n\n\n\nVecLite isn't a traditional ORM adapted for search \u2014 it's **purpose\u2011built for retrieval**. It combines relational filters, BM25, vectors, hybrid fusion, and optional reranking in one lightweight library that runs entirely on your machine.\n\n**Why VecLite exists:**\n- \u26a1 **Local\u2011first, zero infra** \u2014 SQLite + vectors; no services to run\n- \ud83e\uddf0 **No plumbing** \u2014 embeddings + indices managed automatically\n- \ud83e\udde9 **Relational + views** \u2014 enrich chunks without denormalization\n- \ud83d\udd0e **Three search modes** \u2014 BM25, vector, hybrid (+ optional rerank modifier)\n- \u2705 **Atomic embeddings** \u2014 batch once; no rows without vectors on failure\n- \ud83d\udd17 **Simple filters** \u2014 eq/in_/between/JSON contains/ilike/regex\n- \ud83d\udce6 **One folder** \u2014 your entire RAG + DB stack (sqlite.db + vectors/)\n- \ud83d\uddc2\ufe0f **No metadata limits** \u2014 normal SQL columns/JSON, any shape\n- \ud83e\uddf1 **Vector + non\u2011vector tables** \u2014 mix FTS\u2011only and regular tables alongside vectors\n\n---\n\n## Comparison\n\n| Capability | VecLite (SQLite + Vectors) | Typical Managed Vector DBs |\n| --- | --- | --- |\n| Setup | Local\u2011first, zero infra | Hosted service / cluster |\n| Data model | Relational tables + views + JSON | Vector\u2011centric, document/NoSQL |\n| Non\u2011vector queries | Yes (filters, ranges, regex, FTS) | Limited (metadata filters only) |\n| Joins / views | Yes (via SQL views) | No |\n| Keyword search | Yes (BM25 via FTS5) | Rare / not native |\n| Vector search | Yes (cosine) | Yes |\n| Hybrid search | Yes (vector + BM25) | Varies; often custom |\n| Rerank | Optional modifier | External / varies |\n| CRUD | Full insert/update/upsert/delete | Insert/update; deletes vary |\n| Metadata limits | No (normal SQL columns/JSON) | Often constrained (field count/size) |\n| Record size limits | No hard per\u2011row payload limit | Common (~40KB metadata payloads) |\n| Non\u2011vector tables | Yes (regular/FTS\u2011only tables) | No (vector\u2011centric indexes) |\n| Consistency | Atomic batch embeddings | Varies by service |\n| Storage | One folder: sqlite.db + vectors/ | Remote indexes/storage |\n| Best for | Local RAG, agents, notebooks | Production scale, multi\u2011tenant APIs |\n\nVecLite is not a production, multi\u2011tenant vector service \u2014 it\u2019s a compact, relational RAG database you can fully own and query normally.\n\nNote: Many managed vector services impose strict metadata payload limits (e.g., ~40KB per record) and center the data model around vector indexes; VecLite lets you store any shape/size of metadata and create tables without vectors at all.\n\n> **Use VecLite for:** Local\u2011first RAG, agentic retrieval, Jupyter notebooks, desktop/edge apps\n> **Not for:** High\u2011traffic multi\u2011tenant servers; drop\u2011in vector store adapters (integrate via custom retrievers instead)\n\n---\n\n## Quick Start\n\nBuild a semantic search system in 5 lines:\n\n```bash\npip install veclite[voyage] # Includes Voyage AI embeddings\nexport VOYAGE_API_KEY=\"your-key\" # Get from https://www.voyageai.com\n```\n\n```python\nfrom veclite import Client, Schema\nfrom veclite.schema import Table, Integer, Text\n\n# 1. Define schema - embeddings happen automatically\nclass Document(Table):\n __tablename__ = \"documents\"\n id = Integer(primary_key=True)\n content = Text(vector=True, fts=True) # vector + keyword search\n\n# 2. Create database (nested folder)\nschema = Schema()\nschema.add_table(Document)\nclient = Client.create(schema, \"rag_db\") # creates ./rag_db/{sqlite.db, vectors/}\n\n# 3. Insert - embeddings generated automatically\nclient.table(\"documents\").insert([\n {\"content\": \"Python is a programming language\"},\n {\"content\": \"Machine learning uses neural networks\"},\n {\"content\": \"The Solar System has 8 planets\"},\n]).execute()\n\n# 4. Search by meaning (finds ML doc, not Python)\nresults = client.table(\"documents\").vector_search(\n query=\"AI and deep learning\",\n topk=5\n).execute()\n```\n\n**That's it.** No embedding pipelines, no vector databases, no infrastructure.\n\n---\n\n## Atomic Batch Embeddings (Consistency)\n\nEnsure all\u2011or\u2011nothing inserts with batched embeddings:\n\n```python\n# Async example\nasync with async_client.batch_embeddings():\n await async_client.table(\"documents\").insert([...]).execute()\n await async_client.table(\"documents\").insert([...]).execute()\n# If any embedding fails \u2192 rollback SQLite; no vectors written\n```\n\n- Default is atomic: one SQLite transaction; embeddings generated; vectors written; then COMMIT.\n- Non\u2011atomic option: `async with db.batch_embeddings(atomic=False): ...` batches for efficiency and writes failures to an outbox for later retry via `flush_vector_outbox()`.\n\n---\n\n## Search Modes (+ optional rerank modifier)\n\nVecLite provides the complete retrieval stack - perfect for agentic RAG systems that need different search strategies:\n\n### 1. \ud83d\udd0d **Keyword Search (BM25)**\nTraditional full-text search with production-grade BM25 ranking:\n```python\nresults = client.table(\"docs\").keyword_search(\n query=\"machine learning transformers\",\n topk=10\n).execute()\n```\n\n**Use when:** Exact term matching matters (product codes, names, technical terms)\n\n### 2. \ud83c\udfaf **Vector Search (Semantic)**\nFind by meaning, not just keywords:\n```python\nresults = client.table(\"docs\").vector_search(\n query=\"AI tutorials for beginners\", # Matches \"ML guides for newcomers\"\n topk=10\n).execute()\n```\n\n**Use when:** Semantic similarity matters more than exact terms\n\n### 3. \ud83d\ude80 **Hybrid Search (Best of Both)**\nCombines keyword + vector with Reciprocal Rank Fusion:\n```python\nresults = client.table(\"docs\").hybrid_search(\n query=\"transformer architecture\",\n alpha=0.7, # 70% semantic, 30% keyword\n topk=10\n).execute()\n```\n\n**Use when:** You want both precision (keywords) and recall (semantics)\n**Perfect for:** General-purpose RAG retrieval\n\n### \ud83c\udf96\ufe0f Rerank Modifier (optional)\nPost-retrieval modifier to refine candidates:\n```python\nfrom veclite.embeddings import VoyageClient\n\n# Get candidates with hybrid search\ncandidates = client.table(\"docs\") \\\n .hybrid_search(\"quantum computing\", topk=100) \\\n .execute()\n\n# Rerank top 100 \u2192 best 10\nembedder = VoyageClient()\nreranked = embedder.rerank(\n query=\"quantum computing applications\",\n documents=[doc[\"content\"] for doc in candidates.data],\n top_k=10\n)\n```\n\n**Use when:** Quality > speed (2-stage retrieval)\n\n---\n\n\n\n## Perfect for Agentic RAG\n\nVecLite's modular design makes it ideal for agentic systems where the AI chooses retrieval strategies:\n\n```python\nclass RAGAgent:\n def retrieve(self, query: str, strategy: str = \"auto\"):\n if strategy == \"auto\":\n # Agent decides based on query type\n if self._is_technical_query(query):\n return self.keyword_search(query)\n else:\n return self.hybrid_search(query)\n\n elif strategy == \"keyword\":\n return self.db.table(\"docs\").keyword_search(query, topk=10)\n\n elif strategy == \"semantic\":\n return self.db.table(\"docs\").vector_search(query, topk=10)\n\n elif strategy == \"hybrid\":\n return self.db.table(\"docs\").hybrid_search(query, alpha=0.7, topk=10)\n\n elif strategy == \"deep\":\n # Two-stage: hybrid \u2192 rerank\n candidates = self.db.table(\"docs\").hybrid_search(query, topk=100)\n return self.embedder.rerank(query, candidates.data, top_k=10)\n```\n\n**Agents can:**\n- Choose search strategies dynamically\n- Combine multiple retrieval modes\n- Filter by metadata before/after search\n- Iteratively refine with different strategies\n\n---\n\n## Recipe: SEC Filings (Relational + FTS + Vectors)\n\nKeep filings in one DB, pages as FTS\u2011only, and chunks with vectors. Let an agent both retrieve semantically and read exact page ranges.\n\n```python\nfrom veclite import Client, Schema\nfrom veclite.schema import Table, Integer, Text, Boolean\n\nclass Filings(Table):\n __tablename__ = \"filings\"\n id = Integer(primary_key=True)\n ticker = Text(index=True)\n form_type = Text(index=True)\n filing_date = Text(index=True)\n\nclass FilingPages(Table):\n __tablename__ = \"filing_pages\"\n id = Integer(primary_key=True)\n filing_id = Integer(index=True)\n page_number = Integer(index=True)\n content = Text(fts=True) # FTS only\n\nclass FilingChunks(Table):\n __tablename__ = \"filing_chunks\"\n id = Integer(primary_key=True)\n filing_id = Integer(index=True)\n page = Integer(index=True)\n content = Text(vector=True, fts=True) # vectors + FTS\n has_table = Boolean(default=False)\n\nschema = Schema()\nschema.add_table(Filings)\nschema.add_table(FilingPages)\nschema.add_table(FilingChunks)\n\nclient = Client.create(schema, \"sec_db\") # ./sec_db/{sqlite.db, vectors/}\n\n# Hybrid retrieval on chunks within a filing\nq = \"Apple risk factors and competitive challenges\"\nhits = client.table(\"filing_chunks\").hybrid_search(q, topk=10, alpha=0.7) \\\n .eq(\"filing_id\", 12345).execute()\n\n# Read a page window around best hit\nbest = hits.data[0]\npage = best[\"page\"]\nfiling_id = best[\"filing_id\"]\npages = client.table(\"filing_pages\").select(\"*\") \\\n .eq(\"filing_id\", filing_id) \\\n .between(\"page_number\", page - 1, page + 1) \\\n .order(\"page_number\") \\\n .execute()\n```\n\nThis three\u2011table pattern (filings, filing_pages, filing_chunks) gives agents precision (page ranges) and recall (semantic + keyword).\n\n## Automatic Embeddings\n\nNo manual embedding pipeline needed - VecLite handles everything:\n\n```python\n# Mark field for auto-embeddings\nclass Paper(Table):\n __tablename__ = \"papers\"\n id = Integer(primary_key=True)\n title = Text()\n abstract = Text(vector=True, fts=True) # Auto-embed on insert/update\n year = Integer()\n```\n\n**What happens automatically:**\n- \u2705 Embeddings generated on insert/update\n- \u2705 Batching for efficiency\n- \u2705 LMDB caching (avoid re-embedding)\n- \u2705 Vector storage alongside SQLite\n\n**Supported models:**\n- `vector=True` \u2192 voyage-3.5-lite (512D, **default**)\n- `vector=VectorConfig.voyage_3()` \u2192 voyage-3 (1024D)\n- `vector=VectorConfig.voyage_large()` \u2192 voyage-3.5-large (1536D)\n- `contextualized=True` \u2192 voyage-context-3 (contextualized retrieval, 512D default)\n\n---\n\n## Advanced Search Examples\n\n### Filtered Search\n```python\n# Search within filtered subset\nresults = client.table(\"papers\") \\\n .hybrid_search(\"climate impacts\", alpha=0.6, topk=20) \\\n .eq(\"category\", \"science\") \\\n .gt(\"year\", 2020) \\\n .is_not_null(\"peer_reviewed\") \\\n .execute()\n```\n\n### Multi-Field Search\n```python\nclass Article(Table):\n __tablename__ = \"articles\"\n id = Integer(primary_key=True)\n title = Text(vector=True, fts=True)\n body = Text(vector=True, fts=True)\n\n# Search specific field\nresults = client.table(\"articles\").vector_search(\n query=\"AI safety\",\n column=\"title\", # Search titles only\n topk=10\n).execute()\n```\n\n### Contextualized Embeddings (Advanced RAG)\n```python\n# Better retrieval with document context\nclass Filing(Table):\n __tablename__ = \"filings\"\n id = Integer(primary_key=True)\n content = Text(contextualized=True, contextualized_dim=512, fts=True)\n\n# Each chunk embedded with awareness of surrounding chunks\n# \u2192 Higher quality retrieval for long documents\n```\n\n---\n\n## Installation\n\n```bash\n# Core (SQLite + local vectors)\npip install veclite\n\n# With Voyage AI embeddings (recommended)\npip install veclite[voyage]\n\n# With embedding cache (LMDB)\npip install veclite[cache]\n\n# Everything\npip install veclite[all]\n```\n\n**Requirements:**\n- Python 3.9+\n- SQLite 3.35+ (included with Python)\n- NumPy\n\n---\n\n## Sync vs Async\n\nChoose the right API for your use case:\n\n**Sync** - Notebooks, scripts, simple applications:\n```python\nfrom veclite import Client\n\nclient = Client.create(schema, \"db.db\")\nresults = client.table(\"docs\").hybrid_search(\"query\", topk=10).execute()\n```\n\n**Async** - Web apps, concurrent workloads:\n```python\nfrom veclite import AsyncClient\n\nclient = AsyncClient.create(schema, \"db.db\")\nresults = await client.table(\"docs\").hybrid_search(\"query\", topk=10).execute()\n```\n\n---\n\n## When to Use VecLite\n\n### \u2705 **Perfect For**\n- **RAG Systems** - Complete standalone retrieval solution\n- **Agentic RAG** - Agents that choose retrieval strategies dynamically\n- **Semantic Search** - Find documents by meaning, not just keywords\n- **Jupyter Notebooks** - Interactive development and analysis\n- **Desktop Applications** - Local-first semantic search\n- **Edge/IoT Devices** - On-device retrieval without external APIs\n\n### \u274c **NOT For**\n- **Production web servers** - Use Qdrant, Pinecone, Weaviate instead\n- **Multi-tenant SaaS** - VecLite is single-tenant by design\n- **High concurrency** - SQLite write limitations\n\n---\n\n## Documentation\n\n\ud83d\udcda **[Full Documentation](https://veclite.readthedocs.io)**\n\n- [Installation](https://veclite.readthedocs.io/installation.md)\n- [Quickstart Guide](https://veclite.readthedocs.io/getting-started/quickstart.md)\n- [Vector Search](https://veclite.readthedocs.io/search/vector.md)\n- [Keyword Search](https://veclite.readthedocs.io/search/keyword.md)\n- [Hybrid Search](https://veclite.readthedocs.io/search/hybrid.md)\n- [Schema Definition](https://veclite.readthedocs.io/getting-started/schema.md)\n- [API Reference](https://veclite.readthedocs.io/api/client.md)\n\n---\n\n## Testing\n\n```bash\n# Run all 120 tests\npytest tests/\n\n# Run specific test suite\npytest tests/test_vector_search_sync.py\npytest tests/test_hybrid_search_async.py\n```\n\n---\n\n## Contributing\n\nContributions welcome! VecLite is designed to be simple and focused on RAG use cases.\n\n```bash\ngit clone https://github.com/lucasastorian/veclite.git\ncd veclite\npip install -e \".[dev]\"\npytest tests/\n```\n\n---\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n---\n\n## Acknowledgments\n\n- Built on SQLite's FTS5 for BM25 keyword search\n- Inspired by Supabase's fluent query API\n- Optimized for RAG and local-first applications\n- Voyage AI for state-of-the-art embeddings\n\n---\n\n**[View Docs](https://veclite.readthedocs.io)** | **[GitHub](https://github.com/lucasastorian/veclite)** | **[Issues](https://github.com/lucasastorian/veclite/issues)**\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A schema-first SQLite ORM with built-in vector embeddings and hybrid search",
"version": "0.1.0",
"project_urls": {
"Changelog": "https://github.com/yourusername/veclite/blob/main/CHANGELOG.md",
"Documentation": "https://github.com/yourusername/veclite#readme",
"Homepage": "https://github.com/yourusername/veclite",
"Issues": "https://github.com/yourusername/veclite/issues",
"Repository": "https://github.com/yourusername/veclite"
},
"split_keywords": [
"sqlite",
" orm",
" vector-database",
" embeddings",
" semantic-search",
" hybrid-search",
" rag",
" vector-search"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6a64d729b8ef655bfd94c94de7ae84c90212d9b027ab7c1a539937089bc55f53",
"md5": "318acb810723b897d16fc62a7d50d236",
"sha256": "b722f529c8f606c055602184636986bfef389515d89b1a58e239e165054a43bd"
},
"downloads": -1,
"filename": "veclite-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "318acb810723b897d16fc62a7d50d236",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 103675,
"upload_time": "2025-10-19T21:10:27",
"upload_time_iso_8601": "2025-10-19T21:10:27.966855Z",
"url": "https://files.pythonhosted.org/packages/6a/64/d729b8ef655bfd94c94de7ae84c90212d9b027ab7c1a539937089bc55f53/veclite-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "c4b89f429db8623041860e558ea451c8ccd03a4c1cc53aad0dc49f78f4569f88",
"md5": "c4e63a6e8c7e7f5abea283d5534528bd",
"sha256": "ce30558438156e9290c50eb9cd2ed1736f8be773f8f0a0edec05108867a90561"
},
"downloads": -1,
"filename": "veclite-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "c4e63a6e8c7e7f5abea283d5534528bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 82158,
"upload_time": "2025-10-19T21:10:29",
"upload_time_iso_8601": "2025-10-19T21:10:29.501722Z",
"url": "https://files.pythonhosted.org/packages/c4/b8/9f429db8623041860e558ea451c8ccd03a4c1cc53aad0dc49f78f4569f88/veclite-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-19 21:10:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "veclite",
"github_not_found": true,
"lcname": "veclite"
}
JSON
