# SemantiCache
SemantiCache is a semantic caching library that enables efficient storage and retrieval of query-response pairs using FAISS and vector embeddings. It supports cache expiration, trimming strategies, and query similarity search.
## Features
- **Vector-based caching** using FAISS and HuggingFace embeddings
- **Query similarity search** for retrieving semantically relevant responses
- **Automatic cache management** with size and TTL-based trimming
- **Leaderboard tracking** for frequently accessed queries
- **Persistent storage** for cache state management
## Installation
```sh
pip install semanticache
```
## Usage
### Full Documentation
Read the [docs](https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md)
_________________________________________________________________________
### Initializing SemantiCache
```python
from semanticache import SemantiCache
cache = SemantiCache(
trim_by_size=True,
cache_path="./sem_cache",
config_path="./sem_config",
cache_size=100,
ttl=3600,
threshold=0.1,
leaderboard_top_n=5,
log_level="INFO"
)
```
### Storing a Query-Response Pair
```python
cache.set("What is the capital of France?", "Paris")
```
### Retrieving a Cached Response
```python
response = cache.get("What is the capital of France?")
print(response) # Output: Paris
```
### Clearing the Cache
```python
cache.clear(clear_files=True) # Clears all cache entries and files
```
### Reading the Leaderboard
```python
leaderboard = cache.read_leaderboard()
print(leaderboard) # Outputs most frequently accessed queries
```
## Configuration
SemantiCache stores configuration parameters in `./sem_config/sem_config.yaml`. These include:
```yaml
cache:
path: ./sem_cache
name: sem_cache_index
cache_size: 100
ttl: 3600
threshold: 0.1
trim_by_size: True
leaderboard_top_n: 5
```
## Example: Using the Cache with an LLM
This example demonstrates how to check the cache before querying an LLM and how to store responses when needed.
```python
from semanticache import SemantiCache
def call_llm(query):
"""Simulate an LLM call (replace with actual API call)."""
return f"Response for: {query}"
# Initialize the cache
cache = SemantiCache()
# Example query
query = "What is the capital of France?"
# Check if the response is already cached
cached_response = cache.get(query)
if cached_response:
print(f"Cache Hit: {cached_response}")
else:
print("Cache Miss: Querying LLM...")
response = call_llm(query)
cache.set(query, response)
print(f"Stored in cache: {response}")
```
## Advanced Settings
### Cache Trimming
- **By Size:** Keeps the most accessed entries up to `cache_size`
- **By Time (TTL):** Removes entries older than `ttl` seconds
This is toggled by setting `trim_by_size` to `True` or `False` in config file or during initialization in script
### Similarity Threshold
- Determines when a query matches an existing cache entry
- A lower threshold increases exact matches, while a higher one allows more flexible retrieval
## Dependencies
- `FAISS` for vector similarity search
- `HuggingFace` from `Langchain Community` for embedding generation
- `yaml`, `numpy`, `json`, and `pickle` for serialization
## Help
Feel free to reach out to me or create a new issue if you encounter any problems using SemantiCache
## Contribution: Possible Improvements/Ideas
- [ ] More unit tests
- [ ] Less dependence on other libraries
- [ ] Support for alternate vector index engines like ChromaDB, Milvus, etc.
- [ ] More optimized logic where possible
- [ ] Implement more sophisticated ranking and pruning algorithms.
- [ ] Support additional embedding models for improved semantic search.
## Authors
Contributors names and contact info
*Abraham Audu*
* GitHub - [@the_abrahamaudu](https://github.com/theabrahamaudu)
* X (formerly Twitter) - [@the_abrahamaudu](https://x.com/the_abrahamaudu)
* LinkedIn - [@theabrahamaudu](https://www.linkedin.com/in/theabrahamaudu/)
* Instagram - [@the_abrahamaudu](https://www.instagram.com/the_abrahamaudu/)
* YouTube - [@DataCodePy](https://www.youtube.com/@DataCodePy)
## Version History
* See [commit change](https://github.com/theabrahamaudu/SemantiCache/commits/main/)
* See [release history](https://github.com/theabrahamaudu/SemantiCache/releases)
## Acknowledgments
* This library was built on top of `FAISS`, `HuggingFace` and `LangChain`
## License
This project is licensed under the GNU GENERAL PUBLIC LICENSE Version 3.
Raw data
{
"_id": null,
"home_page": null,
"name": "SemantiCache",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "llm, faiss, semantic, cache, langchain, langchain-community, chatbot",
"author": null,
"author_email": "Abraham Audu <abraham.audu.96@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/13/dd/2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68/semanticache-0.1.1.tar.gz",
"platform": null,
"description": "\n\n# SemantiCache\n\nSemantiCache is a semantic caching library that enables efficient storage and retrieval of query-response pairs using FAISS and vector embeddings. It supports cache expiration, trimming strategies, and query similarity search.\n\n## Features\n- **Vector-based caching** using FAISS and HuggingFace embeddings\n- **Query similarity search** for retrieving semantically relevant responses\n- **Automatic cache management** with size and TTL-based trimming\n- **Leaderboard tracking** for frequently accessed queries\n- **Persistent storage** for cache state management\n\n## Installation\n\n```sh\npip install semanticache\n```\n\n## Usage\n\n### Full Documentation\nRead the [docs](https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md)\n_________________________________________________________________________\n\n### Initializing SemantiCache\n\n```python\nfrom semanticache import SemantiCache\n\ncache = SemantiCache(\n trim_by_size=True,\n cache_path=\"./sem_cache\",\n config_path=\"./sem_config\",\n cache_size=100,\n ttl=3600,\n threshold=0.1,\n leaderboard_top_n=5,\n log_level=\"INFO\"\n)\n```\n\n### Storing a Query-Response Pair\n\n```python\ncache.set(\"What is the capital of France?\", \"Paris\")\n```\n\n### Retrieving a Cached Response\n\n```python\nresponse = cache.get(\"What is the capital of France?\")\nprint(response) # Output: Paris\n```\n\n### Clearing the Cache\n\n```python\ncache.clear(clear_files=True) # Clears all cache entries and files\n```\n\n### Reading the Leaderboard\n\n```python\nleaderboard = cache.read_leaderboard()\nprint(leaderboard) # Outputs most frequently accessed queries\n```\n\n## Configuration\nSemantiCache stores configuration parameters in `./sem_config/sem_config.yaml`. These include:\n\n```yaml\ncache:\n path: ./sem_cache\n name: sem_cache_index\n cache_size: 100\n ttl: 3600\n threshold: 0.1\n trim_by_size: True\n leaderboard_top_n: 5\n```\n\n## Example: Using the Cache with an LLM\n\nThis example demonstrates how to check the cache before querying an LLM and how to store responses when needed.\n\n```python\nfrom semanticache import SemantiCache\n\ndef call_llm(query):\n \"\"\"Simulate an LLM call (replace with actual API call).\"\"\"\n return f\"Response for: {query}\"\n\n# Initialize the cache\ncache = SemantiCache()\n\n# Example query\nquery = \"What is the capital of France?\"\n\n# Check if the response is already cached\ncached_response = cache.get(query)\n\nif cached_response:\n print(f\"Cache Hit: {cached_response}\")\nelse:\n print(\"Cache Miss: Querying LLM...\")\n response = call_llm(query)\n cache.set(query, response)\n print(f\"Stored in cache: {response}\")\n```\n\n## Advanced Settings\n### Cache Trimming\n- **By Size:** Keeps the most accessed entries up to `cache_size`\n- **By Time (TTL):** Removes entries older than `ttl` seconds\n\n This is toggled by setting `trim_by_size` to `True` or `False` in config file or during initialization in script\n\n### Similarity Threshold\n- Determines when a query matches an existing cache entry\n- A lower threshold increases exact matches, while a higher one allows more flexible retrieval\n\n## Dependencies\n- `FAISS` for vector similarity search\n- `HuggingFace` from `Langchain Community` for embedding generation\n- `yaml`, `numpy`, `json`, and `pickle` for serialization\n\n## Help\nFeel free to reach out to me or create a new issue if you encounter any problems using SemantiCache\n\n## Contribution: Possible Improvements/Ideas\n\n- [ ] More unit tests\n- [ ] Less dependence on other libraries\n- [ ] Support for alternate vector index engines like ChromaDB, Milvus, etc.\n- [ ] More optimized logic where possible\n- [ ] Implement more sophisticated ranking and pruning algorithms.\n- [ ] Support additional embedding models for improved semantic search.\n\n## Authors\n\nContributors names and contact info\n\n*Abraham Audu*\n\n* GitHub - [@the_abrahamaudu](https://github.com/theabrahamaudu)\n* X (formerly Twitter) - [@the_abrahamaudu](https://x.com/the_abrahamaudu)\n* LinkedIn - [@theabrahamaudu](https://www.linkedin.com/in/theabrahamaudu/)\n* Instagram - [@the_abrahamaudu](https://www.instagram.com/the_abrahamaudu/)\n* YouTube - [@DataCodePy](https://www.youtube.com/@DataCodePy)\n\n## Version History\n\n* See [commit change](https://github.com/theabrahamaudu/SemantiCache/commits/main/)\n* See [release history](https://github.com/theabrahamaudu/SemantiCache/releases)\n\n## Acknowledgments\n\n* This library was built on top of `FAISS`, `HuggingFace` and `LangChain`\n\n## License\nThis project is licensed under the GNU GENERAL PUBLIC LICENSE Version 3.\n\n\n",
"bugtrack_url": null,
"license": "GNU General Public License v3",
"summary": "Semantic caching library for LLM apps",
"version": "0.1.1",
"project_urls": {
"Documentation": "https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md",
"Homepage": "https://github.com/theabrahamaudu/SemantiCache.git",
"Issues": "https://github.com/theabrahamaudu/SemantiCache/issues"
},
"split_keywords": [
"llm",
" faiss",
" semantic",
" cache",
" langchain",
" langchain-community",
" chatbot"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "dcc229f9b2cd04c3dbb39123ad21097491539c49b3727d83b4200d92080f95ed",
"md5": "16583b8e07b5360b615f59a17b0349b4",
"sha256": "c385dbef488436a3ac2d5be572a3b54ca9b7f1a88fc0554e53a9c8ca5684b882"
},
"downloads": -1,
"filename": "semanticache-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "16583b8e07b5360b615f59a17b0349b4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 41524,
"upload_time": "2025-03-03T00:35:44",
"upload_time_iso_8601": "2025-03-03T00:35:44.902690Z",
"url": "https://files.pythonhosted.org/packages/dc/c2/29f9b2cd04c3dbb39123ad21097491539c49b3727d83b4200d92080f95ed/semanticache-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "13dd2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68",
"md5": "fbcb556251d50a0540c46810743c3d7e",
"sha256": "97b88392ead03fc47130dea7e1bea6cde41e1bf24b877de78a7089dfc8136854"
},
"downloads": -1,
"filename": "semanticache-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "fbcb556251d50a0540c46810743c3d7e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 49928,
"upload_time": "2025-03-03T00:35:47",
"upload_time_iso_8601": "2025-03-03T00:35:47.342627Z",
"url": "https://files.pythonhosted.org/packages/13/dd/2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68/semanticache-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-03 00:35:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "theabrahamaudu",
"github_project": "SemantiCache",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "click",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "pytest_cov",
"specs": []
},
{
"name": "Sphinx",
"specs": []
},
{
"name": "coverage",
"specs": []
},
{
"name": "flake8",
"specs": []
},
{
"name": "pyyaml",
"specs": []
},
{
"name": "python-ulid",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "langchain",
"specs": [
[
"==",
"0.1.20"
]
]
},
{
"name": "langchain-community",
"specs": [
[
"==",
"0.0.38"
]
]
},
{
"name": "sentence-transformers",
"specs": [
[
"==",
"3.3.1"
]
]
},
{
"name": "faiss-cpu",
"specs": [
[
"==",
"1.8.0"
]
]
}
],
"lcname": "semanticache"
}
JSON
