SemantiCache


NameSemantiCache JSON
Version 0.1.1 PyPI version JSON
download
home_pageNone
SummarySemantic caching library for LLM apps
upload_time2025-03-03 00:35:47
maintainerNone
docs_urlNone
authorNone
requires_python>=3.10
licenseGNU General Public License v3
keywords llm faiss semantic cache langchain langchain-community chatbot
VCS
bugtrack_url
requirements click pytest pytest_cov Sphinx coverage flake8 pyyaml python-ulid langchain langchain-community sentence-transformers faiss-cpu
Travis-CI No Travis.
coveralls test coverage No coveralls.
            ![Coverage](https://img.shields.io/badge/coverage-75%25-brightgreen)

# SemantiCache

SemantiCache is a semantic caching library that enables efficient storage and retrieval of query-response pairs using FAISS and vector embeddings. It supports cache expiration, trimming strategies, and query similarity search.

## Features
- **Vector-based caching** using FAISS and HuggingFace embeddings
- **Query similarity search** for retrieving semantically relevant responses
- **Automatic cache management** with size and TTL-based trimming
- **Leaderboard tracking** for frequently accessed queries
- **Persistent storage** for cache state management

## Installation

```sh
pip install semanticache
```

## Usage

### Full Documentation
Read the [docs](https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md)
_________________________________________________________________________

### Initializing SemantiCache

```python
from semanticache import SemantiCache

cache = SemantiCache(
    trim_by_size=True,
    cache_path="./sem_cache",
    config_path="./sem_config",
    cache_size=100,
    ttl=3600,
    threshold=0.1,
    leaderboard_top_n=5,
    log_level="INFO"
)
```

### Storing a Query-Response Pair

```python
cache.set("What is the capital of France?", "Paris")
```

### Retrieving a Cached Response

```python
response = cache.get("What is the capital of France?")
print(response)  # Output: Paris
```

### Clearing the Cache

```python
cache.clear(clear_files=True)  # Clears all cache entries and files
```

### Reading the Leaderboard

```python
leaderboard = cache.read_leaderboard()
print(leaderboard)  # Outputs most frequently accessed queries
```

## Configuration
SemantiCache stores configuration parameters in `./sem_config/sem_config.yaml`. These include:

```yaml
cache:
  path: ./sem_cache
  name: sem_cache_index
  cache_size: 100
  ttl: 3600
  threshold: 0.1
  trim_by_size: True
  leaderboard_top_n: 5
```

## Example: Using the Cache with an LLM

This example demonstrates how to check the cache before querying an LLM and how to store responses when needed.

```python
from semanticache import SemantiCache

def call_llm(query):
    """Simulate an LLM call (replace with actual API call)."""
    return f"Response for: {query}"

# Initialize the cache
cache = SemantiCache()

# Example query
query = "What is the capital of France?"

# Check if the response is already cached
cached_response = cache.get(query)

if cached_response:
    print(f"Cache Hit: {cached_response}")
else:
    print("Cache Miss: Querying LLM...")
    response = call_llm(query)
    cache.set(query, response)
    print(f"Stored in cache: {response}")
```

## Advanced Settings
### Cache Trimming
- **By Size:** Keeps the most accessed entries up to `cache_size`
- **By Time (TTL):** Removes entries older than `ttl` seconds

    This is toggled by setting `trim_by_size` to `True` or `False` in config file or during initialization in script

### Similarity Threshold
- Determines when a query matches an existing cache entry
- A lower threshold increases exact matches, while a higher one allows more flexible retrieval

## Dependencies
- `FAISS` for vector similarity search
- `HuggingFace` from `Langchain Community` for embedding generation
- `yaml`, `numpy`, `json`, and `pickle` for serialization

## Help
Feel free to reach out to me or create a new issue if you encounter any problems using SemantiCache

## Contribution: Possible Improvements/Ideas

- [ ] More unit tests
- [ ] Less dependence on other libraries
- [ ] Support for alternate vector index engines like ChromaDB, Milvus, etc.
- [ ] More optimized logic where possible
- [ ] Implement more sophisticated ranking and pruning algorithms.
- [ ] Support additional embedding models for improved semantic search.

## Authors

Contributors names and contact info

*Abraham Audu*

* GitHub - [@the_abrahamaudu](https://github.com/theabrahamaudu)
* X (formerly Twitter) - [@the_abrahamaudu](https://x.com/the_abrahamaudu)
* LinkedIn - [@theabrahamaudu](https://www.linkedin.com/in/theabrahamaudu/)
* Instagram - [@the_abrahamaudu](https://www.instagram.com/the_abrahamaudu/)
* YouTube - [@DataCodePy](https://www.youtube.com/@DataCodePy)

## Version History

* See [commit change](https://github.com/theabrahamaudu/SemantiCache/commits/main/)
* See [release history](https://github.com/theabrahamaudu/SemantiCache/releases)

## Acknowledgments

* This library was built on top of `FAISS`, `HuggingFace` and `LangChain`

## License
This project is licensed under the GNU GENERAL PUBLIC LICENSE Version 3.



            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "SemantiCache",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "llm, faiss, semantic, cache, langchain, langchain-community, chatbot",
    "author": null,
    "author_email": "Abraham Audu <abraham.audu.96@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/13/dd/2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68/semanticache-0.1.1.tar.gz",
    "platform": null,
    "description": "![Coverage](https://img.shields.io/badge/coverage-75%25-brightgreen)\n\n# SemantiCache\n\nSemantiCache is a semantic caching library that enables efficient storage and retrieval of query-response pairs using FAISS and vector embeddings. It supports cache expiration, trimming strategies, and query similarity search.\n\n## Features\n- **Vector-based caching** using FAISS and HuggingFace embeddings\n- **Query similarity search** for retrieving semantically relevant responses\n- **Automatic cache management** with size and TTL-based trimming\n- **Leaderboard tracking** for frequently accessed queries\n- **Persistent storage** for cache state management\n\n## Installation\n\n```sh\npip install semanticache\n```\n\n## Usage\n\n### Full Documentation\nRead the [docs](https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md)\n_________________________________________________________________________\n\n### Initializing SemantiCache\n\n```python\nfrom semanticache import SemantiCache\n\ncache = SemantiCache(\n    trim_by_size=True,\n    cache_path=\"./sem_cache\",\n    config_path=\"./sem_config\",\n    cache_size=100,\n    ttl=3600,\n    threshold=0.1,\n    leaderboard_top_n=5,\n    log_level=\"INFO\"\n)\n```\n\n### Storing a Query-Response Pair\n\n```python\ncache.set(\"What is the capital of France?\", \"Paris\")\n```\n\n### Retrieving a Cached Response\n\n```python\nresponse = cache.get(\"What is the capital of France?\")\nprint(response)  # Output: Paris\n```\n\n### Clearing the Cache\n\n```python\ncache.clear(clear_files=True)  # Clears all cache entries and files\n```\n\n### Reading the Leaderboard\n\n```python\nleaderboard = cache.read_leaderboard()\nprint(leaderboard)  # Outputs most frequently accessed queries\n```\n\n## Configuration\nSemantiCache stores configuration parameters in `./sem_config/sem_config.yaml`. These include:\n\n```yaml\ncache:\n  path: ./sem_cache\n  name: sem_cache_index\n  cache_size: 100\n  ttl: 3600\n  threshold: 0.1\n  trim_by_size: True\n  leaderboard_top_n: 5\n```\n\n## Example: Using the Cache with an LLM\n\nThis example demonstrates how to check the cache before querying an LLM and how to store responses when needed.\n\n```python\nfrom semanticache import SemantiCache\n\ndef call_llm(query):\n    \"\"\"Simulate an LLM call (replace with actual API call).\"\"\"\n    return f\"Response for: {query}\"\n\n# Initialize the cache\ncache = SemantiCache()\n\n# Example query\nquery = \"What is the capital of France?\"\n\n# Check if the response is already cached\ncached_response = cache.get(query)\n\nif cached_response:\n    print(f\"Cache Hit: {cached_response}\")\nelse:\n    print(\"Cache Miss: Querying LLM...\")\n    response = call_llm(query)\n    cache.set(query, response)\n    print(f\"Stored in cache: {response}\")\n```\n\n## Advanced Settings\n### Cache Trimming\n- **By Size:** Keeps the most accessed entries up to `cache_size`\n- **By Time (TTL):** Removes entries older than `ttl` seconds\n\n    This is toggled by setting `trim_by_size` to `True` or `False` in config file or during initialization in script\n\n### Similarity Threshold\n- Determines when a query matches an existing cache entry\n- A lower threshold increases exact matches, while a higher one allows more flexible retrieval\n\n## Dependencies\n- `FAISS` for vector similarity search\n- `HuggingFace` from `Langchain Community` for embedding generation\n- `yaml`, `numpy`, `json`, and `pickle` for serialization\n\n## Help\nFeel free to reach out to me or create a new issue if you encounter any problems using SemantiCache\n\n## Contribution: Possible Improvements/Ideas\n\n- [ ] More unit tests\n- [ ] Less dependence on other libraries\n- [ ] Support for alternate vector index engines like ChromaDB, Milvus, etc.\n- [ ] More optimized logic where possible\n- [ ] Implement more sophisticated ranking and pruning algorithms.\n- [ ] Support additional embedding models for improved semantic search.\n\n## Authors\n\nContributors names and contact info\n\n*Abraham Audu*\n\n* GitHub - [@the_abrahamaudu](https://github.com/theabrahamaudu)\n* X (formerly Twitter) - [@the_abrahamaudu](https://x.com/the_abrahamaudu)\n* LinkedIn - [@theabrahamaudu](https://www.linkedin.com/in/theabrahamaudu/)\n* Instagram - [@the_abrahamaudu](https://www.instagram.com/the_abrahamaudu/)\n* YouTube - [@DataCodePy](https://www.youtube.com/@DataCodePy)\n\n## Version History\n\n* See [commit change](https://github.com/theabrahamaudu/SemantiCache/commits/main/)\n* See [release history](https://github.com/theabrahamaudu/SemantiCache/releases)\n\n## Acknowledgments\n\n* This library was built on top of `FAISS`, `HuggingFace` and `LangChain`\n\n## License\nThis project is licensed under the GNU GENERAL PUBLIC LICENSE Version 3.\n\n\n",
    "bugtrack_url": null,
    "license": "GNU General Public License v3",
    "summary": "Semantic caching library for LLM apps",
    "version": "0.1.1",
    "project_urls": {
        "Documentation": "https://github.com/theabrahamaudu/SemantiCache/blob/main/docs/SemantiCacheDocs.md",
        "Homepage": "https://github.com/theabrahamaudu/SemantiCache.git",
        "Issues": "https://github.com/theabrahamaudu/SemantiCache/issues"
    },
    "split_keywords": [
        "llm",
        " faiss",
        " semantic",
        " cache",
        " langchain",
        " langchain-community",
        " chatbot"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "dcc229f9b2cd04c3dbb39123ad21097491539c49b3727d83b4200d92080f95ed",
                "md5": "16583b8e07b5360b615f59a17b0349b4",
                "sha256": "c385dbef488436a3ac2d5be572a3b54ca9b7f1a88fc0554e53a9c8ca5684b882"
            },
            "downloads": -1,
            "filename": "semanticache-0.1.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "16583b8e07b5360b615f59a17b0349b4",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 41524,
            "upload_time": "2025-03-03T00:35:44",
            "upload_time_iso_8601": "2025-03-03T00:35:44.902690Z",
            "url": "https://files.pythonhosted.org/packages/dc/c2/29f9b2cd04c3dbb39123ad21097491539c49b3727d83b4200d92080f95ed/semanticache-0.1.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "13dd2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68",
                "md5": "fbcb556251d50a0540c46810743c3d7e",
                "sha256": "97b88392ead03fc47130dea7e1bea6cde41e1bf24b877de78a7089dfc8136854"
            },
            "downloads": -1,
            "filename": "semanticache-0.1.1.tar.gz",
            "has_sig": false,
            "md5_digest": "fbcb556251d50a0540c46810743c3d7e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 49928,
            "upload_time": "2025-03-03T00:35:47",
            "upload_time_iso_8601": "2025-03-03T00:35:47.342627Z",
            "url": "https://files.pythonhosted.org/packages/13/dd/2ac3e83f8a42bf5362749dfeb00e2d8cf5c05b0df338cb1f686cea87eb68/semanticache-0.1.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-03 00:35:47",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "theabrahamaudu",
    "github_project": "SemantiCache",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [
        {
            "name": "click",
            "specs": []
        },
        {
            "name": "pytest",
            "specs": []
        },
        {
            "name": "pytest_cov",
            "specs": []
        },
        {
            "name": "Sphinx",
            "specs": []
        },
        {
            "name": "coverage",
            "specs": []
        },
        {
            "name": "flake8",
            "specs": []
        },
        {
            "name": "pyyaml",
            "specs": []
        },
        {
            "name": "python-ulid",
            "specs": [
                [
                    "==",
                    "3.0.0"
                ]
            ]
        },
        {
            "name": "langchain",
            "specs": [
                [
                    "==",
                    "0.1.20"
                ]
            ]
        },
        {
            "name": "langchain-community",
            "specs": [
                [
                    "==",
                    "0.0.38"
                ]
            ]
        },
        {
            "name": "sentence-transformers",
            "specs": [
                [
                    "==",
                    "3.3.1"
                ]
            ]
        },
        {
            "name": "faiss-cpu",
            "specs": [
                [
                    "==",
                    "1.8.0"
                ]
            ]
        }
    ],
    "lcname": "semanticache"
}
        
Elapsed time: 1.21486s