# HyperDB
<div>
<img src="https://github.com/jdagdelen/hyperDB/blob/main/_static/logo.png?raw=true" width="400" alt="HyperDB Logo">
</div>
A hyper-fast local vector database for use with LLM Agents.
Want to invest? [Now accepting SAFEs ($135M cap minimum.)](https://www.youtube.com/watch?v=QH2-TGUlwu4)
Join the [HyperDBiscord](https://discord.gg/8YQzexAA)
## Advantages
* Simple interface compatible with _all_ large language model agents.
* Highly optimized C++ backend vector store with HW accelerated operations via MKL BLAS.
* Enables users to index documents with advanced features such as _ids_ and _metadata_.
## Installation
Install the package from PyPI:
```bash
pip install hyperdb-python
```
## Usage
Here's an example of using HyperDB to store and query documents with information about all 151 original pokemon _in an instant_:
```python
import json
from hyperdb import HyperDB
# Load documents from the JSONL file
documents = []
with open("demo/pokemon.jsonl", "r") as f:
for line in f:
documents.append(json.loads(line))
# Instantiate HyperDB with the list of documents
db = HyperDB(documents, key="info.description")
# Save the HyperDB instance to a file
db.save("demo/pokemon_hyperdb.pickle.gz")
# Load the HyperDB instance from the save file
db.load("demo/pokemon_hyperdb.pickle.gz")
# Query the HyperDB instance with a text input
results = db.query("Likes to sleep.", top_k=5)
```
Returns
```
Name: Snorlax
Pokedex ID: 143
HP: 160
Type: normal
Weakness: fighting
Description: Very lazy. Just eats and sleeps. As its rotund bulk builds, it becomes steadily more slothful.
Name: Drowzee
Pokedex ID: 96
HP: 230
Type: psychic
Weakness: dark
Description: Puts enemies to sleep then eats their dreams. Occasionally gets sick from eating bad dreams.
Name: Pinsir
Pokedex ID: 127
HP: 160
Type: bug
Weakness: fire
Description: When the temperature drops at night, it sleeps on treetops or among roots where it is well hidden.
Name: Abra
Pokedex ID: 63
HP: 160
Type: psychic
Weakness: dark
Description: Sleeps 18 hours a day. If it senses danger, it will teleport itself to safety even as it sleeps.
Name: Venonat
Pokedex ID: 48
HP: 160
Type: bug
Weakness: fire
Description: Lives in the shadows of tall trees where it eats insects. It is attracted by light at night.
```
<img width="600" src="https://raw.githubusercontent.com/jdagdelen/hyperDB/main/_static/0B147C7D-BEB0-4E61-9397-64A460C8CE22.png"/>
*Benchmark Credit: Benim Kıçım
Raw data
{
"_id": null,
"home_page": "https://github.com/jdagdelen/hyperdb",
"name": "hyperdb-python",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "John Dagdelen",
"author_email": "jjdagdelen@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e6/12/545be990a8bceca4d78d04543762b6320c58dbfd5a36f4a847549d078e4a/hyperdb-python-0.1.4.tar.gz",
"platform": null,
"description": "# HyperDB\n<div>\n<img src=\"https://github.com/jdagdelen/hyperDB/blob/main/_static/logo.png?raw=true\" width=\"400\" alt=\"HyperDB Logo\">\n</div>\n\nA hyper-fast local vector database for use with LLM Agents. \n\nWant to invest? [Now accepting SAFEs ($135M cap minimum.)](https://www.youtube.com/watch?v=QH2-TGUlwu4)\n\nJoin the [HyperDBiscord](https://discord.gg/8YQzexAA)\n\n## Advantages\n* Simple interface compatible with _all_ large language model agents. \n* Highly optimized C++ backend vector store with HW accelerated operations via MKL BLAS. \n* Enables users to index documents with advanced features such as _ids_ and _metadata_.\n\n## Installation\n\nInstall the package from PyPI:\n\n```bash\npip install hyperdb-python\n```\n\n## Usage\n\nHere's an example of using HyperDB to store and query documents with information about all 151 original pokemon _in an instant_:\n\n```python\nimport json\nfrom hyperdb import HyperDB\n\n# Load documents from the JSONL file\ndocuments = []\n\nwith open(\"demo/pokemon.jsonl\", \"r\") as f:\n for line in f:\n documents.append(json.loads(line))\n\n# Instantiate HyperDB with the list of documents\ndb = HyperDB(documents, key=\"info.description\")\n\n# Save the HyperDB instance to a file\ndb.save(\"demo/pokemon_hyperdb.pickle.gz\")\n\n# Load the HyperDB instance from the save file\ndb.load(\"demo/pokemon_hyperdb.pickle.gz\")\n\n# Query the HyperDB instance with a text input\nresults = db.query(\"Likes to sleep.\", top_k=5)\n```\n\nReturns \n```\nName: Snorlax\nPokedex ID: 143\nHP: 160\nType: normal\nWeakness: fighting\nDescription: Very lazy. Just eats and sleeps. As its rotund bulk builds, it becomes steadily more slothful.\n\nName: Drowzee\nPokedex ID: 96\nHP: 230\nType: psychic\nWeakness: dark\nDescription: Puts enemies to sleep then eats their dreams. Occasionally gets sick from eating bad dreams.\n\nName: Pinsir\nPokedex ID: 127\nHP: 160\nType: bug\nWeakness: fire\nDescription: When the temperature drops at night, it sleeps on treetops or among roots where it is well hidden.\n\nName: Abra\nPokedex ID: 63\nHP: 160\nType: psychic\nWeakness: dark\nDescription: Sleeps 18 hours a day. If it senses danger, it will teleport itself to safety even as it sleeps.\n\nName: Venonat\nPokedex ID: 48\nHP: 160\nType: bug\nWeakness: fire\nDescription: Lives in the shadows of tall trees where it eats insects. It is attracted by light at night.\n```\n\n<img width=\"600\" src=\"https://raw.githubusercontent.com/jdagdelen/hyperDB/main/_static/0B147C7D-BEB0-4E61-9397-64A460C8CE22.png\"/>\n\n*Benchmark Credit: Benim K\u0131\u00e7\u0131m\n",
"bugtrack_url": null,
"license": "",
"summary": "A hyper-fast local vector database for use with LLM Agents.",
"version": "0.1.4",
"project_urls": {
"Homepage": "https://github.com/jdagdelen/hyperdb"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e612545be990a8bceca4d78d04543762b6320c58dbfd5a36f4a847549d078e4a",
"md5": "cfa0ab988289f8a3b45df933f42c19bf",
"sha256": "faf94b14f1b56579c0c8939812d0a0222cc21930bf1ea971933e5bff1d241f14"
},
"downloads": -1,
"filename": "hyperdb-python-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "cfa0ab988289f8a3b45df933f42c19bf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 5846,
"upload_time": "2023-07-10T00:35:58",
"upload_time_iso_8601": "2023-07-10T00:35:58.708273Z",
"url": "https://files.pythonhosted.org/packages/e6/12/545be990a8bceca4d78d04543762b6320c58dbfd5a36f4a847549d078e4a/hyperdb-python-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-10 00:35:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jdagdelen",
"github_project": "hyperdb",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "numpy",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "openai",
"specs": []
}
],
"lcname": "hyperdb-python"
}
JSON
