insightvault

Name	insightvault JSON
Version	0.0.3 JSON
	download
home_page	None
Summary	Local RAG pipeline runner
upload_time	2025-01-10 18:40:05
maintainer	None
docs_url	None
author	David Kirchhoff
requires_python	>=3.9
license	MIT License Copyright (c) 2024 David Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

# Insight Vault

**Insight Vault** is a local, privacy-focused library for building LLM-based applications. It allows you to store, search, and summarize text completely offline.

With **Insight Vault**, you can:

- **Search** your local knowledge base.
- **Chat** with your documents interactively.
- **Summarize** large sets of information into concise outputs.

All data stays **on your machine**, with no external API calls required.

For more details see the [documentation](https://daved01.github.io/insightvault/).

## 🚀 **Features**

- **Local Inference** — Uses LLAMA for local LLM inference.
- **Local Embeddings** — Embeddings are created using SentenceTransformers.
- **Privacy-first RAG** — Store and query documents locally with ChromaDB.
- **Interactive CLI** — Intuitive CLI interface for searching, managing, and summarizing.

## 📦 **Dependencies**

The following dependencies are required to run **Insight Vault**:

- [**Ollama**](https://ollama.com/) — For local LLM inference.
- [**ChromaDB**](https://docs.trychroma.com/) — Local document storage and vector database.
- [**SentenceTransformers**](https://www.sbert.net/) — Embeddings for better document search and query matching.

## 🔥 **Installation**

To install **Insight Vault**, you can use the following command:

```bash
pip install insightvault
```

## ⚙️ **Usage**

Insight Vault can be used via a simple CLI interface or as a Python library.

If you want to use the `chat` or `summarize` commands, you need to make sure that Ollama is running first.

### CLI

**Adding Documents**

```bash
insightvault manage add-file <path_to_document>
```

This command will add a file from the specified path to the local document database. You can also directly add text. The `manage` command is also used to list all documents, and to delete all documents.

**Searching Documents**

```bash
insightvault search "Explain RAG pipelines"
```

This will search your indexed documents for the query “Explain RAG pipelines” and return the most relevant results.

**Chat**

```bash
insightvault chat "Explain RAG pipelines"
```

This uses RAG which means it takes in a natural language query and returns a response in natural language based on the most relevant documents you have indexed.

**Summarizing Documents**

```bash
insightvault summarize "Explain RAG pipelines"
```

Summarizes the text you provide. The flag `--file` can be used to summarize a file.

### Library

Insightvault provides three apps as part of the library:

- `RAGApp` — For RAG pipelines.
- `SearchApp` — For searching indexed documents.
- `SummarizerApp` — For summarizing text.

For example, to use the `SummarizerApp`, you can do the following:

```python
from insightvault import SummarizerApp

app = SummarizerApp()
await app.summarize("This is a loooong test")
```

See the [API Documentation](TODO!) for more information.

## 🛠️ **Development**

If you want to contribute to Insight Vault or run it locally for development, follow these steps.

1. Clone the Repository

```bash
git clone https://github.com/daved01/insightvault.git
cd insightvault
```

2. Install Development Dependencies

```bash
pip install -e ".[dev]"
```

This installs all required libraries for development, testing, and quality checks.

3. Run Tests

```bash
pytest tests
```

This runs all the unit and integration tests to ensure everything is working properly.

4. Run Quality Checks

We use ruff, mypy, and pre-commit hooks to ensure high code quality.

```bash
# Type checking with mypy
mypy insightvault

# Linting and formatting with ruff
ruff check . --fix
ruff format .
```

Pre-commit Hooks

To automatically check for secrets, format code, and run linters before every commit, set up pre-commit hooks as follows:

```bash
pre-commit install
```

This will install the hooks and run them automatically before each commit.

---

## 📤 **Publishing**

To publish a new version of Insight Vault to PyPI:

1. Update the version in pyproject.toml.
2. Build the package:

```bash
python -m build
```

3. Publish the package:

```bash
twine upload dist/*
```

---

## 💡 **Contributing**

We welcome contributions of all kinds. Whether it’s bug fixes, new features, or improving the documentation. Please open an issue or submit a pull request.

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "insightvault",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": null,
    "keywords": null,
    "author": "David Kirchhoff",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/79/ea/750aa21e0bd90e1ada5a6860d8cc2175c4844db652bcaac9e37e21d34855/insightvault-0.0.3.tar.gz",
    "platform": null,
    "description": "# Insight Vault\n\n<p align=\"center\">\n <a href=\"https://www.python.org/downloads/release/python-312/\"><img src=\"https://img.shields.io/badge/python-3.12-green.svg\" alt=\"Python 3.12\"></a>\n <a href=\"https://opensource.org/licenses/MIT\"><img src=\"https://img.shields.io/badge/License-MIT-green.svg\" alt=\"License: MIT\"></a>\n  <a href=\"https://pypi.org/project/insightvault\"><img alt=\"PyPI - Version\" src=\"https://img.shields.io/pypi/v/insightvault?color=blue\"></a>\n <img src=\"https://github.com/daved01/insightvault/actions/workflows/quality-checks-main.yml/badge.svg\" alt=\"GitHub CI\">\n</p>\n\n**Insight Vault** is a local, privacy-focused library for building LLM-based applications. It allows you to store, search, and summarize text completely offline.\n\nWith **Insight Vault**, you can:\n\n- **Search** your local knowledge base.\n- **Chat** with your documents interactively.\n- **Summarize** large sets of information into concise outputs.\n\nAll data stays **on your machine**, with no external API calls required.\n\nFor more details see the [documentation](https://daved01.github.io/insightvault/).\n\n## \ud83d\ude80 **Features**\n\n- **Local Inference** \u2014 Uses LLAMA for local LLM inference.\n- **Local Embeddings** \u2014 Embeddings are created using SentenceTransformers.\n- **Privacy-first RAG** \u2014 Store and query documents locally with ChromaDB.\n- **Interactive CLI** \u2014 Intuitive CLI interface for searching, managing, and summarizing.\n\n## \ud83d\udce6 **Dependencies**\n\nThe following dependencies are required to run **Insight Vault**:\n\n- [**Ollama**](https://ollama.com/) \u2014 For local LLM inference.\n- [**ChromaDB**](https://docs.trychroma.com/) \u2014 Local document storage and vector database.\n- [**SentenceTransformers**](https://www.sbert.net/) \u2014 Embeddings for better document search and query matching.\n\n## \ud83d\udd25 **Installation**\n\nTo install **Insight Vault**, you can use the following command:\n\n```bash\npip install insightvault\n```\n\n## \u2699\ufe0f **Usage**\n\nInsight Vault can be used via a simple CLI interface or as a Python library.\n\nIf you want to use the `chat` or `summarize` commands, you need to make sure that Ollama is running first.\n\n### CLI\n\n**Adding Documents**\n\n```bash\ninsightvault manage add-file <path_to_document>\n```\n\nThis command will add a file from the specified path to the local document database. You can also directly add text. The `manage` command is also used to list all documents, and to delete all documents.\n\n**Searching Documents**\n\n```bash\ninsightvault search \"Explain RAG pipelines\"\n```\n\nThis will search your indexed documents for the query \u201cExplain RAG pipelines\u201d and return the most relevant results.\n\n**Chat**\n\n```bash\ninsightvault chat \"Explain RAG pipelines\"\n```\n\nThis uses RAG which means it takes in a natural language query and returns a response in natural language based on the most relevant documents you have indexed.\n\n**Summarizing Documents**\n\n```bash\ninsightvault summarize \"Explain RAG pipelines\"\n```\n\nSummarizes the text you provide. The flag `--file` can be used to summarize a file.\n\n### Library\n\nInsightvault provides three apps as part of the library:\n\n- `RAGApp` \u2014 For RAG pipelines.\n- `SearchApp` \u2014 For searching indexed documents.\n- `SummarizerApp` \u2014 For summarizing text.\n\nFor example, to use the `SummarizerApp`, you can do the following:\n\n```python\nfrom insightvault import SummarizerApp\n\napp = SummarizerApp()\nawait app.summarize(\"This is a loooong test\")\n```\n\nSee the [API Documentation](TODO!) for more information.\n\n## \ud83d\udee0\ufe0f **Development**\n\nIf you want to contribute to Insight Vault or run it locally for development, follow these steps.\n\n1. Clone the Repository\n\n```bash\ngit clone https://github.com/daved01/insightvault.git\ncd insightvault\n```\n\n2. Install Development Dependencies\n\n```bash\npip install -e \".[dev]\"\n```\n\nThis installs all required libraries for development, testing, and quality checks.\n\n3. Run Tests\n\n```bash\npytest tests\n```\n\nThis runs all the unit and integration tests to ensure everything is working properly.\n\n4. Run Quality Checks\n\nWe use ruff, mypy, and pre-commit hooks to ensure high code quality.\n\n```bash\n# Type checking with mypy\nmypy insightvault\n\n# Linting and formatting with ruff\nruff check . --fix\nruff format .\n```\n\nPre-commit Hooks\n\nTo automatically check for secrets, format code, and run linters before every commit, set up pre-commit hooks as follows:\n\n```bash\npre-commit install\n```\n\nThis will install the hooks and run them automatically before each commit.\n\n---\n\n## \ud83d\udce4 **Publishing**\n\nTo publish a new version of Insight Vault to PyPI:\n\n1. Update the version in pyproject.toml.\n2. Build the package:\n\n```bash\npython -m build\n```\n\n3. Publish the package:\n\n```bash\ntwine upload dist/*\n```\n\n---\n\n## \ud83d\udca1 **Contributing**\n\nWe welcome contributions of all kinds. Whether it\u2019s bug fixes, new features, or improving the documentation. Please open an issue or submit a pull request.\n",
    "bugtrack_url": null,
    "license": "MIT License  Copyright (c) 2024 David  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:  The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.  THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
    "summary": "Local RAG pipeline runner",
    "version": "0.0.3",
    "project_urls": null,
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "1cc17d88ada2620236d017fc1ccdb73d906709d0872a9b3d1a8c68bcc8b690cb",
                "md5": "4ea4577563af7c11ff5c2bb0333a1eb6",
                "sha256": "be80d1dc199fb00e4e8445e7a45ec644b6f7544c2a1effeb8f87467dad46fa64"
            },
            "downloads": -1,
            "filename": "insightvault-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "4ea4577563af7c11ff5c2bb0333a1eb6",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 18058,
            "upload_time": "2025-01-10T18:40:03",
            "upload_time_iso_8601": "2025-01-10T18:40:03.845309Z",
            "url": "https://files.pythonhosted.org/packages/1c/c1/7d88ada2620236d017fc1ccdb73d906709d0872a9b3d1a8c68bcc8b690cb/insightvault-0.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "79ea750aa21e0bd90e1ada5a6860d8cc2175c4844db652bcaac9e37e21d34855",
                "md5": "1b4abeb7affa9e7036287564532b369d",
                "sha256": "10d50468f7d588695f2963b42677ead86c4eaeabf822697919654538a7acc142"
            },
            "downloads": -1,
            "filename": "insightvault-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "1b4abeb7affa9e7036287564532b369d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 29273,
            "upload_time": "2025-01-10T18:40:05",
            "upload_time_iso_8601": "2025-01-10T18:40:05.164103Z",
            "url": "https://files.pythonhosted.org/packages/79/ea/750aa21e0bd90e1ada5a6860d8cc2175c4844db652bcaac9e37e21d34855/insightvault-0.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-10 18:40:05",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "insightvault"
}

David Kirchhoff