# Needle RAG tools for Haystack
[](https://pypi.org/project/needle-haystack)
[](https://pypi.org/project/needle-haystack)
This package provides `NeedleDocumentStore` and `NeedleEmbeddingRetriever` component for use in Haystack projects.
## Usage ⚡️
Get started by installing the package via `pip`.
```bash
pip install needle-haystack
```
### API Keys
We will show you building a common RAG pipeline using Needle tools and OpenAI generator.
For using these tools you must set your environment variables, `NEEDLE_API_KEY` and `OPENAI_API_KEY` respectively.
You can get your Needle API key from from [Developer settings](https://needle-ai.com/dashboard/settings).
### Example Pipeline 🧱
In Needle document stores are called collections. For detailed information, see our [docs](https://docs.needle-ai.com).
You can create a reference to your Needle collection using `NeedleDocumentStore` and use `NeedleEmbeddingRetriever` to retrieve documents from it.
```python
from needle_haystack import NeedleDocumentStore, NeedleEmbeddingRetriever
document_store = NeedleDocumentStore(collection_id="<your-collection-id>")
retriever = NeedleEmbeddingRetriever(document_store=document_store)
```
Use the retriever in a Haystack pipeline. Example:
```python
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder
prompt_template = """
Given the following retrieved documents, generate a concise and informative answer to the query:
Query: {{query}}
Documents:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
Answer:
"""
prompt_builder = PromptBuilder(template=prompt_template)
llm = OpenAIGenerator()
# Add components to pipeline
pipeline = Pipeline()
pipeline.add_component("retriever", retriever)
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
# Connect the components
pipeline.connect("retriever", "prompt_builder.documents")
pipeline.connect("prompt_builder", "llm")
```
Run your RAG pipeline:
```python
prompt = "What is the topic of the news?"
result = basic_rag_pipeline.run({
"retriever": {"text": prompt},
"prompt_builder": {"query": prompt}
})
# Print final answer
print(result['llm']['replies'][0])
```
# Support 📞
For detailed guides, take a look at our [docs](https://docs.needle-ai.com). If you have questions or requests you can contact us in our [Discord channel](https://discord.gg/JzJcHgTyZx).
# License
`needle-haystack` is distributed under the terms of the MIT license.
Raw data
{
"_id": null,
"home_page": null,
"name": "needle-haystack-ai",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "needle, api, retrieval-augmented generation, rag, information-retrieval, artificial intelligence, ai",
"author": "Onur Eken",
"author_email": "m.onureken@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/83/bf/5a6c2af4191ff67800ce92c86ac2e0f2f8d03bec038c6ef158ffb304fffc/needle_haystack_ai-0.1.0.tar.gz",
"platform": null,
"description": "# Needle RAG tools for Haystack\n\n[](https://pypi.org/project/needle-haystack)\n[](https://pypi.org/project/needle-haystack)\n\nThis package provides `NeedleDocumentStore` and `NeedleEmbeddingRetriever` component for use in Haystack projects.\n\n## Usage \u26a1\ufe0f\n\nGet started by installing the package via `pip`.\n\n```bash\npip install needle-haystack\n```\n\n### API Keys\n\nWe will show you building a common RAG pipeline using Needle tools and OpenAI generator.\nFor using these tools you must set your environment variables, `NEEDLE_API_KEY` and `OPENAI_API_KEY` respectively.\n\nYou can get your Needle API key from from [Developer settings](https://needle-ai.com/dashboard/settings).\n\n### Example Pipeline \ud83e\uddf1\n\nIn Needle document stores are called collections. For detailed information, see our [docs](https://docs.needle-ai.com).\nYou can create a reference to your Needle collection using `NeedleDocumentStore` and use `NeedleEmbeddingRetriever` to retrieve documents from it.\n\n```python\nfrom needle_haystack import NeedleDocumentStore, NeedleEmbeddingRetriever\n\ndocument_store = NeedleDocumentStore(collection_id=\"<your-collection-id>\")\nretriever = NeedleEmbeddingRetriever(document_store=document_store)\n```\n\nUse the retriever in a Haystack pipeline. Example:\n\n```python\nfrom haystack import Pipeline\nfrom haystack.components.generators import OpenAIGenerator\nfrom haystack.components.builders import PromptBuilder\n\nprompt_template = \"\"\"\nGiven the following retrieved documents, generate a concise and informative answer to the query:\n\nQuery: {{query}}\nDocuments:\n{% for doc in documents %}\n {{ doc.content }}\n{% endfor %}\n\nAnswer:\n\"\"\"\n\nprompt_builder = PromptBuilder(template=prompt_template)\nllm = OpenAIGenerator()\n\n# Add components to pipeline\npipeline = Pipeline()\npipeline.add_component(\"retriever\", retriever)\npipeline.add_component(\"prompt_builder\", prompt_builder)\npipeline.add_component(\"llm\", llm)\n\n# Connect the components\npipeline.connect(\"retriever\", \"prompt_builder.documents\")\npipeline.connect(\"prompt_builder\", \"llm\")\n```\n\nRun your RAG pipeline:\n\n```python\nprompt = \"What is the topic of the news?\"\n\nresult = basic_rag_pipeline.run({\n \"retriever\": {\"text\": prompt},\n \"prompt_builder\": {\"query\": prompt}\n})\n\n# Print final answer\nprint(result['llm']['replies'][0])\n```\n\n# Support \ud83d\udcde\n\nFor detailed guides, take a look at our [docs](https://docs.needle-ai.com). If you have questions or requests you can contact us in our [Discord channel](https://discord.gg/JzJcHgTyZx). \n\n# License\n\n`needle-haystack` is distributed under the terms of the MIT license.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Needle RAG tools for Haystack",
"version": "0.1.0",
"project_urls": {
"documentation": "https://docs.needle-ai.com",
"homepage": "https://needle-ai.com",
"issues": "https://github.com/JANHMS/needle-haystack/issues",
"repository": "https://github.com/JANHMS/needle-haystack"
},
"split_keywords": [
"needle",
" api",
" retrieval-augmented generation",
" rag",
" information-retrieval",
" artificial intelligence",
" ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1ac93cda0d9c606cd203a341b9de462a1b0379c5b1308b1caea32c220cadf43c",
"md5": "4b4233010c559dcefa81b24a8b87cc0a",
"sha256": "f7bc10b1c6cc50c2e54d538c977e14361dd448c6e18c0f14f3174a20c6ddefbc"
},
"downloads": -1,
"filename": "needle_haystack_ai-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4b4233010c559dcefa81b24a8b87cc0a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 5431,
"upload_time": "2024-08-29T10:24:24",
"upload_time_iso_8601": "2024-08-29T10:24:24.924384Z",
"url": "https://files.pythonhosted.org/packages/1a/c9/3cda0d9c606cd203a341b9de462a1b0379c5b1308b1caea32c220cadf43c/needle_haystack_ai-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "83bf5a6c2af4191ff67800ce92c86ac2e0f2f8d03bec038c6ef158ffb304fffc",
"md5": "297978237681f439dd514bfa0a074cd1",
"sha256": "e91c6a59373cffc31a7f31728d81ad11121154367a8525701a24be2bf903a99b"
},
"downloads": -1,
"filename": "needle_haystack_ai-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "297978237681f439dd514bfa0a074cd1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 5069,
"upload_time": "2024-08-29T10:24:26",
"upload_time_iso_8601": "2024-08-29T10:24:26.415835Z",
"url": "https://files.pythonhosted.org/packages/83/bf/5a6c2af4191ff67800ce92c86ac2e0f2f8d03bec038c6ef158ffb304fffc/needle_haystack_ai-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-29 10:24:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "JANHMS",
"github_project": "needle-haystack",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "needle-haystack-ai"
}
JSON
