# Argilla-Haystack
Argilla is an open-source platform for data-centric LLM development. Integrates human and model feedback loops for continuous LLM refinement and oversight.
With Argilla's Python SDK and adaptable UI, you can create human and model-in-the-loop workflows for:
- Supervised fine-tuning
- Preference tuning (RLHF, DPO, RLAIF, and more)
- Small, specialized NLP models
- Scalable evaluation.
## Getting Started
You first need to install argilla and argilla-haystack as follows:
```bash
pip install argilla argilla-haystack
```
You will need to an Argilla Server running to monitor the LLM. You can either install the server locally or have it on HuggingFace Spaces. For a complete guide on how to install and initialize the server, you can refer to the [Quickstart Guide](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html).
## Usage
You can use your Haystack agent with Argilla with just a simple step. After the agent is created, we will need to call the handler to log the data into Argilla.
Let us create a simple pipeline with a conversational agent. Also, we will use GPT3.5 from OpenAI as our LLM. For this, you will need a valid API key from OpenAI. You can have more info and get one via [this link](https://openai.com/blog/openai-api).
After you get your API key, let us import the key.
```python
import os
from getpass import getpass
openai_api_key = os.getenv("OPENAI_API_KEY", None) or getpass("Enter OpenAI API key:")
```
With the code snippet below, let us create the agent.
```python
from haystack.nodes import PromptNode
from haystack.agents.memory import ConversationSummaryMemory
from haystack.agents.conversational import ConversationalAgent
# Define the node with the model
prompt_node = PromptNode(
model_name_or_path="gpt-3.5-turbo-instruct", api_key=openai_api_key, max_length=256, stop_words=["Human"]
)
summary_memory = ConversationSummaryMemory(prompt_node)
conversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)
```
Let us import the ArgillaCallback and run it. Note that the dataset with the given name will be pulled from Argilla server. If the dataset does not exist, it will be created with the given name.
```python
from argilla_haystack import ArgillaCallback
api_key = "argilla.apikey"
api_url = "http://localhost:6900/"
dataset_name = "conversational_ai"
ArgillaCallback(agent=conversational_agent, dataset_name=dataset_name, api_url=api_url, api_key=api_key)
```
Now, let us run the agent to obtain a response. The prompt given and the response obtained will be logged in to Argilla server.
```python
conversational_agent.run("Tell me three most interesting things about Istanbul, Turkey")
```
## Other Use Cases
Please refer to this [notebook](https://github.com/argilla-io/argilla-haystack/blob/feat/1-feature-create-argillacallback-for-haystack/docs/use_argilla_callback_in_haystack.ipynb) for a more detailed example.
Raw data
{
"_id": null,
"home_page": null,
"name": "argilla-haystack",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "annotation,llm,monitoring",
"author": null,
"author_email": "Argilla <admin@argilla.io>",
"download_url": "https://files.pythonhosted.org/packages/d0/f7/65867ab89496672bc2a3cc96bf5ce574802b33d0d6bf3fd2e6b7a7740d4d/argilla_haystack-0.0.1b0.tar.gz",
"platform": null,
"description": "# Argilla-Haystack\n\nArgilla is an open-source platform for data-centric LLM development. Integrates human and model feedback loops for continuous LLM refinement and oversight.\n\nWith Argilla's Python SDK and adaptable UI, you can create human and model-in-the-loop workflows for:\n\n- Supervised fine-tuning\n- Preference tuning (RLHF, DPO, RLAIF, and more)\n- Small, specialized NLP models\n- Scalable evaluation.\n\n## Getting Started\n\nYou first need to install argilla and argilla-haystack as follows:\n\n```bash\npip install argilla argilla-haystack\n```\n\nYou will need to an Argilla Server running to monitor the LLM. You can either install the server locally or have it on HuggingFace Spaces. For a complete guide on how to install and initialize the server, you can refer to the [Quickstart Guide](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html). \n\n## Usage\n\nYou can use your Haystack agent with Argilla with just a simple step. After the agent is created, we will need to call the handler to log the data into Argilla. \n\nLet us create a simple pipeline with a conversational agent. Also, we will use GPT3.5 from OpenAI as our LLM. For this, you will need a valid API key from OpenAI. You can have more info and get one via [this link](https://openai.com/blog/openai-api).\n\nAfter you get your API key, let us import the key.\n\n```python\nimport os\nfrom getpass import getpass\n\nopenai_api_key = os.getenv(\"OPENAI_API_KEY\", None) or getpass(\"Enter OpenAI API key:\")\n```\n\nWith the code snippet below, let us create the agent.\n\n```python\nfrom haystack.nodes import PromptNode\nfrom haystack.agents.memory import ConversationSummaryMemory\nfrom haystack.agents.conversational import ConversationalAgent\n\n# Define the node with the model\nprompt_node = PromptNode(\n model_name_or_path=\"gpt-3.5-turbo-instruct\", api_key=openai_api_key, max_length=256, stop_words=[\"Human\"]\n)\nsummary_memory = ConversationSummaryMemory(prompt_node)\nconversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)\n```\n\nLet us import the ArgillaCallback and run it. Note that the dataset with the given name will be pulled from Argilla server. If the dataset does not exist, it will be created with the given name.\n\n```python\nfrom argilla_haystack import ArgillaCallback\n\napi_key = \"argilla.apikey\"\napi_url = \"http://localhost:6900/\"\ndataset_name = \"conversational_ai\"\n\nArgillaCallback(agent=conversational_agent, dataset_name=dataset_name, api_url=api_url, api_key=api_key)\n```\n\nNow, let us run the agent to obtain a response. The prompt given and the response obtained will be logged in to Argilla server.\n\n```python\nconversational_agent.run(\"Tell me three most interesting things about Istanbul, Turkey\")\n```\n\n\n\n## Other Use Cases\n\nPlease refer to this [notebook](https://github.com/argilla-io/argilla-haystack/blob/feat/1-feature-create-argillacallback-for-haystack/docs/use_argilla_callback_in_haystack.ipynb) for a more detailed example.\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Argilla-Haystack Integration",
"version": "0.0.1b0",
"project_urls": {
"Documentation": "https://github.com/argilla-io/argilla-haystack",
"Issues": "https://github.com/argilla-io/argilla-haystack/issues",
"Source": "https://github.com/argilla-io/argilla-haystack"
},
"split_keywords": [
"annotation",
"llm",
"monitoring"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "06939c24bcc27e3005d7d0b29625a0abf30a5ae5736c6196645aa7fffbab8971",
"md5": "5ea68ea9377eac7962710258f43cae8a",
"sha256": "40faec26ac89bacdc4065c5035488e96b921dab1dbdd0f8d3056f968850485f1"
},
"downloads": -1,
"filename": "argilla_haystack-0.0.1b0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5ea68ea9377eac7962710258f43cae8a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11422,
"upload_time": "2024-01-19T12:18:31",
"upload_time_iso_8601": "2024-01-19T12:18:31.215182Z",
"url": "https://files.pythonhosted.org/packages/06/93/9c24bcc27e3005d7d0b29625a0abf30a5ae5736c6196645aa7fffbab8971/argilla_haystack-0.0.1b0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d0f765867ab89496672bc2a3cc96bf5ce574802b33d0d6bf3fd2e6b7a7740d4d",
"md5": "92534fedb373f1760351bcc7168398b2",
"sha256": "183f966be683e17585a828706239b401b79b4fc01da1d8589b2b32c2b3be43a1"
},
"downloads": -1,
"filename": "argilla_haystack-0.0.1b0.tar.gz",
"has_sig": false,
"md5_digest": "92534fedb373f1760351bcc7168398b2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 440351,
"upload_time": "2024-01-19T12:18:33",
"upload_time_iso_8601": "2024-01-19T12:18:33.347446Z",
"url": "https://files.pythonhosted.org/packages/d0/f7/65867ab89496672bc2a3cc96bf5ce574802b33d0d6bf3fd2e6b7a7740d4d/argilla_haystack-0.0.1b0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-19 12:18:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "argilla-io",
"github_project": "argilla-haystack",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "argilla-haystack"
}
JSON
