# <img src="https://raw.githubusercontent.com/vectara/py-vectara-agentic/main/.github/assets/Vectara-logo.png" alt="Vectara Logo" width="30" height="30" style="vertical-align: middle;"> vectara-agentic
<p align="center">
<a href="https://vectara.github.io/vectara-agentic-docs">Documentation</a> ·
<a href="#examples">Examples</a> ·
<a href="https://discord.gg/S9dwgCNEFs">Discord</a>
</p>
<p align="center">
<a href="https://opensource.org/licenses/Apache-2.0">
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License">
</a>
<a href="https://github.com/vectara/py-vectara-agentic/graphs/commit-activity">
<img src="https://img.shields.io/badge/Maintained%3F-yes-green.svg" alt="Maintained">
</a>
<a href="https://twitter.com/vectara">
<img src="https://img.shields.io/twitter/follow/vectara.svg?style=social&label=Follow%20%40Vectara" alt="Twitter">
</a>
</p>
## ✨ Overview
`vectara-agentic` is a Python library for developing powerful AI assistants and agents using Vectara and Agentic-RAG. It leverages the LlamaIndex Agent framework and provides helper functions to quickly create tools that connect to Vectara corpora.
<p align="center">
<img src="https://raw.githubusercontent.com/vectara/py-vectara-agentic/main/.github/assets/diagram1.png" alt="Agentic RAG diagram" width="100%" style="vertical-align: middle;">
</p>
### Features
- Enables easy creation of custom AI assistants and agents.
- Create a Vectara RAG tool or search tool with a single line of code.
- Supports `ReAct`, `OpenAIAgent`, `LATS` and `LLMCompiler` agent types.
- Includes pre-built tools for various domains (e.g., finance, legal).
- Integrates with various LLM inference services like OpenAI, Anthropic, Gemini, GROQ, Together.AI, Cohere, Bedrock and Fireworks
- Built-in support for observability with Arize Phoenix
### 📚 Example AI Assistants
Check out our example AI assistants:
- [Financial Assistant](https://huggingface.co/spaces/vectara/finance-chat)
- [Justice Harvard Teaching Assistant](https://huggingface.co/spaces/vectara/Justice-Harvard)
- [Legal Assistant](https://huggingface.co/spaces/vectara/legal-agent)
- [EV Assistant](https://huggingface.co/spaces/vectara/ev-assistant)
### Prerequisites
- [Vectara account](https://console.vectara.com/signup/?utm_source=github&utm_medium=code&utm_term=DevRel&utm_content=vectara-agentic&utm_campaign=github-code-DevRel-vectara-agentic)
- A Vectara corpus with an [API key](https://docs.vectara.com/docs/api-keys)
- [Python 3.10 or higher](https://www.python.org/downloads/)
- OpenAI API key (or API keys for Anthropic, TOGETHER.AI, Fireworks AI, Bedrock, Cohere, GEMINI or GROQ, if you choose to use them)
### Installation
```bash
pip install vectara-agentic
```
## 🚀 Quick Start
### 1. Initialize the Vectara tool factory
```python
import os
from vectara_agentic.tools import VectaraToolFactory
vec_factory = VectaraToolFactory(
vectara_api_key=os.environ['VECTARA_API_KEY'],
vectara_customer_id=os.environ['VECTARA_CUSTOMER_ID'],
vectara_corpus_key=os.environ['VECTARA_CORPUS_KEY']
)
```
### 2. Create a Vectara RAG Tool
A RAG tool calls the full Vectara RAG pipeline to provide summarized responses to queries grounded in data.
```python
from pydantic import BaseModel, Field
years = list(range(2020, 2024))
tickers = {
"AAPL": "Apple Computer",
"GOOG": "Google",
"AMZN": "Amazon",
"SNOW": "Snowflake",
}
class QueryFinancialReportsArgs(BaseModel):
query: str = Field(..., description="The user query.")
year: int | str = Field(..., description=f"The year this query relates to. An integer between {min(years)} and {max(years)} or a string specifying a condition on the year (example: '>2020').")
ticker: str = Field(..., description=f"The company ticker. Must be a valid ticket symbol from the list {tickers.keys()}.")
query_financial_reports_tool = vec_factory.create_rag_tool(
tool_name="query_financial_reports",
tool_description="Query financial reports for a company and year",
tool_args_schema=QueryFinancialReportsArgs,
lambda_val=0.005,
summary_num_results=7,
# Additional arguments
)
```
See the [docs](https://vectara.github.io/vectara-agentic-docs/) for additional arguments to customize your Vectara RAG tool.
### 3. Create other tools (optional)
In addition to RAG tools, you can generate a lot of other types of tools the agent can use. These could be mathematical tools, tools
that call other APIs to get more information, or any other type of tool.
See [Agent Tools](#agent-tools) for more information.
### 4. Create your agent
```python
from vectara_agentic import Agent
agent = Agent(
tools=[query_financial_reports_tool],
topic="10-K financial reports",
custom_instructions="""
- You are a helpful financial assistant in conversation with a user. Use your financial expertise when crafting a query to the tool, to ensure you get the most accurate information.
- You can answer questions, provide insights, or summarize any information from financial reports.
- A user may refer to a company's ticker instead of its full name - consider those the same when a user is asking about a company.
- When calculating a financial metric, make sure you have all the information from tools to complete the calculation.
- In many cases you may need to query tools on each sub-metric separately before computing the final metric.
- When using a tool to obtain financial data, consider the fact that information for a certain year may be reported in the following year's report.
- Report financial data in a consistent manner. For example if you report revenue in thousands, always report revenue in thousands.
"""
)
```
See the [docs](https://vectara.github.io/vectara-agentic-docs/) for additional arguments, including `agent_progress_callback` and `query_logging_callback`.
### 5. Run your agent
```python
res = agent.chat("What was the revenue for Apple in 2021?")
print(res.response)
```
Note that:
1. `vectara-agentic` also supports `achat()` and two streaming variants `stream_chat()` and `astream_chat()`.
2. The response types from `chat()` and `achat()` are of type `AgentResponse`. If you just need the actual string
response it's available as the `response` variable, or just use `str()`. For advanced use-cases you can look
at other `AgentResponse` variables [such as `sources`](https://github.com/run-llama/llama_index/blob/659f9faaafbecebb6e6c65f42143c0bf19274a37/llama-index-core/llama_index/core/chat_engine/types.py#L53).
## 🧰 Vectara tools
`vectara-agentic` provides two helper functions to connect with Vectara RAG
* `create_rag_tool()` to create an agent tool that connects with a Vectara corpus for querying.
* `create_search_tool()` to create a tool to search a Vectara corpus and return a list of matching documents.
See the documentation for the full list of arguments for `create_rag_tool()` and `create_search_tool()`,
to understand how to configure Vectara query performed by those tools.
### Creating a Vectara RAG tool
A Vectara RAG tool is often the main workhorse for any Agentic RAG application, and enables the agent to query
one or more Vectara RAG corpora.
The tool generated always includes the `query` argument, followed by 1 or more optional arguments used for
metadata filtering, defined by `tool_args_schema`.
For example, in the quickstart example the schema is:
```
class QueryFinancialReportsArgs(BaseModel):
query: str = Field(..., description="The user query.")
year: int | str = Field(..., description=f"The year this query relates to. An integer between {min(years)} and {max(years)} or a string specifying a condition on the year (example: '>2020').")
ticker: str = Field(..., description=f"The company ticker. Must be a valid ticket symbol from the list {tickers.keys()}.")
```
The `query` is required and is always the query string.
The other arguments are optional and will be interpreted as Vectara metadata filters.
For example, in the example above, the agent may call the `query_financial_reports_tool` tool with
query='what is the revenue?', year=2022 and ticker='AAPL'. Subsequently the RAG tool will issue
a Vectara RAG query with the same query, but with metadata filtering (doc.year=2022 and doc.ticker='AAPL').
There are also additional cool features supported here:
* An argument can be a condition, for example year='>2022' translates to the correct metadata
filtering condition doc.year>2022
* if `fixed_filter` is defined in the RAG tool, it provides a constant metadata filtering that is always applied.
For example, if fixed_filter=`doc.filing_type='10K'` then a query with query='what is the reveue', year=2022
and ticker='AAPL' would translate into query='what is the revenue' with metadata filtering condition of
"doc.year=2022 AND doc.ticker='AAPL' and doc.filing_type='10K'"
Note that `tool_args_type` is an optional dictionary that indicates the level at which metadata filtering
is applied for each argument (`doc` or `part`)
### Creating a Vectara search tool
The Vectara search tool allows the agent to list documents that match a query.
This can be helpful to the agent to answer queries like "how many documents discuss the iPhone?" or other
similar queries that require a response in terms of a list of matching documents.
## 🛠️ Agent Tools at a Glance
`vectara-agentic` provides a few tools out of the box (see ToolsCatalog for details):
1. **Standard tools**:
- `summarize_text`: a tool to summarize a long text into a shorter summary (uses LLM)
- `rephrase_text`: a tool to rephrase a given text, given a set of rephrase instructions (uses LLM)
These tools use an LLM and so would use the `Tools` LLM specified in your `AgentConfig`.
To instantiate them:
```python
from vectara_agentic.tools_catalog import ToolsCatalog
summarize_text = ToolsCatalog(agent_config).summarize_text
```
This ensures the summarize_text tool is configured with the proper LLM provider and model as
specified in the Agent configuration.
2. **Legal tools**: a set of tools for the legal vertical, such as:
- `summarize_legal_text`: summarize legal text with a certain point of view
- `critique_as_judge`: critique a legal text as a judge, providing their perspective
3. **Financial tools**: based on tools from Yahoo! Finance:
- tools to understand the financials of a public company like: `balance_sheet`, `income_statement`, `cash_flow`
- `stock_news`: provides news about a company
- `stock_analyst_recommendations`: provides stock analyst recommendations for a company.
4. **Database tools**: providing tools to inspect and query a database
- `list_tables`: list all tables in the database
- `describe_tables`: describe the schema of tables in the database
- `load_data`: returns data based on a SQL query
- `load_sample_data`: returns the first 25 rows of a table
- `load_unique_values`: returns the top unique values for a given column
In addition, we include various other tools from LlamaIndex ToolSpecs:
* Tavily search and EXA.AI
* arxiv
* neo4j & Kuzu for Graph DB integration
* Google tools (including gmail, calendar, and search)
* Slack
Note that some of these tools may require API keys as environment variables
You can create your own tool directly from a Python function using the `create_tool()` method of the `ToolsFactory` class:
```python
def mult_func(x, y):
return x * y
mult_tool = ToolsFactory().create_tool(mult_func)
```
Note: When you define your own Python functions as tools, implement them at the top module level,
and not as nested functions. Nested functions are not supported if you use serialization
(dumps/loads or from_dict/to_dict).
## 🛠️ Configuration
## Configuring Vectara-agentic
The main way to control the behavior of `vectara-agentic` is by passing an `AgentConfig` object to your `Agent` when creating it.
For example:
```python
agent_config = AgentConfig(
agent_type = AgentType.REACT,
main_llm_provider = ModelProvider.ANTHROPIC,
main_llm_model_name = 'claude-3-5-sonnet-20241022',
tool_llm_provider = ModelProvider.TOGETHER,
tool_llm_model_name = 'meta-llama/Llama-3.3-70B-Instruct-Turbo'
)
agent = Agent(
tools=[query_financial_reports_tool],
topic="10-K financial reports",
custom_instructions="You are a helpful financial assistant in conversation with a user.",
agent_config=agent_config
)
```
The `AgentConfig` object may include the following items:
- `agent_type`: the agent type. Valid values are `REACT`, `LLMCOMPILER`, `LATS` or `OPENAI` (default: `OPENAI`).
- `main_llm_provider` and `tool_llm_provider`: the LLM provider for main agent and for the tools. Valid values are `OPENAI`, `ANTHROPIC`, `TOGETHER`, `GROQ`, `COHERE`, `BEDROCK`, `GEMINI` or `FIREWORKS` (default: `OPENAI`).
- `main_llm_model_name` and `tool_llm_model_name`: agent model name for agent and tools (default depends on provider).
- `observer`: the observer type; should be `ARIZE_PHOENIX` or if undefined no observation framework will be used.
- `endpoint_api_key`: a secret key if using the API endpoint option (defaults to `dev-api-key`)
If any of these are not provided, `AgentConfig` first tries to read the values from the OS environment.
## Configuring Vectara RAG or search tools
When creating a `VectaraToolFactory`, you can pass in a `vectara_api_key`, `vectara_customer_id`, and `vectara_corpus_id` to the factory.
If not passed in, it will be taken from the environment variables (`VECTARA_API_KEY` and `VECTARA_CORPUS_KEY`). Note that `VECTARA_CORPUS_KEY` can be a single KEY or a comma-separated list of KEYs (if you want to query multiple corpora).
These values will be used as credentials when creating Vectara tools - in `create_rag_tool()` and `create_search_tool()`.
## Setting up a privately hosted LLM
If you want to setup vectara-agentic to use your own self-hosted LLM endpoint, follow the example below
```python
config = AgentConfig(
agent_type=AgentType.REACT,
main_llm_provider=ModelProvider.PRIVATE,
main_llm_model_name="meta-llama/Meta-Llama-3.1-8B-Instruct",
private_llm_api_base="http://vllm-server.company.com/v1",
private_llm_api_key="TEST_API_KEY",
)
agent = Agent(agent_config=config, tools=tools, topic=topic,
custom_instructions=custom_instructions)
```
In this case we specify the Main LLM provider to be privately hosted with Llama-3.1-8B as the model.
- The `ModelProvider.PRIVATE` specifies a privately hosted LLM.
- The `private_llm_api_base` specifies the api endpoint to use, and the `private_llm_api_key`
specifies the private API key requires to use this service.
## ℹ️ Additional Information
### About Custom Instructions for your Agent
The custom instructions you provide to the agent guide its behavior.
Here are some guidelines when creating your instructions:
- Write precise and clear instructions, without overcomplicating.
- Consider edge cases and unusual or atypical scenarios.
- Be cautious to not over-specify behavior based on your primary use-case, as it may limit the agent's ability to behave properly in others.
### Diagnostics
The `Agent` class defines a few helpful methods to help you understand the internals of your application.
* The `report()` method prints out the agent object’s type, the tools, and the LLMs used for the main agent and tool calling.
* The `token_counts()` method tells you how many tokens you have used in the current session for both the main agent and tool calling LLMs. This can be helpful if you want to track spend by token.
### Serialization
The `Agent` class supports serialization. Use the `dumps()` to serialize and `loads()` to read back from a serialized stream.
Note: due to cloudpickle limitations, if a tool contains Python `weakref` objects, serialization won't work and an exception will be raised.
### Observability
vectara-agentic supports observability via the existing integration of LlamaIndex and Arize Phoenix.
First, set `VECTARA_AGENTIC_OBSERVER_TYPE` to `ARIZE_PHOENIX` in `AgentConfig` (or env variable).
Then you can use Arize Phoenix in three ways:
1. **Locally**.
1. If you have a local phoenix server that you've run using e.g. `python -m phoenix.server.main serve`, vectara-agentic will send all traces to it.
2. If not, vectara-agentic will run a local instance during the agent's lifecycle, and will close it when finished.
3. In both cases, traces will be sent to the local instance, and you can see the dashboard at `http://localhost:6006`
2. **Hosted Instance**. In this case the traces are sent to the Phoenix instances hosted on Arize.
1. Go to `https://app.phoenix.arize.com`, setup an account if you don't have one.
2. create an API key and put it in the `PHOENIX_API_KEY` environment variable - this indicates you want to use the hosted version.
3. To view the traces go to `https://app.phoenix.arize.com`.
Now when you run your agent, all call traces are sent to Phoenix and recorded.
In addition, vectara-agentic also records `FCS` (factual consistency score, aka HHEM) values into Arize for every Vectara RAG call. You can see those results in the `Feedback` column of the arize UI.
## 🌐 API Endpoint
`vectara-agentic` can be easily hosted locally or on a remote machine behind an API endpoint, by following theses steps:
### Step 1: Setup your API key
Ensure that you have your API key set up as an environment variable:
```
export VECTARA_AGENTIC_API_KEY=<YOUR-ENDPOINT-API-KEY>
```
if you don't specify an Endpoint API key it uses the default "dev-api-key".
### Step 2: Start the API Server
Initialize the agent and start the FastAPI server by following this example:
```
from vectara_agentic.agent import Agent
from vectara_agentic.agent_endpoint import start_app
agent = Agent(...) # Initialize your agent with appropriate parameters
start_app(agent)
```
You can customize the host and port by passing them as arguments to `start_app()`:
* Default: host="0.0.0.0" and port=8000.
For example:
```
start_app(agent, host="0.0.0.0", port=8000)
```
### Step 3: Access the API Endpoint
Once the server is running, you can interact with it using curl or any HTTP client. For example:
```
curl -G "http://<remote-server-ip>:8000/chat" \
--data-urlencode "message=What is Vectara?" \
-H "X-API-Key: <YOUR-ENDPOINT-API-KEY>"
```
## 🤝 Contributing
We welcome contributions! Please see our [contributing guide](https://github.com/vectara/py-vectara-agentic/blob/main/CONTRIBUTING.md) for more information.
## 📝 License
This project is licensed under the Apache 2.0 License. See the [LICENSE](https://github.com/vectara/py-vectara-agentic/blob/master/LICENSE) file for details.
## 📞 Contact
- Website: [vectara.com](https://vectara.com)
- Twitter: [@vectara](https://twitter.com/vectara)
- GitHub: [@vectara](https://github.com/vectara)
- LinkedIn: [@vectara](https://www.linkedin.com/company/vectara/)
- Discord: [Join our community](https://discord.gg/GFb8gMz6UH)
Raw data
{
"_id": null,
"home_page": "https://github.com/vectara/py-vectara-agentic",
"name": "vectara-agentic",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "LLM, NLP, RAG, Agentic-RAG, AI assistant, AI Agent, Vectara",
"author": "Ofer Mendelevitch",
"author_email": "ofer@vectara.com",
"download_url": "https://files.pythonhosted.org/packages/22/0e/206f7201b6a44e8ff228fb775c21c3752118ebb792fdcb11cd71c75fd98f/vectara_agentic-0.2.1.tar.gz",
"platform": null,
"description": "# <img src=\"https://raw.githubusercontent.com/vectara/py-vectara-agentic/main/.github/assets/Vectara-logo.png\" alt=\"Vectara Logo\" width=\"30\" height=\"30\" style=\"vertical-align: middle;\"> vectara-agentic\n\n<p align=\"center\">\n <a href=\"https://vectara.github.io/vectara-agentic-docs\">Documentation</a> \u00b7\n <a href=\"#examples\">Examples</a> \u00b7\n <a href=\"https://discord.gg/S9dwgCNEFs\">Discord</a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://opensource.org/licenses/Apache-2.0\">\n <img src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\" alt=\"License\">\n </a>\n <a href=\"https://github.com/vectara/py-vectara-agentic/graphs/commit-activity\">\n <img src=\"https://img.shields.io/badge/Maintained%3F-yes-green.svg\" alt=\"Maintained\">\n </a>\n <a href=\"https://twitter.com/vectara\">\n <img src=\"https://img.shields.io/twitter/follow/vectara.svg?style=social&label=Follow%20%40Vectara\" alt=\"Twitter\">\n </a>\n</p>\n\n## \u2728 Overview\n\n`vectara-agentic` is a Python library for developing powerful AI assistants and agents using Vectara and Agentic-RAG. It leverages the LlamaIndex Agent framework and provides helper functions to quickly create tools that connect to Vectara corpora.\n\n<p align=\"center\">\n<img src=\"https://raw.githubusercontent.com/vectara/py-vectara-agentic/main/.github/assets/diagram1.png\" alt=\"Agentic RAG diagram\" width=\"100%\" style=\"vertical-align: middle;\">\n</p>\n\n### Features\n\n- Enables easy creation of custom AI assistants and agents.\n- Create a Vectara RAG tool or search tool with a single line of code.\n- Supports `ReAct`, `OpenAIAgent`, `LATS` and `LLMCompiler` agent types.\n- Includes pre-built tools for various domains (e.g., finance, legal).\n- Integrates with various LLM inference services like OpenAI, Anthropic, Gemini, GROQ, Together.AI, Cohere, Bedrock and Fireworks\n- Built-in support for observability with Arize Phoenix\n\n### \ud83d\udcda Example AI Assistants\n\nCheck out our example AI assistants:\n\n- [Financial Assistant](https://huggingface.co/spaces/vectara/finance-chat)\n- [Justice Harvard Teaching Assistant](https://huggingface.co/spaces/vectara/Justice-Harvard)\n- [Legal Assistant](https://huggingface.co/spaces/vectara/legal-agent)\n- [EV Assistant](https://huggingface.co/spaces/vectara/ev-assistant)\n\n### Prerequisites\n\n- [Vectara account](https://console.vectara.com/signup/?utm_source=github&utm_medium=code&utm_term=DevRel&utm_content=vectara-agentic&utm_campaign=github-code-DevRel-vectara-agentic)\n- A Vectara corpus with an [API key](https://docs.vectara.com/docs/api-keys)\n- [Python 3.10 or higher](https://www.python.org/downloads/)\n- OpenAI API key (or API keys for Anthropic, TOGETHER.AI, Fireworks AI, Bedrock, Cohere, GEMINI or GROQ, if you choose to use them)\n\n### Installation\n\n```bash\npip install vectara-agentic\n```\n\n## \ud83d\ude80 Quick Start\n\n### 1. Initialize the Vectara tool factory\n\n```python\nimport os\nfrom vectara_agentic.tools import VectaraToolFactory\n\nvec_factory = VectaraToolFactory(\n vectara_api_key=os.environ['VECTARA_API_KEY'],\n vectara_customer_id=os.environ['VECTARA_CUSTOMER_ID'],\n vectara_corpus_key=os.environ['VECTARA_CORPUS_KEY']\n)\n```\n\n### 2. Create a Vectara RAG Tool\n\nA RAG tool calls the full Vectara RAG pipeline to provide summarized responses to queries grounded in data.\n\n```python\nfrom pydantic import BaseModel, Field\n\nyears = list(range(2020, 2024))\ntickers = {\n \"AAPL\": \"Apple Computer\",\n \"GOOG\": \"Google\",\n \"AMZN\": \"Amazon\",\n \"SNOW\": \"Snowflake\",\n}\n\nclass QueryFinancialReportsArgs(BaseModel):\n query: str = Field(..., description=\"The user query.\")\n year: int | str = Field(..., description=f\"The year this query relates to. An integer between {min(years)} and {max(years)} or a string specifying a condition on the year (example: '>2020').\")\n ticker: str = Field(..., description=f\"The company ticker. Must be a valid ticket symbol from the list {tickers.keys()}.\")\n\nquery_financial_reports_tool = vec_factory.create_rag_tool(\n tool_name=\"query_financial_reports\",\n tool_description=\"Query financial reports for a company and year\",\n tool_args_schema=QueryFinancialReportsArgs,\n lambda_val=0.005,\n summary_num_results=7, \n # Additional arguments\n)\n```\n\nSee the [docs](https://vectara.github.io/vectara-agentic-docs/) for additional arguments to customize your Vectara RAG tool.\n\n### 3. Create other tools (optional)\n\nIn addition to RAG tools, you can generate a lot of other types of tools the agent can use. These could be mathematical tools, tools \nthat call other APIs to get more information, or any other type of tool.\n\nSee [Agent Tools](#agent-tools) for more information.\n\n### 4. Create your agent\n\n```python\nfrom vectara_agentic import Agent\n\nagent = Agent(\n tools=[query_financial_reports_tool],\n topic=\"10-K financial reports\",\n custom_instructions=\"\"\"\n - You are a helpful financial assistant in conversation with a user. Use your financial expertise when crafting a query to the tool, to ensure you get the most accurate information.\n - You can answer questions, provide insights, or summarize any information from financial reports.\n - A user may refer to a company's ticker instead of its full name - consider those the same when a user is asking about a company.\n - When calculating a financial metric, make sure you have all the information from tools to complete the calculation.\n - In many cases you may need to query tools on each sub-metric separately before computing the final metric.\n - When using a tool to obtain financial data, consider the fact that information for a certain year may be reported in the following year's report.\n - Report financial data in a consistent manner. For example if you report revenue in thousands, always report revenue in thousands.\n \"\"\"\n)\n```\n\nSee the [docs](https://vectara.github.io/vectara-agentic-docs/) for additional arguments, including `agent_progress_callback` and `query_logging_callback`.\n\n### 5. Run your agent\n\n```python\nres = agent.chat(\"What was the revenue for Apple in 2021?\")\nprint(res.response)\n```\n\nNote that:\n1. `vectara-agentic` also supports `achat()` and two streaming variants `stream_chat()` and `astream_chat()`.\n2. The response types from `chat()` and `achat()` are of type `AgentResponse`. If you just need the actual string\n response it's available as the `response` variable, or just use `str()`. For advanced use-cases you can look \n at other `AgentResponse` variables [such as `sources`](https://github.com/run-llama/llama_index/blob/659f9faaafbecebb6e6c65f42143c0bf19274a37/llama-index-core/llama_index/core/chat_engine/types.py#L53).\n\n## \ud83e\uddf0 Vectara tools\n\n`vectara-agentic` provides two helper functions to connect with Vectara RAG\n* `create_rag_tool()` to create an agent tool that connects with a Vectara corpus for querying. \n* `create_search_tool()` to create a tool to search a Vectara corpus and return a list of matching documents.\n\nSee the documentation for the full list of arguments for `create_rag_tool()` and `create_search_tool()`, \nto understand how to configure Vectara query performed by those tools.\n\n### Creating a Vectara RAG tool\n\nA Vectara RAG tool is often the main workhorse for any Agentic RAG application, and enables the agent to query \none or more Vectara RAG corpora. \n\nThe tool generated always includes the `query` argument, followed by 1 or more optional arguments used for \nmetadata filtering, defined by `tool_args_schema`.\n\nFor example, in the quickstart example the schema is:\n\n```\nclass QueryFinancialReportsArgs(BaseModel):\n query: str = Field(..., description=\"The user query.\")\n year: int | str = Field(..., description=f\"The year this query relates to. An integer between {min(years)} and {max(years)} or a string specifying a condition on the year (example: '>2020').\")\n ticker: str = Field(..., description=f\"The company ticker. Must be a valid ticket symbol from the list {tickers.keys()}.\")\n```\n\nThe `query` is required and is always the query string.\nThe other arguments are optional and will be interpreted as Vectara metadata filters.\n\nFor example, in the example above, the agent may call the `query_financial_reports_tool` tool with \nquery='what is the revenue?', year=2022 and ticker='AAPL'. Subsequently the RAG tool will issue\na Vectara RAG query with the same query, but with metadata filtering (doc.year=2022 and doc.ticker='AAPL').\n\nThere are also additional cool features supported here:\n* An argument can be a condition, for example year='>2022' translates to the correct metadata \n filtering condition doc.year>2022\n* if `fixed_filter` is defined in the RAG tool, it provides a constant metadata filtering that is always applied.\n For example, if fixed_filter=`doc.filing_type='10K'` then a query with query='what is the reveue', year=2022\n and ticker='AAPL' would translate into query='what is the revenue' with metadata filtering condition of\n \"doc.year=2022 AND doc.ticker='AAPL' and doc.filing_type='10K'\"\n\nNote that `tool_args_type` is an optional dictionary that indicates the level at which metadata filtering\nis applied for each argument (`doc` or `part`)\n\n### Creating a Vectara search tool\n\nThe Vectara search tool allows the agent to list documents that match a query.\nThis can be helpful to the agent to answer queries like \"how many documents discuss the iPhone?\" or other\nsimilar queries that require a response in terms of a list of matching documents.\n\n## \ud83d\udee0\ufe0f Agent Tools at a Glance\n\n`vectara-agentic` provides a few tools out of the box (see ToolsCatalog for details):\n\n1. **Standard tools**: \n- `summarize_text`: a tool to summarize a long text into a shorter summary (uses LLM)\n- `rephrase_text`: a tool to rephrase a given text, given a set of rephrase instructions (uses LLM)\nThese tools use an LLM and so would use the `Tools` LLM specified in your `AgentConfig`.\nTo instantiate them:\n\n```python\nfrom vectara_agentic.tools_catalog import ToolsCatalog\nsummarize_text = ToolsCatalog(agent_config).summarize_text\n```\n\nThis ensures the summarize_text tool is configured with the proper LLM provider and model as \nspecified in the Agent configuration.\n\n2. **Legal tools**: a set of tools for the legal vertical, such as:\n- `summarize_legal_text`: summarize legal text with a certain point of view\n- `critique_as_judge`: critique a legal text as a judge, providing their perspective\n\n3. **Financial tools**: based on tools from Yahoo! Finance:\n- tools to understand the financials of a public company like: `balance_sheet`, `income_statement`, `cash_flow`\n- `stock_news`: provides news about a company\n- `stock_analyst_recommendations`: provides stock analyst recommendations for a company.\n\n4. **Database tools**: providing tools to inspect and query a database\n- `list_tables`: list all tables in the database\n- `describe_tables`: describe the schema of tables in the database\n- `load_data`: returns data based on a SQL query\n- `load_sample_data`: returns the first 25 rows of a table\n- `load_unique_values`: returns the top unique values for a given column\n\nIn addition, we include various other tools from LlamaIndex ToolSpecs:\n* Tavily search and EXA.AI\n* arxiv\n* neo4j & Kuzu for Graph DB integration\n* Google tools (including gmail, calendar, and search)\n* Slack\n\nNote that some of these tools may require API keys as environment variables\n\nYou can create your own tool directly from a Python function using the `create_tool()` method of the `ToolsFactory` class:\n\n```python\ndef mult_func(x, y):\n return x * y\n\nmult_tool = ToolsFactory().create_tool(mult_func)\n```\n\nNote: When you define your own Python functions as tools, implement them at the top module level,\nand not as nested functions. Nested functions are not supported if you use serialization \n(dumps/loads or from_dict/to_dict).\n\n## \ud83d\udee0\ufe0f Configuration\n\n## Configuring Vectara-agentic\n\nThe main way to control the behavior of `vectara-agentic` is by passing an `AgentConfig` object to your `Agent` when creating it.\nFor example:\n\n```python\nagent_config = AgentConfig(\n agent_type = AgentType.REACT,\n main_llm_provider = ModelProvider.ANTHROPIC,\n main_llm_model_name = 'claude-3-5-sonnet-20241022',\n tool_llm_provider = ModelProvider.TOGETHER,\n tool_llm_model_name = 'meta-llama/Llama-3.3-70B-Instruct-Turbo'\n)\n\nagent = Agent(\n tools=[query_financial_reports_tool],\n topic=\"10-K financial reports\",\n custom_instructions=\"You are a helpful financial assistant in conversation with a user.\",\n agent_config=agent_config\n)\n```\n\nThe `AgentConfig` object may include the following items:\n- `agent_type`: the agent type. Valid values are `REACT`, `LLMCOMPILER`, `LATS` or `OPENAI` (default: `OPENAI`).\n- `main_llm_provider` and `tool_llm_provider`: the LLM provider for main agent and for the tools. Valid values are `OPENAI`, `ANTHROPIC`, `TOGETHER`, `GROQ`, `COHERE`, `BEDROCK`, `GEMINI` or `FIREWORKS` (default: `OPENAI`).\n- `main_llm_model_name` and `tool_llm_model_name`: agent model name for agent and tools (default depends on provider).\n- `observer`: the observer type; should be `ARIZE_PHOENIX` or if undefined no observation framework will be used.\n- `endpoint_api_key`: a secret key if using the API endpoint option (defaults to `dev-api-key`)\n\nIf any of these are not provided, `AgentConfig` first tries to read the values from the OS environment.\n\n## Configuring Vectara RAG or search tools\n\nWhen creating a `VectaraToolFactory`, you can pass in a `vectara_api_key`, `vectara_customer_id`, and `vectara_corpus_id` to the factory. \n\nIf not passed in, it will be taken from the environment variables (`VECTARA_API_KEY` and `VECTARA_CORPUS_KEY`). Note that `VECTARA_CORPUS_KEY` can be a single KEY or a comma-separated list of KEYs (if you want to query multiple corpora).\n\nThese values will be used as credentials when creating Vectara tools - in `create_rag_tool()` and `create_search_tool()`.\n\n## Setting up a privately hosted LLM\n\nIf you want to setup vectara-agentic to use your own self-hosted LLM endpoint, follow the example below\n\n```python\n config = AgentConfig(\n agent_type=AgentType.REACT,\n main_llm_provider=ModelProvider.PRIVATE,\n main_llm_model_name=\"meta-llama/Meta-Llama-3.1-8B-Instruct\",\n private_llm_api_base=\"http://vllm-server.company.com/v1\",\n private_llm_api_key=\"TEST_API_KEY\",\n )\n agent = Agent(agent_config=config, tools=tools, topic=topic,\n custom_instructions=custom_instructions)\n```\n\nIn this case we specify the Main LLM provider to be privately hosted with Llama-3.1-8B as the model.\n- The `ModelProvider.PRIVATE` specifies a privately hosted LLM.\n- The `private_llm_api_base` specifies the api endpoint to use, and the `private_llm_api_key`\n specifies the private API key requires to use this service.\n\n## \u2139\ufe0f Additional Information\n\n### About Custom Instructions for your Agent\n\nThe custom instructions you provide to the agent guide its behavior.\nHere are some guidelines when creating your instructions:\n- Write precise and clear instructions, without overcomplicating.\n- Consider edge cases and unusual or atypical scenarios.\n- Be cautious to not over-specify behavior based on your primary use-case, as it may limit the agent's ability to behave properly in others.\n\n### Diagnostics\n\nThe `Agent` class defines a few helpful methods to help you understand the internals of your application. \n* The `report()` method prints out the agent object\u2019s type, the tools, and the LLMs used for the main agent and tool calling.\n* The `token_counts()` method tells you how many tokens you have used in the current session for both the main agent and tool calling LLMs. This can be helpful if you want to track spend by token.\n\n### Serialization\n\nThe `Agent` class supports serialization. Use the `dumps()` to serialize and `loads()` to read back from a serialized stream.\n\nNote: due to cloudpickle limitations, if a tool contains Python `weakref` objects, serialization won't work and an exception will be raised.\n\n### Observability\n\nvectara-agentic supports observability via the existing integration of LlamaIndex and Arize Phoenix.\nFirst, set `VECTARA_AGENTIC_OBSERVER_TYPE` to `ARIZE_PHOENIX` in `AgentConfig` (or env variable).\n\nThen you can use Arize Phoenix in three ways: \n1. **Locally**. \n 1. If you have a local phoenix server that you've run using e.g. `python -m phoenix.server.main serve`, vectara-agentic will send all traces to it.\n 2. If not, vectara-agentic will run a local instance during the agent's lifecycle, and will close it when finished.\n 3. In both cases, traces will be sent to the local instance, and you can see the dashboard at `http://localhost:6006`\n2. **Hosted Instance**. In this case the traces are sent to the Phoenix instances hosted on Arize.\n 1. Go to `https://app.phoenix.arize.com`, setup an account if you don't have one.\n 2. create an API key and put it in the `PHOENIX_API_KEY` environment variable - this indicates you want to use the hosted version.\n 3. To view the traces go to `https://app.phoenix.arize.com`.\n\nNow when you run your agent, all call traces are sent to Phoenix and recorded. \nIn addition, vectara-agentic also records `FCS` (factual consistency score, aka HHEM) values into Arize for every Vectara RAG call. You can see those results in the `Feedback` column of the arize UI.\n\n## \ud83c\udf10 API Endpoint\n\n`vectara-agentic` can be easily hosted locally or on a remote machine behind an API endpoint, by following theses steps:\n\n### Step 1: Setup your API key\nEnsure that you have your API key set up as an environment variable:\n\n```\nexport VECTARA_AGENTIC_API_KEY=<YOUR-ENDPOINT-API-KEY>\n```\n\nif you don't specify an Endpoint API key it uses the default \"dev-api-key\".\n\n### Step 2: Start the API Server\nInitialize the agent and start the FastAPI server by following this example:\n\n\n```\nfrom vectara_agentic.agent import Agent\nfrom vectara_agentic.agent_endpoint import start_app\nagent = Agent(...) # Initialize your agent with appropriate parameters\nstart_app(agent)\n```\n\nYou can customize the host and port by passing them as arguments to `start_app()`:\n* Default: host=\"0.0.0.0\" and port=8000.\nFor example:\n```\nstart_app(agent, host=\"0.0.0.0\", port=8000)\n```\n\n### Step 3: Access the API Endpoint\nOnce the server is running, you can interact with it using curl or any HTTP client. For example:\n\n```\ncurl -G \"http://<remote-server-ip>:8000/chat\" \\\n--data-urlencode \"message=What is Vectara?\" \\\n-H \"X-API-Key: <YOUR-ENDPOINT-API-KEY>\"\n```\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! Please see our [contributing guide](https://github.com/vectara/py-vectara-agentic/blob/main/CONTRIBUTING.md) for more information.\n\n## \ud83d\udcdd License\n\nThis project is licensed under the Apache 2.0 License. See the [LICENSE](https://github.com/vectara/py-vectara-agentic/blob/master/LICENSE) file for details.\n\n## \ud83d\udcde Contact\n\n- Website: [vectara.com](https://vectara.com)\n- Twitter: [@vectara](https://twitter.com/vectara)\n- GitHub: [@vectara](https://github.com/vectara)\n- LinkedIn: [@vectara](https://www.linkedin.com/company/vectara/)\n- Discord: [Join our community](https://discord.gg/GFb8gMz6UH)\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python package for creating AI Assistants and AI Agents with Vectara",
"version": "0.2.1",
"project_urls": {
"Documentation": "https://vectara.github.io/vectara-agentic-docs/",
"Homepage": "https://github.com/vectara/py-vectara-agentic"
},
"split_keywords": [
"llm",
" nlp",
" rag",
" agentic-rag",
" ai assistant",
" ai agent",
" vectara"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "78068fc05d809af9ceb3fbe0ea588aa4338b2020eeba06aebfceb18cc7688ccc",
"md5": "26f667fc0844ed38056c89f4139d55bf",
"sha256": "27ab29927ffcc00f14b46930df5dcb033077f14240743f9be72c36520a444a49"
},
"downloads": -1,
"filename": "vectara_agentic-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "26f667fc0844ed38056c89f4139d55bf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 47918,
"upload_time": "2025-02-21T07:51:38",
"upload_time_iso_8601": "2025-02-21T07:51:38.486348Z",
"url": "https://files.pythonhosted.org/packages/78/06/8fc05d809af9ceb3fbe0ea588aa4338b2020eeba06aebfceb18cc7688ccc/vectara_agentic-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "220e206f7201b6a44e8ff228fb775c21c3752118ebb792fdcb11cd71c75fd98f",
"md5": "5b7c324404446285e5727b66ec16cbec",
"sha256": "b8578a2d25ea1c295938aad7e86a898e636c5897be5565aa76d61683f9bb2131"
},
"downloads": -1,
"filename": "vectara_agentic-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "5b7c324404446285e5727b66ec16cbec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 50261,
"upload_time": "2025-02-21T07:51:39",
"upload_time_iso_8601": "2025-02-21T07:51:39.723485Z",
"url": "https://files.pythonhosted.org/packages/22/0e/206f7201b6a44e8ff228fb775c21c3752118ebb792fdcb11cd71c75fd98f/vectara_agentic-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-21 07:51:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "vectara",
"github_project": "py-vectara-agentic",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "llama-index",
"specs": [
[
"==",
"0.12.11"
]
]
},
{
"name": "llama-index-indices-managed-vectara",
"specs": [
[
"==",
"0.4.0"
]
]
},
{
"name": "llama-index-agent-llm-compiler",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-agent-lats",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-agent-openai",
"specs": [
[
"==",
"0.4.3"
]
]
},
{
"name": "llama-index-llms-openai",
"specs": [
[
"==",
"0.3.18"
]
]
},
{
"name": "llama-index-llms-anthropic",
"specs": [
[
"==",
"0.6.4"
]
]
},
{
"name": "llama-index-llms-together",
"specs": [
[
"==",
"0.3.1"
]
]
},
{
"name": "llama-index-llms-groq",
"specs": [
[
"==",
"0.3.1"
]
]
},
{
"name": "llama-index-llms-fireworks",
"specs": [
[
"==",
"0.3.1"
]
]
},
{
"name": "llama-index-llms-cohere",
"specs": [
[
"==",
"0.4.0"
]
]
},
{
"name": "llama-index-llms-gemini",
"specs": [
[
"==",
"0.4.4"
]
]
},
{
"name": "llama-index-llms-bedrock",
"specs": [
[
"==",
"0.3.3"
]
]
},
{
"name": "llama-index-tools-yahoo-finance",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-arxiv",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-database",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-google",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-tavily_research",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-neo4j",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-graph-stores-kuzu",
"specs": [
[
"==",
"0.6.0"
]
]
},
{
"name": "llama-index-tools-slack",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "llama-index-tools-exa",
"specs": [
[
"==",
"0.3.0"
]
]
},
{
"name": "tavily-python",
"specs": [
[
"==",
"0.5.0"
]
]
},
{
"name": "exa-py",
"specs": [
[
"==",
"1.8.5"
]
]
},
{
"name": "yahoo-finance",
"specs": [
[
"==",
"1.4.0"
]
]
},
{
"name": "openinference-instrumentation-llama-index",
"specs": [
[
"==",
"3.1.4"
]
]
},
{
"name": "opentelemetry-proto",
"specs": [
[
"==",
"1.26.0"
]
]
},
{
"name": "arize-phoenix",
"specs": [
[
"==",
"7.11.0"
]
]
},
{
"name": "arize-phoenix-otel",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"4.25.5"
]
]
},
{
"name": "tokenizers",
"specs": [
[
">=",
"0.20"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.10.3"
]
]
},
{
"name": "retrying",
"specs": [
[
"==",
"1.3.4"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "tiktoken",
"specs": [
[
"==",
"0.8.0"
]
]
},
{
"name": "cloudpickle",
"specs": [
[
">=",
"3.1.1"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.27.2"
]
]
}
],
"lcname": "vectara-agentic"
}
