<p align="center">
<h1 align="center">
<img src="assets/genie_worksheets_circle.png" width=100px>
<br>
<b>GenieWorksheets</b>
<br>
<a href="https://arxiv.org/abs/2407.05674">
<img src="https://img.shields.io/badge/cs.CL-2407.05674-b31b1b"
alt="arXiv">
</a>
<a href="https://ws.genie.stanford.edu/">
<img src="https://img.shields.io/badge/website-genie.stanford.edu-blue"
alt="Website">
</a>
<a href="https://ws.genie.stanford.edu/docs/">
<img src="https://img.shields.io/badge/docs-genie.stanford.edu-blue"
alt="Docs">
</a>
</h1>
</p>
<p align="center">
Framework for creating reliable conversational agents
</p>
Genie is a programmable framework for creating task-oriented conversational
agents that are designed to handle complex user interactions and knowledge
access. Unlike LLMs, Genie provides reliable grounded responses, with
controllable agent policies through its expressive specification, Genie
Worksheet. In contrast to dialog trees, it is resilient to diverse user queries,
helpful with knowledge sources, and offers ease of programming policies through
its declarative paradigm.
[Research Preprint](https://arxiv.org/abs/2407.05674): To be presented at ACL 2025
<img src="assets/banner.jpg">
## Installation
To install Genie, we recommend using uv ([UV installation guide](https://github.com/astral-sh/uv?tab=readme-ov-file#installation))
```bash
git clone https://github.com/stanford-oval/genie-worksheets.git
cd worksheets
uv venv
source venv/bin/activate
uv sync
```
## Creating Agents
Example agents are present in `experiments/agents/` directory. You can use them
as a reference to create your own agents.
### Load the model configuration
```python
from worksheets import Config
import os
# Define path to the prompts
current_dir = os.path.dirname(os.path.realpath(__file__))
prompt_dir = os.path.join(current_dir, "prompts")
# Load config from YAML file
config = Config.load_from_yaml(os.path.join(current_dir, "config.yaml"))
```
You can also define the configuration programmatically:
```python
from worksheets import Config, AzureModelConfig
import os
config = Config(
semantic_parser=AzureModelConfig(
model_name="azure/gpt-4o",
api_key=os.getenv("AZURE_OPENAI_WS_KEY"),
api_version=os.getenv("AZURE_WS_API_VERSION"),
azure_endpoint=os.getenv("AZURE_WS_ENDPOINT"),
),
response_generator=AzureModelConfig(
model_name="azure/gpt-4o",
api_key=os.getenv("AZURE_OPENAI_WS_KEY"),
api_version=os.getenv("AZURE_WS_API_VERSION"),
azure_endpoint=os.getenv("AZURE_WS_ENDPOINT"),
),
knowledge_parser=AzureModelConfig(
model_name="gpt-4o",
api_key=os.getenv("AZURE_OPENAI_WS_KEY"),
api_version=os.getenv("AZURE_WS_API_VERSION"),
azure_endpoint=os.getenv("AZURE_WS_ENDPOINT"),
),
knowledge_base=AzureModelConfig(
model_name="azure/gpt-4o",
api_key=os.getenv("AZURE_OPENAI_WS_KEY"),
api_version=os.getenv("AZURE_WS_API_VERSION"),
azure_endpoint=os.getenv("AZURE_WS_ENDPOINT"),
),
prompt_dir=prompt_dir,
)
```
### Define your API functions
```python
from worksheets.agent.config import agent_api
from worksheets.core.worksheet import get_genie_fields_from_ws
from uuid import uuid4
@agent_api("course_detail_to_individual_params", "Get course details")
def course_detail_to_individual_params(course_detail):
if course_detail.value is None:
return {}
course_detail = course_detail.value
course_detail = {}
for field in get_genie_fields_from_ws(course_detail):
course_detail[field.name] = field.value
return course_detail
@agent_api("courses_to_take_oval", "Final API to enroll into a course")
def courses_to_take_oval(**kwargs):
return {"success": True, "transaction_id": uuid4()}
@agent_api("is_course_full", "Check if a course is full")
def is_course_full(course_id, **kwargs):
# Implementation here
return False
```
### Define your starting prompt
You can load your starting prompt from a template file:
```python
from worksheets.agent.builder import TemplateLoader
starting_prompt = TemplateLoader.load(
os.path.join(current_dir, "starting_prompt.md"), format="jinja2"
)
```
Or define it inline:
```python
starting_prompt = """Hello! I'm the Course Enrollment Assistant. I can help you with:
- Selecting a course: just say find me programming courses
- Enrolling into a course.
- Asking me any question related to courses and their requirement criteria.
How can I help you today?"""
```
### Define the Agent
```python
from worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser
agent = (
AgentBuilder(
name="Course Enrollment Assistant",
description="You are a course enrollment assistant. You can help students with course selection and enrollment.",
starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,
)
.with_knowledge_base(
SUQLKnowledgeBase,
tables_with_primary_keys={
"courses": "course_id",
"ratings": "rating_id",
"offerings": "course_id",
"programs": "program_id",
},
database_name="course_assistant",
embedding_server_address="http://127.0.0.1:8509",
source_file_mapping={
"course_assistant_general_info.txt": os.path.join(
current_dir, "course_assistant_general_info.txt"
)
},
postprocessing_fn=postprocess_suql,
result_postprocessing_fn=None,
db_username="select_user",
db_password="select_user",
)
.with_parser(
SUQLReActParser,
example_path=os.path.join(current_dir, "examples.txt"),
instruction_path=os.path.join(current_dir, "instructions.txt"),
table_schema_path=os.path.join(current_dir, "table_schema.txt"),
)
.with_gsheet_specification("YOUR_SPREADSHEET_ID_HERE")
.build(config)
)
```
### Load the agent from a CSV file
You can also load the agent from a CSV file. Instead of using the `with_gsheet_specification` method, you can use the `with_csv_specification` method.
```python
from worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser
import os
agent = (
AgentBuilder(
name="Course Enrollment Assistant",
description="You are a course enrollment assistant. You can help students with course selection and enrollment.",
starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,
)
.with_csv_specification(os.path.join(current_dir, "course_enrollment.csv"))
.build(config)
)
```
### Load the agent from a JSON file
You can also load the agent from a JSON file. Instead of using the `with_gsheet_specification` method, you can use the `with_json_specification` method.
```python
from worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser
import os
agent = (
AgentBuilder(
name="Course Enrollment Assistant",
description="You are a course enrollment assistant. You can help students with course selection and enrollment.",
starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,
)
.with_json_specification(os.path.join(current_dir, "course_enrollment.json"))
.build(config)
)
```
A sample JSON file is present in `experiments/domain_agents/course_enroll/course_enrollment.json`.
### Run the conversation loop
```python
import asyncio
from worksheets import conversation_loop
if __name__ == "__main__":
# Run the conversation loop in the terminal
asyncio.run(conversation_loop(agent, "output_state_path.json", debug=True))
```
### Add prompts
For each agent you need to create prompts for:
- Semantic parsing: `semantic_parsing.prompt`
- Response generation: `response_generator.prompt`
Place these prompts in the prompt directory that you specify while creating the
agent.
You can copy basic annotated prompts from `experiments/sample_prompts/`
directory. Make change where we have `TODO`. You need two provide a few
guidelines in the prompt that will help the LLM to perform better and some
examples. Please `experiments/domain_agents/course_enroll/prompts/` for inspiration!
### Spreadsheet Specification
To create a new agent, you should have a Google Service Account and create a new spreadsheet.
You can follow the instructions [here](https://cloud.google.com/iam/docs/service-account-overview) to create a Google Service Account.
Share the created spreadsheet with the service account email.
You should save the service_account key as `service_account.json` in the `worksheets/` directory.
Here is a starter worksheet that you can use for your reference: [Starter Worksheet](https://docs.google.com/spreadsheets/d/1ST1ixBogjEEzEhMeb-kVyf-JxGRMjtlRR6z4G2sjyb4/edit?usp=sharing)
Here is a sample spreadsheet for a restaurant agent: [Restaurant Agent](https://docs.google.com/spreadsheets/d/1FXg5VFrdxQlUyld3QmKKL9BN1lLIhAtQTJjCHyNOU_Y/edit?usp=sharing)
Please note that we only use the specification defined in the first sheet of the spreadsheet.
## LLM Config
You should modify `llm_config.example.yaml` and save it as `llm_config.yaml` from the directory where you run the agent.
### Running the Agent (Web Interface)
Create a folder `frontend/` under `experiments/agents/<agent_name>` and create a `app_*` file.
You can run the agent in a web interface by running:
**NOTE:** You should run the agent in the `frontend` directory to preserve the frontend assets.
For restaurant agent:
```bash
cd experiments/domain_agents/yelpbot/frontend/
chainlit run app_restaurant.py --port 8800
```
## Cite our work
If you use Genie in your research or applications, please cite our work:
```
@article{genieworksheets,
title={Coding Reliable LLM-based Integrated Task and Knowledge Agents with GenieWorksheets},
author={Joshi, Harshit and Liu, Shicheng and Chen, James and Weigle, Robert and Lam, Monica S},
journal={arXiv preprint arXiv:2407.05674},
year={2024}
}
```
GenieWorksheets logo is designed with the help of DALL-E.
Raw data
{
"_id": null,
"home_page": null,
"name": "genie-worksheets",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": "Harshit Joshi <harshitj@stanford.edu>",
"keywords": "agents, conversational-ai, knowledge-agents, llm, worksheets",
"author": null,
"author_email": "Harshit Joshi <harshitj@stanford.edu>",
"download_url": "https://files.pythonhosted.org/packages/d7/82/7fac1bef1b3ceec78857d9af585593667e7b5144bda98027f2e0de005654/genie_worksheets-1.0.0b3.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <h1 align=\"center\">\n <img src=\"assets/genie_worksheets_circle.png\" width=100px>\n <br>\n <b>GenieWorksheets</b>\n <br>\n <a href=\"https://arxiv.org/abs/2407.05674\">\n <img src=\"https://img.shields.io/badge/cs.CL-2407.05674-b31b1b\"\n alt=\"arXiv\">\n </a>\n <a href=\"https://ws.genie.stanford.edu/\">\n <img src=\"https://img.shields.io/badge/website-genie.stanford.edu-blue\"\n alt=\"Website\">\n </a>\n <a href=\"https://ws.genie.stanford.edu/docs/\">\n <img src=\"https://img.shields.io/badge/docs-genie.stanford.edu-blue\"\n alt=\"Docs\">\n </a>\n </h1>\n</p>\n<p align=\"center\">\n Framework for creating reliable conversational agents\n</p>\n\n\nGenie is a programmable framework for creating task-oriented conversational\nagents that are designed to handle complex user interactions and knowledge\naccess. Unlike LLMs, Genie provides reliable grounded responses, with \ncontrollable agent policies through its expressive specification, Genie \nWorksheet. In contrast to dialog trees, it is resilient to diverse user queries,\nhelpful with knowledge sources, and offers ease of programming policies through\n its declarative paradigm.\n\n[Research Preprint](https://arxiv.org/abs/2407.05674): To be presented at ACL 2025\n\n<img src=\"assets/banner.jpg\">\n\n## Installation\n\nTo install Genie, we recommend using uv ([UV installation guide](https://github.com/astral-sh/uv?tab=readme-ov-file#installation))\n\n\n```bash\ngit clone https://github.com/stanford-oval/genie-worksheets.git\ncd worksheets\nuv venv\nsource venv/bin/activate\nuv sync\n```\n\n## Creating Agents\n\nExample agents are present in `experiments/agents/` directory. You can use them\nas a reference to create your own agents.\n\n### Load the model configuration\n\n```python\nfrom worksheets import Config\nimport os\n\n# Define path to the prompts\ncurrent_dir = os.path.dirname(os.path.realpath(__file__))\nprompt_dir = os.path.join(current_dir, \"prompts\")\n\n# Load config from YAML file\nconfig = Config.load_from_yaml(os.path.join(current_dir, \"config.yaml\"))\n```\n\nYou can also define the configuration programmatically:\n\n```python\nfrom worksheets import Config, AzureModelConfig\nimport os\n\nconfig = Config(\n semantic_parser=AzureModelConfig(\n model_name=\"azure/gpt-4o\",\n api_key=os.getenv(\"AZURE_OPENAI_WS_KEY\"),\n api_version=os.getenv(\"AZURE_WS_API_VERSION\"),\n azure_endpoint=os.getenv(\"AZURE_WS_ENDPOINT\"),\n ),\n response_generator=AzureModelConfig(\n model_name=\"azure/gpt-4o\",\n api_key=os.getenv(\"AZURE_OPENAI_WS_KEY\"),\n api_version=os.getenv(\"AZURE_WS_API_VERSION\"),\n azure_endpoint=os.getenv(\"AZURE_WS_ENDPOINT\"),\n ),\n knowledge_parser=AzureModelConfig(\n model_name=\"gpt-4o\",\n api_key=os.getenv(\"AZURE_OPENAI_WS_KEY\"),\n api_version=os.getenv(\"AZURE_WS_API_VERSION\"),\n azure_endpoint=os.getenv(\"AZURE_WS_ENDPOINT\"),\n ),\n knowledge_base=AzureModelConfig(\n model_name=\"azure/gpt-4o\",\n api_key=os.getenv(\"AZURE_OPENAI_WS_KEY\"),\n api_version=os.getenv(\"AZURE_WS_API_VERSION\"),\n azure_endpoint=os.getenv(\"AZURE_WS_ENDPOINT\"),\n ),\n prompt_dir=prompt_dir,\n)\n```\n\n### Define your API functions\n\n```python\nfrom worksheets.agent.config import agent_api\nfrom worksheets.core.worksheet import get_genie_fields_from_ws\nfrom uuid import uuid4\n\n@agent_api(\"course_detail_to_individual_params\", \"Get course details\")\ndef course_detail_to_individual_params(course_detail):\n if course_detail.value is None:\n return {}\n course_detail = course_detail.value\n course_detail = {}\n for field in get_genie_fields_from_ws(course_detail):\n course_detail[field.name] = field.value\n\n return course_detail\n\n@agent_api(\"courses_to_take_oval\", \"Final API to enroll into a course\")\ndef courses_to_take_oval(**kwargs):\n return {\"success\": True, \"transaction_id\": uuid4()}\n\n@agent_api(\"is_course_full\", \"Check if a course is full\")\ndef is_course_full(course_id, **kwargs):\n # Implementation here\n return False\n```\n\n### Define your starting prompt\n\nYou can load your starting prompt from a template file:\n\n```python\nfrom worksheets.agent.builder import TemplateLoader\n\nstarting_prompt = TemplateLoader.load(\n os.path.join(current_dir, \"starting_prompt.md\"), format=\"jinja2\"\n)\n```\n\nOr define it inline:\n\n```python\nstarting_prompt = \"\"\"Hello! I'm the Course Enrollment Assistant. I can help you with:\n- Selecting a course: just say find me programming courses\n- Enrolling into a course. \n- Asking me any question related to courses and their requirement criteria.\n\nHow can I help you today?\"\"\"\n```\n\n### Define the Agent\n\n```python\nfrom worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser\n\nagent = (\n AgentBuilder(\n name=\"Course Enrollment Assistant\",\n description=\"You are a course enrollment assistant. You can help students with course selection and enrollment.\",\n starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,\n )\n .with_knowledge_base(\n SUQLKnowledgeBase,\n tables_with_primary_keys={\n \"courses\": \"course_id\",\n \"ratings\": \"rating_id\",\n \"offerings\": \"course_id\",\n \"programs\": \"program_id\",\n },\n database_name=\"course_assistant\",\n embedding_server_address=\"http://127.0.0.1:8509\",\n source_file_mapping={\n \"course_assistant_general_info.txt\": os.path.join(\n current_dir, \"course_assistant_general_info.txt\"\n )\n },\n postprocessing_fn=postprocess_suql,\n result_postprocessing_fn=None,\n db_username=\"select_user\",\n db_password=\"select_user\",\n )\n .with_parser(\n SUQLReActParser,\n example_path=os.path.join(current_dir, \"examples.txt\"),\n instruction_path=os.path.join(current_dir, \"instructions.txt\"),\n table_schema_path=os.path.join(current_dir, \"table_schema.txt\"),\n )\n .with_gsheet_specification(\"YOUR_SPREADSHEET_ID_HERE\")\n .build(config)\n)\n```\n\n### Load the agent from a CSV file\nYou can also load the agent from a CSV file. Instead of using the `with_gsheet_specification` method, you can use the `with_csv_specification` method.\n\n```python\nfrom worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser\nimport os\n\nagent = (\n AgentBuilder(\n name=\"Course Enrollment Assistant\",\n description=\"You are a course enrollment assistant. You can help students with course selection and enrollment.\",\n starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,\n )\n .with_csv_specification(os.path.join(current_dir, \"course_enrollment.csv\"))\n .build(config)\n)\n```\n\n### Load the agent from a JSON file\nYou can also load the agent from a JSON file. Instead of using the `with_gsheet_specification` method, you can use the `with_json_specification` method.\n\n```python\nfrom worksheets import AgentBuilder, SUQLKnowledgeBase, SUQLReActParser\nimport os\n\nagent = (\n AgentBuilder(\n name=\"Course Enrollment Assistant\",\n description=\"You are a course enrollment assistant. You can help students with course selection and enrollment.\",\n starting_prompt=starting_prompt.render() if hasattr(starting_prompt, 'render') else starting_prompt,\n )\n .with_json_specification(os.path.join(current_dir, \"course_enrollment.json\"))\n .build(config)\n)\n```\n\nA sample JSON file is present in `experiments/domain_agents/course_enroll/course_enrollment.json`.\n\n### Run the conversation loop\n\n```python\nimport asyncio\nfrom worksheets import conversation_loop\n\nif __name__ == \"__main__\":\n # Run the conversation loop in the terminal\n asyncio.run(conversation_loop(agent, \"output_state_path.json\", debug=True))\n```\n\n\n### Add prompts\nFor each agent you need to create prompts for:\n- Semantic parsing: `semantic_parsing.prompt`\n- Response generation: `response_generator.prompt`\n\nPlace these prompts in the prompt directory that you specify while creating the\nagent.\n\nYou can copy basic annotated prompts from `experiments/sample_prompts/` \ndirectory. Make change where we have `TODO`. You need two provide a few \nguidelines in the prompt that will help the LLM to perform better and some \nexamples. Please `experiments/domain_agents/course_enroll/prompts/` for inspiration!\n\n\n### Spreadsheet Specification\n\nTo create a new agent, you should have a Google Service Account and create a new spreadsheet. \nYou can follow the instructions [here](https://cloud.google.com/iam/docs/service-account-overview) to create a Google Service Account.\nShare the created spreadsheet with the service account email.\n\nYou should save the service_account key as `service_account.json` in the `worksheets/` directory.\n\nHere is a starter worksheet that you can use for your reference: [Starter Worksheet](https://docs.google.com/spreadsheets/d/1ST1ixBogjEEzEhMeb-kVyf-JxGRMjtlRR6z4G2sjyb4/edit?usp=sharing)\n\nHere is a sample spreadsheet for a restaurant agent: [Restaurant Agent](https://docs.google.com/spreadsheets/d/1FXg5VFrdxQlUyld3QmKKL9BN1lLIhAtQTJjCHyNOU_Y/edit?usp=sharing)\n\nPlease note that we only use the specification defined in the first sheet of the spreadsheet.\n\n## LLM Config\nYou should modify `llm_config.example.yaml` and save it as `llm_config.yaml` from the directory where you run the agent.\n\n### Running the Agent (Web Interface)\n\nCreate a folder `frontend/` under `experiments/agents/<agent_name>` and create a `app_*` file.\n\nYou can run the agent in a web interface by running:\n\n**NOTE:** You should run the agent in the `frontend` directory to preserve the frontend assets.\n\nFor restaurant agent:\n```bash\ncd experiments/domain_agents/yelpbot/frontend/\nchainlit run app_restaurant.py --port 8800\n```\n\n## Cite our work\n\nIf you use Genie in your research or applications, please cite our work:\n\n```\n@article{genieworksheets,\n title={Coding Reliable LLM-based Integrated Task and Knowledge Agents with GenieWorksheets},\n author={Joshi, Harshit and Liu, Shicheng and Chen, James and Weigle, Robert and Lam, Monica S},\n journal={arXiv preprint arXiv:2407.05674},\n year={2024}\n}\n```\n\nGenieWorksheets logo is designed with the help of DALL-E.",
"bugtrack_url": null,
"license": "MIT",
"summary": "Genie Worksheets: A high-level, expressive and declarative specification for creating knowledge agents",
"version": "1.0.0b3",
"project_urls": {
"Bug Tracker": "https://github.com/stanford-oval/genie-worksheets/issues",
"Documentation": "https://ws.genie.stanford.edu/docs/",
"Homepage": "https://ws.genie.stanford.edu/",
"Repository": "https://github.com/stanford-oval/genie-worksheets",
"Research Paper": "https://arxiv.org/abs/2407.05674"
},
"split_keywords": [
"agents",
" conversational-ai",
" knowledge-agents",
" llm",
" worksheets"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "73dc118453341077a9f7fbafb60fb2719bebffded70a8b283fc788a6d380bc63",
"md5": "dc12046286896769481252b8b886f811",
"sha256": "9e3c8aa0583b4fda2efb686b7fcf1d58898bb5c2082a66b6a07ac5dde224dbbb"
},
"downloads": -1,
"filename": "genie_worksheets-1.0.0b3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "dc12046286896769481252b8b886f811",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 104484,
"upload_time": "2025-07-20T05:55:45",
"upload_time_iso_8601": "2025-07-20T05:55:45.126839Z",
"url": "https://files.pythonhosted.org/packages/73/dc/118453341077a9f7fbafb60fb2719bebffded70a8b283fc788a6d380bc63/genie_worksheets-1.0.0b3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d7827fac1bef1b3ceec78857d9af585593667e7b5144bda98027f2e0de005654",
"md5": "0825777dc53e0f9357fc956db456f261",
"sha256": "c92e2404e2deb41931bf510698eb0c852f5b9912fb4ea4b1adce07b6fbde61b0"
},
"downloads": -1,
"filename": "genie_worksheets-1.0.0b3.tar.gz",
"has_sig": false,
"md5_digest": "0825777dc53e0f9357fc956db456f261",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 11655504,
"upload_time": "2025-07-20T05:55:46",
"upload_time_iso_8601": "2025-07-20T05:55:46.836754Z",
"url": "https://files.pythonhosted.org/packages/d7/82/7fac1bef1b3ceec78857d9af585593667e7b5144bda98027f2e0de005654/genie_worksheets-1.0.0b3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-20 05:55:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "stanford-oval",
"github_project": "genie-worksheets",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "genie-worksheets"
}
JSON
