| Name | gpt-json JSON |
| Version |
0.5.1
JSON |
| download |
| home_page | None |
| Summary | Structured and typehinted GPT responses in Python. |
| upload_time | 2024-07-24 22:42:30 |
| maintainer | None |
| docs_url | None |
| author | Pierce Freeman |
| requires_python | <4.0,>=3.11 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# gpt-json
`gpt-json` is a wrapper around GPT that allows for declarative definition of expected output format. Set up a schema, write a prompt, and get results back as beautiful typehinted objects.
This library introduces the following features:
- 🏗️ Pydantic schema definitions for type casting and validations
- 🧵 Templating of prompts to allow for dynamic content
- 🔎 Supports Vision API, Function Calling, and standard chat prompts
- 🚕 Lightweight transformations of the output to fix broken json
- ♻️ Retry logic for the most common API failures
- 📋 Predict single-objects and lists of objects
- ✈️ Lightweight dependencies: only OpenAI, pydantic, and backoff
## Getting Started
```bash
pip install gpt-json
```
Here's how to use it to generate a schema for simple tasks:
```python
import asyncio
from gpt_json import GPTJSON, GPTMessage, GPTMessageRole
from pydantic import BaseModel
class SentimentSchema(BaseModel):
sentiment: str
SYSTEM_PROMPT = """
Analyze the sentiment of the given text.
Respond with the following JSON schema:
{json_schema}
"""
async def runner():
gpt_json = GPTJSON[SentimentSchema](API_KEY)
payload = await gpt_json.run(
messages=[
GPTMessage(
role=GPTMessageRole.SYSTEM,
content=SYSTEM_PROMPT,
),
GPTMessage(
role=GPTMessageRole.USER,
content="Text: I love this product. It's the best thing ever!",
)
]
)
print(payload.response)
print(f"Detected sentiment: {payload.response.sentiment}")
asyncio.run(runner())
```
```bash
sentiment='positive'
Detected sentiment: positive
```
The `json_schema` is a special keyword that will be replaced with the schema definition at runtime. You should always include this in your payload to ensure the model knows how to format results. However, you can play around with _where_ to include this schema definition; in the system prompt, in the user prompt, at the beginning, or at the end.
You can either typehint the model to return a BaseSchema back, or to provide a list of multiple BaseSchema. Both of these work:
```python
from gpt_json.gpt import GPTJSON, ListResponse
gpt_json_single = GPTJSON[SentimentSchema](API_KEY)
gpt_json_multiple = GPTJSON[ListResponse[SentimentSchema]](API_KEY)
```
If you want to get more specific about how you expect the model to populate a field, add hints about the value through the "description" field. This helps the model understand what you're looking for, and will help it generate better results.
```python
from pydantic import BaseModel, Field
class SentimentSchema(BaseModel):
sentiment: int = Field(description="Either -1, 0, or 1.")
```
```
sentiment=1
Detected sentiment: 1
```
## Prompt Variables
In addition to the `json_schema` template keyword, you can also add arbitrary variables into your messages. This allows you to more easily insert user generated content or dynamically generate prompts based on the results of previous messages.
```python
class QuoteSchema(BaseModel):
quotes: list[str]
SYSTEM_PROMPT = """
Generate fictitious quotes that are {sentiment}.
{json_schema}
"""
gpt_json = GPTJSON[QuoteSchema](API_KEY)
response = await gpt_json.run(
messages=[
GPTMessage(
role=GPTMessageRole.SYSTEM,
content=SYSTEM_PROMPT,
),
],
format_variables={"sentiment": "happy"},
)
```
When calling the `.run()` function you can pass it the values that should be filled in this template. This also extends to field descriptions as well, so you can specify custom behavior on a per-field basis.
```python
class QuoteSchema(BaseModel):
quotes: list[str] = Field(description="Max quantity {max_items}.")
SYSTEM_PROMPT = """
Generate fictitious quotes that are {sentiment}.
{json_schema}
"""
gpt_json = GPTJSON[QuoteSchema](API_KEY)
response = await gpt_json.run(
messages=[
GPTMessage(
role=GPTMessageRole.SYSTEM,
content=SYSTEM_PROMPT,
),
],
format_variables={"sentiment": "happy", "max_items": 5},
)
```
## Function Calls
`gpt-3.5-turbo-0613` and `gpt-4-0613` were fine-tuned to support a specific syntax for function calls. We support this syntax in `gpt-json` as well. Here's an example of how to use it:
```python
class UnitType(Enum):
CELSIUS = "celsius"
FAHRENHEIT = "fahrenheit"
class GetCurrentWeatherRequest(BaseModel):
location: str = Field(description="The city and state, e.g. San Francisco, CA")
unit: UnitType | None = None
class DataPayload(BaseModel):
data: str
def get_current_weather(request: GetCurrentWeatherRequest):
"""
Get the current weather in a given location
"""
weather_info = {
"location": request.location,
"temperature": "72",
"unit": request.unit,
"forecast": ["sunny", "windy"],
}
return json_dumps(weather_info)
async def runner():
gpt_json = GPTJSON[DataPayload](API_KEY, functions=[get_current_weather])
response = await gpt_json.run(
messages=[
GPTMessage(
role=GPTMessageRole.USER,
content="What's the weather like in Boston, in F?",
),
],
)
assert response.function_call == get_current_weather
assert response.function_arg == GetCurrentWeatherRequest(
location="Boston", unit=UnitType.FAHRENHEIT
)
```
The response provides the original function alongside a formatted Pydantic object. If users want to execute the function, they can run response.function_call(response.function_arg). We will parse the get_current_weather function and the GetCurrentWeatherRequest parameter into the format that GPT expects, so it is more likely to return you a correct function execution.
GPT makes no guarantees about the validity of the returned functions. They could hallucinate a function name or the function signature. To address these cases, the run() function may now throw two new exceptions:
`InvalidFunctionResponse` - The function name is incorrect.
`InvalidFunctionParameters` - The function name is correct, but doesn't match the input schema that was provided.
## Other Configurations
The `GPTJSON` class supports other configuration parameters at initialization.
| Parameter | Type | Description |
| --------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| model | GPTModelVersion \| str | (default: GPTModelVersion.GPT_4) - For convenience we provide the currently supported GPT model versions in the `GPTModelVersion` enum. You can also pass a string value if you want to use another more specific architecture. |
| auto_trim | bool | (default: False) - If your input prompt is too long, perhaps because of dynamic injected content, will automatically truncate the text to create enough room for the model's response. |
| auto_trim_response_overhead | int | (default: 0) - If you're using auto_trim, configures the max amount of tokens to allow in the model's response. |
| \*\*kwargs | Any | Any other parameters you want to pass to the underlying `GPT` class, will just be a passthrough. |
## Transformations
GPT (especially GPT-4) is relatively good at formatting responses at JSON, but it's not perfect. Some of the more common issues are:
- _Response truncation_: Since GPT is not internally aware of its response length limit, JSON payloads will sometimes exhaust the available token space. This results in a broken JSON payload where much of the data is valid but the JSON object is not closed, which is not valid syntax. There are many cases where this behavior is actually okay for production applications - for instance, if you list 100 generated strings, it's sometimes okay for you to take the 70 that actually rendered. In this case, `gpt-json` will attempt to fix the truncated payload by recreating the JSON object and closing it.
- _Boolean variables_: GPT will sometimes confuse valid JSON boolean values with the boolean tokens that are used in other languages. The most common is generating `True` instead of `true`. `gpt-json` will attempt to fix these values.
When calling `gpt_json.run()`, we return a tuple of values:
```python
payload = await gpt_json.run(...)
print(transformations.fix_transforms)
```
```bash
FixTransforms(fixed_truncation=True, fixed_bools=False)
```
The first object is your generated Pydantic model. The second object is our correction storage object `FixTransforms`. This dataclass contains flags for each of the supported transformation cases that are sketched out above. This allows you to determine whether the response was explicitly parsed from the GPT JSON, or was passed through some middlelayers to get a correct output. From there you can accept or reject the response based on your own business logic.
_Where you can help_: There are certainly more areas of common (and not-so-common failures). If you see these, please add a test case to the unit tests. If you can write a handler to help solve the general case, please do so. Otherwise flag it as a `pytest.xfail` and we'll add it to the backlog.
## Testing
We use poetry for package management. To run the bundled tests, clone the package from github.
```bash
poetry install
poetry run pytest .
```
Our focus is on making unit tests as robust as possible. The variability with GPT should be in its language model, not in its JSON behavior! This is still certainly a work in progress. If you see an edge case that isn't covered, please add it to the test suite.
## Comparison to Other Libraries
A non-exhaustive list of other libraries that address the same problem. None of them were fully compatible with my deployment (hence this library), but check them out:
[jsonformer](https://github.com/1rgs/jsonformer) - Works with any Huggingface model, whereas `gpt-json` is specifically tailored towards the GPT-X family. GPT doesn't output logit probabilities or allow fixed decoder templating so the same approach can't apply.
## Formatting
We use black and mypy for formatting. You can set up a pre-commit git hook to do this automatically via the `./lint.sh` helper file.
If you perform a bulk reformatting to the codebase, you should add your most recent commit to the `.git-blame-ignore-revs` file and run:
```
git config blame.ignoreRevsFile .git-blame-ignore-revs
```
Raw data
{
"_id": null,
"home_page": null,
"name": "gpt-json",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": null,
"author": "Pierce Freeman",
"author_email": "pierce@freeman.vc",
"download_url": "https://files.pythonhosted.org/packages/4c/cc/3bedda9c11cad93f817afa2cc803d8a59ac3231310787ddce8dc703c3cce/gpt_json-0.5.1.tar.gz",
"platform": null,
"description": "# gpt-json\n\n`gpt-json` is a wrapper around GPT that allows for declarative definition of expected output format. Set up a schema, write a prompt, and get results back as beautiful typehinted objects.\n\nThis library introduces the following features:\n\n- \ud83c\udfd7\ufe0f Pydantic schema definitions for type casting and validations\n- \ud83e\uddf5 Templating of prompts to allow for dynamic content\n- \ud83d\udd0e Supports Vision API, Function Calling, and standard chat prompts\n- \ud83d\ude95 Lightweight transformations of the output to fix broken json\n- \u267b\ufe0f Retry logic for the most common API failures\n- \ud83d\udccb Predict single-objects and lists of objects\n- \u2708\ufe0f Lightweight dependencies: only OpenAI, pydantic, and backoff\n\n## Getting Started\n\n```bash\npip install gpt-json\n```\n\nHere's how to use it to generate a schema for simple tasks:\n\n```python\nimport asyncio\n\nfrom gpt_json import GPTJSON, GPTMessage, GPTMessageRole\nfrom pydantic import BaseModel\n\nclass SentimentSchema(BaseModel):\n sentiment: str\n\nSYSTEM_PROMPT = \"\"\"\nAnalyze the sentiment of the given text.\n\nRespond with the following JSON schema:\n\n{json_schema}\n\"\"\"\n\nasync def runner():\n gpt_json = GPTJSON[SentimentSchema](API_KEY)\n payload = await gpt_json.run(\n messages=[\n GPTMessage(\n role=GPTMessageRole.SYSTEM,\n content=SYSTEM_PROMPT,\n ),\n GPTMessage(\n role=GPTMessageRole.USER,\n content=\"Text: I love this product. It's the best thing ever!\",\n )\n ]\n )\n print(payload.response)\n print(f\"Detected sentiment: {payload.response.sentiment}\")\n\nasyncio.run(runner())\n```\n\n```bash\nsentiment='positive'\nDetected sentiment: positive\n```\n\nThe `json_schema` is a special keyword that will be replaced with the schema definition at runtime. You should always include this in your payload to ensure the model knows how to format results. However, you can play around with _where_ to include this schema definition; in the system prompt, in the user prompt, at the beginning, or at the end.\n\nYou can either typehint the model to return a BaseSchema back, or to provide a list of multiple BaseSchema. Both of these work:\n\n```python\nfrom gpt_json.gpt import GPTJSON, ListResponse\n\ngpt_json_single = GPTJSON[SentimentSchema](API_KEY)\ngpt_json_multiple = GPTJSON[ListResponse[SentimentSchema]](API_KEY)\n```\n\nIf you want to get more specific about how you expect the model to populate a field, add hints about the value through the \"description\" field. This helps the model understand what you're looking for, and will help it generate better results.\n\n```python\nfrom pydantic import BaseModel, Field\n\nclass SentimentSchema(BaseModel):\n sentiment: int = Field(description=\"Either -1, 0, or 1.\")\n```\n\n```\nsentiment=1\nDetected sentiment: 1\n```\n\n## Prompt Variables\n\nIn addition to the `json_schema` template keyword, you can also add arbitrary variables into your messages. This allows you to more easily insert user generated content or dynamically generate prompts based on the results of previous messages.\n\n```python\nclass QuoteSchema(BaseModel):\n quotes: list[str]\n\nSYSTEM_PROMPT = \"\"\"\nGenerate fictitious quotes that are {sentiment}.\n\n{json_schema}\n\"\"\"\n\ngpt_json = GPTJSON[QuoteSchema](API_KEY)\nresponse = await gpt_json.run(\n messages=[\n GPTMessage(\n role=GPTMessageRole.SYSTEM,\n content=SYSTEM_PROMPT,\n ),\n ],\n format_variables={\"sentiment\": \"happy\"},\n)\n```\n\nWhen calling the `.run()` function you can pass it the values that should be filled in this template. This also extends to field descriptions as well, so you can specify custom behavior on a per-field basis.\n\n```python\nclass QuoteSchema(BaseModel):\n quotes: list[str] = Field(description=\"Max quantity {max_items}.\")\n\nSYSTEM_PROMPT = \"\"\"\nGenerate fictitious quotes that are {sentiment}.\n\n{json_schema}\n\"\"\"\n\ngpt_json = GPTJSON[QuoteSchema](API_KEY)\nresponse = await gpt_json.run(\n messages=[\n GPTMessage(\n role=GPTMessageRole.SYSTEM,\n content=SYSTEM_PROMPT,\n ),\n ],\n format_variables={\"sentiment\": \"happy\", \"max_items\": 5},\n)\n```\n\n## Function Calls\n\n`gpt-3.5-turbo-0613` and `gpt-4-0613` were fine-tuned to support a specific syntax for function calls. We support this syntax in `gpt-json` as well. Here's an example of how to use it:\n\n```python\nclass UnitType(Enum):\n CELSIUS = \"celsius\"\n FAHRENHEIT = \"fahrenheit\"\n\n\nclass GetCurrentWeatherRequest(BaseModel):\n location: str = Field(description=\"The city and state, e.g. San Francisco, CA\")\n unit: UnitType | None = None\n\n\nclass DataPayload(BaseModel):\n data: str\n\n\ndef get_current_weather(request: GetCurrentWeatherRequest):\n \"\"\"\n Get the current weather in a given location\n \"\"\"\n weather_info = {\n \"location\": request.location,\n \"temperature\": \"72\",\n \"unit\": request.unit,\n \"forecast\": [\"sunny\", \"windy\"],\n }\n return json_dumps(weather_info)\n\n\nasync def runner():\n gpt_json = GPTJSON[DataPayload](API_KEY, functions=[get_current_weather])\n response = await gpt_json.run(\n messages=[\n GPTMessage(\n role=GPTMessageRole.USER,\n content=\"What's the weather like in Boston, in F?\",\n ),\n ],\n )\n\n assert response.function_call == get_current_weather\n assert response.function_arg == GetCurrentWeatherRequest(\n location=\"Boston\", unit=UnitType.FAHRENHEIT\n )\n```\n\nThe response provides the original function alongside a formatted Pydantic object. If users want to execute the function, they can run response.function_call(response.function_arg). We will parse the get_current_weather function and the GetCurrentWeatherRequest parameter into the format that GPT expects, so it is more likely to return you a correct function execution.\n\nGPT makes no guarantees about the validity of the returned functions. They could hallucinate a function name or the function signature. To address these cases, the run() function may now throw two new exceptions:\n\n`InvalidFunctionResponse` - The function name is incorrect.\n`InvalidFunctionParameters` - The function name is correct, but doesn't match the input schema that was provided.\n\n## Other Configurations\n\nThe `GPTJSON` class supports other configuration parameters at initialization.\n\n| Parameter | Type | Description |\n| --------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| model | GPTModelVersion \\| str | (default: GPTModelVersion.GPT_4) - For convenience we provide the currently supported GPT model versions in the `GPTModelVersion` enum. You can also pass a string value if you want to use another more specific architecture. |\n| auto_trim | bool | (default: False) - If your input prompt is too long, perhaps because of dynamic injected content, will automatically truncate the text to create enough room for the model's response. |\n| auto_trim_response_overhead | int | (default: 0) - If you're using auto_trim, configures the max amount of tokens to allow in the model's response. |\n| \\*\\*kwargs | Any | Any other parameters you want to pass to the underlying `GPT` class, will just be a passthrough. |\n\n## Transformations\n\nGPT (especially GPT-4) is relatively good at formatting responses at JSON, but it's not perfect. Some of the more common issues are:\n\n- _Response truncation_: Since GPT is not internally aware of its response length limit, JSON payloads will sometimes exhaust the available token space. This results in a broken JSON payload where much of the data is valid but the JSON object is not closed, which is not valid syntax. There are many cases where this behavior is actually okay for production applications - for instance, if you list 100 generated strings, it's sometimes okay for you to take the 70 that actually rendered. In this case, `gpt-json` will attempt to fix the truncated payload by recreating the JSON object and closing it.\n- _Boolean variables_: GPT will sometimes confuse valid JSON boolean values with the boolean tokens that are used in other languages. The most common is generating `True` instead of `true`. `gpt-json` will attempt to fix these values.\n\nWhen calling `gpt_json.run()`, we return a tuple of values:\n\n```python\npayload = await gpt_json.run(...)\n\nprint(transformations.fix_transforms)\n```\n\n```bash\nFixTransforms(fixed_truncation=True, fixed_bools=False)\n```\n\nThe first object is your generated Pydantic model. The second object is our correction storage object `FixTransforms`. This dataclass contains flags for each of the supported transformation cases that are sketched out above. This allows you to determine whether the response was explicitly parsed from the GPT JSON, or was passed through some middlelayers to get a correct output. From there you can accept or reject the response based on your own business logic.\n\n_Where you can help_: There are certainly more areas of common (and not-so-common failures). If you see these, please add a test case to the unit tests. If you can write a handler to help solve the general case, please do so. Otherwise flag it as a `pytest.xfail` and we'll add it to the backlog.\n\n## Testing\n\nWe use poetry for package management. To run the bundled tests, clone the package from github.\n\n```bash\npoetry install\npoetry run pytest .\n```\n\nOur focus is on making unit tests as robust as possible. The variability with GPT should be in its language model, not in its JSON behavior! This is still certainly a work in progress. If you see an edge case that isn't covered, please add it to the test suite.\n\n## Comparison to Other Libraries\n\nA non-exhaustive list of other libraries that address the same problem. None of them were fully compatible with my deployment (hence this library), but check them out:\n\n[jsonformer](https://github.com/1rgs/jsonformer) - Works with any Huggingface model, whereas `gpt-json` is specifically tailored towards the GPT-X family. GPT doesn't output logit probabilities or allow fixed decoder templating so the same approach can't apply.\n\n## Formatting\n\nWe use black and mypy for formatting. You can set up a pre-commit git hook to do this automatically via the `./lint.sh` helper file.\n\nIf you perform a bulk reformatting to the codebase, you should add your most recent commit to the `.git-blame-ignore-revs` file and run:\n\n```\ngit config blame.ignoreRevsFile .git-blame-ignore-revs\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Structured and typehinted GPT responses in Python.",
"version": "0.5.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a8e0ae420c146082dc4ba7193c776933ef895d85829e8cd3745eae74daf6870d",
"md5": "9d85c9f943f9f7403c5f5dc8b3812351",
"sha256": "dd0d36a727ce7e86ba8cc68a753b97b07fb6b14679cd4a572de65a37d3b45709"
},
"downloads": -1,
"filename": "gpt_json-0.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9d85c9f943f9f7403c5f5dc8b3812351",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 39006,
"upload_time": "2024-07-24T22:42:28",
"upload_time_iso_8601": "2024-07-24T22:42:28.841157Z",
"url": "https://files.pythonhosted.org/packages/a8/e0/ae420c146082dc4ba7193c776933ef895d85829e8cd3745eae74daf6870d/gpt_json-0.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4ccc3bedda9c11cad93f817afa2cc803d8a59ac3231310787ddce8dc703c3cce",
"md5": "da78e2d50211f23c8bf85dbb8990db59",
"sha256": "d7c02da83ac0c61f7f235bc6208452d8515a9363b9d9dc6475f9418926975bb0"
},
"downloads": -1,
"filename": "gpt_json-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "da78e2d50211f23c8bf85dbb8990db59",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 33960,
"upload_time": "2024-07-24T22:42:30",
"upload_time_iso_8601": "2024-07-24T22:42:30.330932Z",
"url": "https://files.pythonhosted.org/packages/4c/cc/3bedda9c11cad93f817afa2cc803d8a59ac3231310787ddce8dc703c3cce/gpt_json-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-24 22:42:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "gpt-json"
}
