| Name | groqcloud JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | |
| Summary | The official Python library for the groq API |
| upload_time | 2024-02-16 00:19:10 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Groq Python API library
[](https://pypi.org/project/groq/)
The Groq Python library provides convenient access to the Groq REST API from any Python 3.7+
application. The library includes type definitions for all request params and response fields,
and offers both synchronous and asynchronous clients powered by [httpx](https://github.com/encode/httpx).
## Documentation
The REST API documentation can be found [on console.groq.com](https://console.groq.com/docs). The full API of this library can be found in [api.md](api.md).
## Installation
```sh
pip install groq
```
## Usage
The full API of this library can be found in [api.md](api.md).
```python
import os
from groq import Groq
client = Groq(
# This is the default and can be omitted
api_key=os.environ.get("GROQ_API_KEY"),
)
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
}
],
model="mixtral-8x7b-32768",
)
print(chat_completion.choices_0.message.content)
```
While you can provide an `api_key` keyword argument,
we recommend using [python-dotenv](https://pypi.org/project/python-dotenv/)
to add `GROQ_API_KEY="My API Key"` to your `.env` file
so that your API Key is not stored in source control.
## Async usage
Simply import `AsyncGroq` instead of `Groq` and use `await` with each API call:
```python
import os
import asyncio
from groq import AsyncGroq
client = AsyncGroq(
# This is the default and can be omitted
api_key=os.environ.get("GROQ_API_KEY"),
)
async def main() -> None:
chat_completion = await client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
}
],
model="mixtral-8x7b-32768",
)
print(chat_completion.choices_0.message.content)
asyncio.run(main())
```
Functionality between the synchronous and asynchronous clients is otherwise identical.
## Using types
Nested request parameters are [TypedDicts](https://docs.python.org/3/library/typing.html#typing.TypedDict). Responses are [Pydantic models](https://docs.pydantic.dev), which provide helper methods for things like:
- Serializing back into JSON, `model.model_dump_json(indent=2, exclude_unset=True)`
- Converting to a dictionary, `model.model_dump(exclude_unset=True)`
Typed requests and responses provide autocomplete and documentation within your editor. If you would like to see type errors in VS Code to help catch bugs earlier, set `python.analysis.typeCheckingMode` to `basic`.
## Handling errors
When the library is unable to connect to the API (for example, due to network connection problems or a timeout), a subclass of `groq.APIConnectionError` is raised.
When the API returns a non-success status code (that is, 4xx or 5xx
response), a subclass of `groq.APIStatusError` is raised, containing `status_code` and `response` properties.
All errors inherit from `groq.APIError`.
```python
import groq
from groq import Groq
client = Groq()
try:
client.chat.completions.create(
messages=[
{
"role": "system",
"content": "You are a helpful assisstant.",
},
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
},
],
model="mixtral-8x7b-32768",
)
except groq.APIConnectionError as e:
print("The server could not be reached")
print(e.__cause__) # an underlying Exception, likely raised within httpx.
except groq.RateLimitError as e:
print("A 429 status code was received; we should back off a bit.")
except groq.APIStatusError as e:
print("Another non-200-range status code was received")
print(e.status_code)
print(e.response)
```
Error codes are as followed:
| Status Code | Error Type |
| ----------- | -------------------------- |
| 400 | `BadRequestError` |
| 401 | `AuthenticationError` |
| 403 | `PermissionDeniedError` |
| 404 | `NotFoundError` |
| 422 | `UnprocessableEntityError` |
| 429 | `RateLimitError` |
| >=500 | `InternalServerError` |
| N/A | `APIConnectionError` |
### Retries
Certain errors are automatically retried 2 times by default, with a short exponential backoff.
Connection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict,
429 Rate Limit, and >=500 Internal errors are all retried by default.
You can use the `max_retries` option to configure or disable retry settings:
```python
from groq import Groq
# Configure the default for all requests:
client = Groq(
# default is 2
max_retries=0,
)
# Or, configure per-request:
client.with_options(max_retries=5).chat.completions.create(
messages=[
{
"role": "system",
"content": "You are a helpful assisstant.",
},
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
},
],
model="mixtral-8x7b-32768",
)
```
### Timeouts
By default requests time out after 1 minute. You can configure this with a `timeout` option,
which accepts a float or an [`httpx.Timeout`](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) object:
```python
from groq import Groq
# Configure the default for all requests:
client = Groq(
# 20 seconds (default is 1 minute)
timeout=20.0,
)
# More granular control:
client = Groq(
timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
)
# Override per-request:
client.with_options(timeout=5 * 1000).chat.completions.create(
messages=[
{
"role": "system",
"content": "You are a helpful assisstant.",
},
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
},
],
model="mixtral-8x7b-32768",
)
```
On timeout, an `APITimeoutError` is thrown.
Note that requests that time out are [retried twice by default](#retries).
## Advanced
### Logging
We use the standard library [`logging`](https://docs.python.org/3/library/logging.html) module.
You can enable logging by setting the environment variable `GROQ_LOG` to `debug`.
```shell
$ export GROQ_LOG=debug
```
### How to tell whether `None` means `null` or missing
In an API response, a field may be explicitly `null`, or missing entirely; in either case, its value is `None` in this library. You can differentiate the two cases with `.model_fields_set`:
```py
if response.my_field is None:
if 'my_field' not in response.model_fields_set:
print('Got json like {}, without a "my_field" key present at all.')
else:
print('Got json like {"my_field": null}.')
```
### Accessing raw response data (e.g. headers)
The "raw" Response object can be accessed by prefixing `.with_raw_response.` to any HTTP method call, e.g.,
```py
from groq import Groq
client = Groq()
response = client.chat.completions.with_raw_response.create(
messages=[{
"role": "system",
"content": "You are a helpful assisstant.",
}, {
"role": "user",
"content": "Explain the importance of low latency LLMs",
}],
model="mixtral-8x7b-32768",
)
print(response.headers.get('X-My-Header'))
completion = response.parse() # get the object that `chat.completions.create()` would have returned
print(completion.id)
```
These methods return an [`APIResponse`](https://github.com/groq/groq-python/tree/stainless/src/groq/_response.py) object.
The async client returns an [`AsyncAPIResponse`](https://github.com/groq/groq-python/tree/stainless/src/groq/_response.py) with the same structure, the only difference being `await`able methods for reading the response content.
#### `.with_streaming_response`
The above interface eagerly reads the full response body when you make the request, which may not always be what you want.
To stream the response body, use `.with_streaming_response` instead, which requires a context manager and only reads the response body once you call `.read()`, `.text()`, `.json()`, `.iter_bytes()`, `.iter_text()`, `.iter_lines()` or `.parse()`. In the async client, these are async methods.
```python
with client.chat.completions.with_streaming_response.create(
messages=[
{
"role": "system",
"content": "You are a helpful assisstant.",
},
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
},
],
model="mixtral-8x7b-32768",
) as response:
print(response.headers.get("X-My-Header"))
for line in response.iter_lines():
print(line)
```
The context manager is required so that the response will reliably be closed.
### Configuring the HTTP client
You can directly override the [httpx client](https://www.python-httpx.org/api/#client) to customize it for your use case, including:
- Support for proxies
- Custom transports
- Additional [advanced](https://www.python-httpx.org/advanced/#client-instances) functionality
```python
import httpx
from groq import Groq
client = Groq(
# Or use the `GROQ_BASE_URL` env var
base_url="http://my.test.server.example.com:8083",
http_client=httpx.Client(
proxies="http://my.test.proxy.example.com",
transport=httpx.HTTPTransport(local_address="0.0.0.0"),
),
)
```
### Managing HTTP resources
By default the library closes underlying HTTP connections whenever the client is [garbage collected](https://docs.python.org/3/reference/datamodel.html#object.__del__). You can manually close the client using the `.close()` method if desired, or with a context manager that closes when exiting.
## Versioning
This package generally follows [SemVer](https://semver.org/spec/v2.0.0.html) conventions, though certain backwards-incompatible changes may be released as minor versions:
1. Changes that only affect static types, without breaking runtime behavior.
2. Changes to library internals which are technically public but not intended or documented for external use. _(Please open a GitHub issue to let us know if you are relying on such internals)_.
3. Changes that we do not expect to impact the vast majority of users in practice.
We take backwards-compatibility seriously and work hard to ensure you can rely on a smooth upgrade experience.
We are keen for your feedback; please open an [issue](https://www.github.com/groq/groq-python/issues) with questions, bugs, or suggestions.
## Requirements
Python 3.7 or higher.
Raw data
{
"_id": null,
"home_page": "",
"name": "groqcloud",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "",
"author_email": "Groq <grea@groq.com>",
"download_url": "https://files.pythonhosted.org/packages/42/36/7260c910137974591ec81e35a53e70a178f6cca89cd4b0effe896b0e3406/groqcloud-0.4.0.tar.gz",
"platform": null,
"description": "# Groq Python API library\n\n[](https://pypi.org/project/groq/)\n\nThe Groq Python library provides convenient access to the Groq REST API from any Python 3.7+\napplication. The library includes type definitions for all request params and response fields,\nand offers both synchronous and asynchronous clients powered by [httpx](https://github.com/encode/httpx).\n\n## Documentation\n\nThe REST API documentation can be found [on console.groq.com](https://console.groq.com/docs). The full API of this library can be found in [api.md](api.md).\n\n## Installation\n\n```sh\npip install groq\n```\n\n## Usage\n\nThe full API of this library can be found in [api.md](api.md).\n\n```python\nimport os\nfrom groq import Groq\n\nclient = Groq(\n # This is the default and can be omitted\n api_key=os.environ.get(\"GROQ_API_KEY\"),\n)\n\nchat_completion = client.chat.completions.create(\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n }\n ],\n model=\"mixtral-8x7b-32768\",\n)\nprint(chat_completion.choices_0.message.content)\n```\n\nWhile you can provide an `api_key` keyword argument,\nwe recommend using [python-dotenv](https://pypi.org/project/python-dotenv/)\nto add `GROQ_API_KEY=\"My API Key\"` to your `.env` file\nso that your API Key is not stored in source control.\n\n## Async usage\n\nSimply import `AsyncGroq` instead of `Groq` and use `await` with each API call:\n\n```python\nimport os\nimport asyncio\nfrom groq import AsyncGroq\n\nclient = AsyncGroq(\n # This is the default and can be omitted\n api_key=os.environ.get(\"GROQ_API_KEY\"),\n)\n\n\nasync def main() -> None:\n chat_completion = await client.chat.completions.create(\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n }\n ],\n model=\"mixtral-8x7b-32768\",\n )\n print(chat_completion.choices_0.message.content)\n\n\nasyncio.run(main())\n```\n\nFunctionality between the synchronous and asynchronous clients is otherwise identical.\n\n## Using types\n\nNested request parameters are [TypedDicts](https://docs.python.org/3/library/typing.html#typing.TypedDict). Responses are [Pydantic models](https://docs.pydantic.dev), which provide helper methods for things like:\n\n- Serializing back into JSON, `model.model_dump_json(indent=2, exclude_unset=True)`\n- Converting to a dictionary, `model.model_dump(exclude_unset=True)`\n\nTyped requests and responses provide autocomplete and documentation within your editor. If you would like to see type errors in VS Code to help catch bugs earlier, set `python.analysis.typeCheckingMode` to `basic`.\n\n## Handling errors\n\nWhen the library is unable to connect to the API (for example, due to network connection problems or a timeout), a subclass of `groq.APIConnectionError` is raised.\n\nWhen the API returns a non-success status code (that is, 4xx or 5xx\nresponse), a subclass of `groq.APIStatusError` is raised, containing `status_code` and `response` properties.\n\nAll errors inherit from `groq.APIError`.\n\n```python\nimport groq\nfrom groq import Groq\n\nclient = Groq()\n\ntry:\n client.chat.completions.create(\n messages=[\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful assisstant.\",\n },\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n },\n ],\n model=\"mixtral-8x7b-32768\",\n )\nexcept groq.APIConnectionError as e:\n print(\"The server could not be reached\")\n print(e.__cause__) # an underlying Exception, likely raised within httpx.\nexcept groq.RateLimitError as e:\n print(\"A 429 status code was received; we should back off a bit.\")\nexcept groq.APIStatusError as e:\n print(\"Another non-200-range status code was received\")\n print(e.status_code)\n print(e.response)\n```\n\nError codes are as followed:\n\n| Status Code | Error Type |\n| ----------- | -------------------------- |\n| 400 | `BadRequestError` |\n| 401 | `AuthenticationError` |\n| 403 | `PermissionDeniedError` |\n| 404 | `NotFoundError` |\n| 422 | `UnprocessableEntityError` |\n| 429 | `RateLimitError` |\n| >=500 | `InternalServerError` |\n| N/A | `APIConnectionError` |\n\n### Retries\n\nCertain errors are automatically retried 2 times by default, with a short exponential backoff.\nConnection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict,\n429 Rate Limit, and >=500 Internal errors are all retried by default.\n\nYou can use the `max_retries` option to configure or disable retry settings:\n\n```python\nfrom groq import Groq\n\n# Configure the default for all requests:\nclient = Groq(\n # default is 2\n max_retries=0,\n)\n\n# Or, configure per-request:\nclient.with_options(max_retries=5).chat.completions.create(\n messages=[\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful assisstant.\",\n },\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n },\n ],\n model=\"mixtral-8x7b-32768\",\n)\n```\n\n### Timeouts\n\nBy default requests time out after 1 minute. You can configure this with a `timeout` option,\nwhich accepts a float or an [`httpx.Timeout`](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) object:\n\n```python\nfrom groq import Groq\n\n# Configure the default for all requests:\nclient = Groq(\n # 20 seconds (default is 1 minute)\n timeout=20.0,\n)\n\n# More granular control:\nclient = Groq(\n timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0),\n)\n\n# Override per-request:\nclient.with_options(timeout=5 * 1000).chat.completions.create(\n messages=[\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful assisstant.\",\n },\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n },\n ],\n model=\"mixtral-8x7b-32768\",\n)\n```\n\nOn timeout, an `APITimeoutError` is thrown.\n\nNote that requests that time out are [retried twice by default](#retries).\n\n## Advanced\n\n### Logging\n\nWe use the standard library [`logging`](https://docs.python.org/3/library/logging.html) module.\n\nYou can enable logging by setting the environment variable `GROQ_LOG` to `debug`.\n\n```shell\n$ export GROQ_LOG=debug\n```\n\n### How to tell whether `None` means `null` or missing\n\nIn an API response, a field may be explicitly `null`, or missing entirely; in either case, its value is `None` in this library. You can differentiate the two cases with `.model_fields_set`:\n\n```py\nif response.my_field is None:\n if 'my_field' not in response.model_fields_set:\n print('Got json like {}, without a \"my_field\" key present at all.')\n else:\n print('Got json like {\"my_field\": null}.')\n```\n\n### Accessing raw response data (e.g. headers)\n\nThe \"raw\" Response object can be accessed by prefixing `.with_raw_response.` to any HTTP method call, e.g.,\n\n```py\nfrom groq import Groq\n\nclient = Groq()\nresponse = client.chat.completions.with_raw_response.create(\n messages=[{\n \"role\": \"system\",\n \"content\": \"You are a helpful assisstant.\",\n }, {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n }],\n model=\"mixtral-8x7b-32768\",\n)\nprint(response.headers.get('X-My-Header'))\n\ncompletion = response.parse() # get the object that `chat.completions.create()` would have returned\nprint(completion.id)\n```\n\nThese methods return an [`APIResponse`](https://github.com/groq/groq-python/tree/stainless/src/groq/_response.py) object.\n\nThe async client returns an [`AsyncAPIResponse`](https://github.com/groq/groq-python/tree/stainless/src/groq/_response.py) with the same structure, the only difference being `await`able methods for reading the response content.\n\n#### `.with_streaming_response`\n\nThe above interface eagerly reads the full response body when you make the request, which may not always be what you want.\n\nTo stream the response body, use `.with_streaming_response` instead, which requires a context manager and only reads the response body once you call `.read()`, `.text()`, `.json()`, `.iter_bytes()`, `.iter_text()`, `.iter_lines()` or `.parse()`. In the async client, these are async methods.\n\n```python\nwith client.chat.completions.with_streaming_response.create(\n messages=[\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful assisstant.\",\n },\n {\n \"role\": \"user\",\n \"content\": \"Explain the importance of low latency LLMs\",\n },\n ],\n model=\"mixtral-8x7b-32768\",\n) as response:\n print(response.headers.get(\"X-My-Header\"))\n\n for line in response.iter_lines():\n print(line)\n```\n\nThe context manager is required so that the response will reliably be closed.\n\n### Configuring the HTTP client\n\nYou can directly override the [httpx client](https://www.python-httpx.org/api/#client) to customize it for your use case, including:\n\n- Support for proxies\n- Custom transports\n- Additional [advanced](https://www.python-httpx.org/advanced/#client-instances) functionality\n\n```python\nimport httpx\nfrom groq import Groq\n\nclient = Groq(\n # Or use the `GROQ_BASE_URL` env var\n base_url=\"http://my.test.server.example.com:8083\",\n http_client=httpx.Client(\n proxies=\"http://my.test.proxy.example.com\",\n transport=httpx.HTTPTransport(local_address=\"0.0.0.0\"),\n ),\n)\n```\n\n### Managing HTTP resources\n\nBy default the library closes underlying HTTP connections whenever the client is [garbage collected](https://docs.python.org/3/reference/datamodel.html#object.__del__). You can manually close the client using the `.close()` method if desired, or with a context manager that closes when exiting.\n\n## Versioning\n\nThis package generally follows [SemVer](https://semver.org/spec/v2.0.0.html) conventions, though certain backwards-incompatible changes may be released as minor versions:\n\n1. Changes that only affect static types, without breaking runtime behavior.\n2. Changes to library internals which are technically public but not intended or documented for external use. _(Please open a GitHub issue to let us know if you are relying on such internals)_.\n3. Changes that we do not expect to impact the vast majority of users in practice.\n\nWe take backwards-compatibility seriously and work hard to ensure you can rely on a smooth upgrade experience.\n\nWe are keen for your feedback; please open an [issue](https://www.github.com/groq/groq-python/issues) with questions, bugs, or suggestions.\n\n## Requirements\n\nPython 3.7 or higher.\n",
"bugtrack_url": null,
"license": "",
"summary": "The official Python library for the groq API",
"version": "0.4.0",
"project_urls": {
"Homepage": "https://github.com/groq/groq-python",
"Repository": "https://github.com/groq/groq-python"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "07e4c1521cfa37ba15798b739a923449983e000ad2ca56ec39e8d7453b334ff1",
"md5": "3da2b3546347b3dd190fb7a5ca253c99",
"sha256": "3e71ad870a1e1679790b9a4f21a74ae806e30025b1e84197b51191179ffdf2a5"
},
"downloads": -1,
"filename": "groqcloud-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3da2b3546347b3dd190fb7a5ca253c99",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 65534,
"upload_time": "2024-02-16T00:19:05",
"upload_time_iso_8601": "2024-02-16T00:19:05.910836Z",
"url": "https://files.pythonhosted.org/packages/07/e4/c1521cfa37ba15798b739a923449983e000ad2ca56ec39e8d7453b334ff1/groqcloud-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "42367260c910137974591ec81e35a53e70a178f6cca89cd4b0effe896b0e3406",
"md5": "b812c99e509000bcce9bcd30a1959854",
"sha256": "8db520fb75cc9671eeffa376273d203ec6f31f449d2eef6e33a72843058e1522"
},
"downloads": -1,
"filename": "groqcloud-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "b812c99e509000bcce9bcd30a1959854",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 52318,
"upload_time": "2024-02-16T00:19:10",
"upload_time_iso_8601": "2024-02-16T00:19:10.190845Z",
"url": "https://files.pythonhosted.org/packages/42/36/7260c910137974591ec81e35a53e70a178f6cca89cd4b0effe896b0e3406/groqcloud-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-16 00:19:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "groq",
"github_project": "groq-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "groqcloud"
}
