| Name | ai21 JSON |
| Version |
3.0.0
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-11-11 12:49:45 |
| maintainer | None |
| docs_url | None |
| author | AI21 Labs |
| requires_python | <4.0,>=3.8 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<h1 align="center">
<a href="https://github.com/AI21Labs/ai21-python">AI21 Labs Python SDK</a>
</h1>
[//]: # "Add when public"
[//]: # '<a href="https://github.com/AI21Labs/ai21/actions?query=workflow%3ATest+event%3Apush+branch%3Amain"><img src="https://github.com/AI21Labs/ai21/actions/workflows/test.yaml/badge.svg" alt="Test"></a>'
[//]: # '<a href="https://pypi.org/project/ai21" target="_blank"><img src="https://img.shields.io/pypi/pyversions/ai21?color=%2334D058" alt="Supported Python versions"></a>'
<p align="center">
<a href="https://github.com/AI21Labs/ai21-python/actions/workflows/test.yaml"><img src="https://github.com/AI21Labs/ai21-python/actions/workflows/test.yaml/badge.svg?branch=main" alt="Test"></a>
<a href="https://github.com/AI21Labs/ai21-python/actions/workflows/integration-tests.yaml"><img src="https://github.com/AI21Labs/ai21-python/actions/workflows/integration-tests.yaml/badge.svg?branch=main" alt="Integration Tests"></a>
<a href="https://pypi.org/project/ai21" target="_blank"><img src="https://img.shields.io/pypi/v/ai21?color=%2334D058&label=pypi%20package" alt="Package version"></a>
<a href="https://python-poetry.org/" target="_blank"><img src="https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json" alt="Poetry"></a>
<a href="https://pypi.org/project/ai21" target="_blank"><img src="https://img.shields.io/pypi/pyversions/ai21?color=%2334D058" alt="Supported Python versions"></a>
<a href="https://github.com/semantic-release/semantic-release" target="_blank"><img src="https://img.shields.io/badge/semantic--release-python-e10079?logo=semantic-release" alt="Semantic Release Support"></a>
<a href="https://opensource.org/licenses/Apache-2.0" target="_blank"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License"></a>
</p>
---
## Table of Contents
- [Examples](#examples-tldr) 🗂️
- [Migration from v1.3.4 and below](#migration-from-v134-and-below)
- [AI21 Official Documentation](#Documentation)
- [Installation](#Installation) 💿
- [Usage - Chat Completions](#Usage)
- [Conversational RAG (Beta)](#Conversational-RAG-Beta)
- [Older Models Support Usage](#Older-Models-Support-Usage)
- [More Models](#More-Models)
- [Streaming](#Streaming)
- [TSMs](#TSMs)
- [Token Counting](#Token-Counting)
- [Environment Variables](#Environment-Variables)
- [Error Handling](#Error-Handling)
- [Cloud Providers](#Cloud-Providers) ☁️
- [AWS](#AWS)
- [Bedrock](#Bedrock)
- [SageMaker](#SageMaker)
- [Azure](#Azure)
- [Vertex](#Vertex)
## Examples (tl;dr)
If you want to quickly get a glance how to use the AI21 Python SDK and jump straight to business, you can check out the examples. Take a look at our models and see them in action! Several examples and demonstrations have been put together to show our models' functionality and capabilities.
### [Check out the Examples](examples/)
Feel free to dive in, experiment, and adapt these examples to suit your needs. We believe they'll help you get up and running quickly.
## Migration from v1.3.4 and below
In `v2.0.0` we introduced a new SDK that is not backwards compatible with the previous version.
This version allows for non-static instances of the client, defined parameters to each resource, modelized responses and
more.
<details>
<summary>Migration Examples</summary>
### Instance creation (not available in v1.3.4 and below)
```python
from ai21 import AI21Client
client = AI21Client(api_key='my_api_key')
# or set api_key in environment variable - AI21_API_KEY and then
client = AI21Client()
```
We No longer support static methods for each resource, instead we have a client instance that has a method for each
allowing for more flexibility and better control.
### Completion before/after
```diff
prompt = "some prompt"
- import ai21
- response = ai21.Completion.execute(model="j2-light", prompt=prompt, maxTokens=2)
+ from ai21 import AI21Client
+ client = ai21.AI21Client()
+ response = client.completion.create(model="j2-light", prompt=prompt, max_tokens=2)
```
This applies to all resources. You would now need to create a client instance and use it to call the resource method.
### Tokenization and Token counting before/after
```diff
- response = ai21.Tokenization.execute(text=prompt)
- print(len(response)) # number of tokens
+ from ai21 import AI21Client
+ client = AI21Client()
+ token_count = client.count_tokens(text=prompt)
```
### Key Access in Response Objects before/after
It is no longer possible to access the response object as a dictionary. Instead, you can access the response object as an object with attributes.
```diff
- import ai21
- response = ai21.Summarize.execute(source="some text", sourceType="TEXT")
- response["summary"]
+ from ai21 import AI21Client
+ from ai21.models import DocumentType
+ client = AI21Client()
+ response = client.summarize.create(source="some text", source_type=DocumentType.TEXT)
+ response.summary
```
---
### AWS Client Creations
### Bedrock Client creation before/after
```diff
- import ai21
- destination = ai21.BedrockDestination(model_id=ai21.BedrockModelID.J2_MID_V1)
- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)
+ from ai21 import AI21BedrockClient, BedrockModelID
+ client = AI21BedrockClient()
+ response = client.completion.create(prompt=prompt, max_tokens=1000, model_id=BedrockModelID.J2_MID_V1)
```
### SageMaker Client creation before/after
```diff
- import ai21
- destination = ai21.SageMakerDestination("j2-mid-test-endpoint")
- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)
+ from ai21 import AI21SageMakerClient
+ client = AI21SageMakerClient(endpoint_name="j2-mid-test-endpoint")
+ response = client.completion.create(prompt=prompt, max_tokens=1000)
```
</details>
## Documentation
---
The full documentation for the REST API can be found on [docs.ai21.com](https://docs.ai21.com/).
## Installation
---
```bash
pip install ai21
```
## Usage
---
```python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
client = AI21Client(
# defaults to os.enviorn.get('AI21_API_KEY')
api_key='my_api_key',
)
system = "You're a support engineer in a SaaS company"
messages = [
ChatMessage(content=system, role="system"),
ChatMessage(content="Hello, I need help with a signup process.", role="user"),
]
chat_completions = client.chat.completions.create(
messages=messages,
model="jamba-1.5-mini",
)
```
### Async Usage
You can use the `AsyncAI21Client` to make asynchronous requests.
There is no difference between the sync and the async client in terms of usage.
```python
import asyncio
from ai21 import AsyncAI21Client
from ai21.models.chat import ChatMessage
system = "You're a support engineer in a SaaS company"
messages = [
ChatMessage(content=system, role="system"),
ChatMessage(content="Hello, I need help with a signup process.", role="user"),
]
client = AsyncAI21Client(
# defaults to os.enviorn.get('AI21_API_KEY')
api_key='my_api_key',
)
async def main():
response = await client.chat.completions.create(
messages=messages,
model="jamba-1.5-mini",
)
print(response)
asyncio.run(main())
```
A more detailed example can be found [here](examples/studio/chat/chat_completions.py).
## Older Models Support Usage
<details>
<summary>Examples</summary>
### Supported Models:
- j2-light
- [j2-ultra](#Chat)
- [j2-mid](#Completion)
- [jamba-instruct](#Chat-Completion)
you can read more about the models [here](https://docs.ai21.com/reference/j2-complete-api-ref#jurassic-2-models).
### Chat
```python
from ai21 import AI21Client
from ai21.models import RoleType
from ai21.models import ChatMessage
system = "You're a support engineer in a SaaS company"
messages = [
ChatMessage(text="Hello, I need help with a signup process.", role=RoleType.USER),
ChatMessage(text="Hi Alice, I can help you with that. What seems to be the problem?", role=RoleType.ASSISTANT),
ChatMessage(text="I am having trouble signing up for your product with my Google account.", role=RoleType.USER),
]
client = AI21Client()
chat_response = client.chat.create(
system=system,
messages=messages,
model="j2-ultra",
)
```
For a more detailed example, see the chat [examples](examples/studio/chat.py).
### Completion
```python
from ai21 import AI21Client
client = AI21Client()
completion_response = client.completion.create(
prompt="This is a test prompt",
model="j2-mid",
)
```
### Chat Completion
```python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
system = "You're a support engineer in a SaaS company"
messages = [
ChatMessage(content=system, role="system"),
ChatMessage(content="Hello, I need help with a signup process.", role="user"),
ChatMessage(content="Hi Alice, I can help you with that. What seems to be the problem?", role="assistant"),
ChatMessage(content="I am having trouble signing up for your product with my Google account.", role="user"),
]
client = AI21Client()
response = client.chat.completions.create(
messages=messages,
model="jamba-instruct",
max_tokens=100,
temperature=0.7,
top_p=1.0,
stop=["\n"],
)
print(response)
```
Note that jamba-instruct supports async and streaming as well.
</details>
For a more detailed example, see the completion [examples](examples/studio/chat/chat_completions.py).
---
## Streaming
We currently support streaming for the Chat Completions API in Jamba.
```python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
messages = [ChatMessage(content="What is the meaning of life?", role="user")]
client = AI21Client()
response = client.chat.completions.create(
messages=messages,
model="jamba-instruct",
stream=True,
)
for chunk in response:
print(chunk.choices[0].delta.content, end="")
```
### Async Streaming
```python
import asyncio
from ai21 import AsyncAI21Client
from ai21.models.chat import ChatMessage
messages = [ChatMessage(content="What is the meaning of life?", role="user")]
client = AsyncAI21Client()
async def main():
response = await client.chat.completions.create(
messages=messages,
model="jamba-1.5-mini",
stream=True,
)
async for chunk in response:
print(chunk.choices[0].delta.content, end="")
asyncio.run(main())
```
---
### Conversational RAG (Beta)
Like chat, but with the ability to retrieve information from your Studio library.
```python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
messages = [
ChatMessage(content="Ask a question about your files", role="user"),
]
client = AI21Client()
client.library.files.create(
file_path="path/to/file",
path="path/to/file/in/library",
labels=["my_file_label"],
)
chat_response = client.beta.conversational_rag.create(
messages=messages,
labels=["my_file_label"],
)
```
For a more detailed example, see the chat [sync](examples/studio/conversational_rag/conversational_rag.py) and [async](examples/studio/conversational_rag/async_conversational_rag.py) examples.
---
### File Upload
```python
from ai21 import AI21Client
client = AI21Client()
file_id = client.library.files.create(
file_path="path/to/file",
path="path/to/file/in/library",
labels=["label1", "label2"],
public_url="www.example.com",
)
uploaded_file = client.library.files.get(file_id)
```
## Token Counting
---
By using the `count_tokens` method, you can estimate the billing for a given request.
```python
from ai21.tokenizers import get_tokenizer
tokenizer = get_tokenizer(name="jamba-tokenizer")
total_tokens = tokenizer.count_tokens(text="some text") # returns int
print(total_tokens)
```
### Async Usage
```python
from ai21.tokenizers import get_async_tokenizer
## Your async function code
#...
tokenizer = await get_async_tokenizer(name="jamba-tokenizer")
total_tokens = await tokenizer.count_tokens(text="some text") # returns int
print(total_tokens)
```
Available tokenizers are:
- `jamba-tokenizer`
- `j2-tokenizer`
For more information on AI21 Tokenizers, see the [documentation](https://github.com/AI21Labs/ai21-tokenizer).
## Environment Variables
---
You can set several environment variables to configure the client.
### Logging
We use the standard library [`logging`](https://docs.python.org/3/library/logging.html) module.
To enable logging, set the `AI21_LOG_LEVEL` environment variable.
```bash
$ export AI21_LOG_LEVEL=debug
```
### Other Important Environment Variables
- `AI21_API_KEY` - Your API key. If not set, you must pass it to the client constructor.
- `AI21_API_VERSION` - The API version. Defaults to `v1`.
- `AI21_API_HOST` - The API host. Defaults to `https://api.ai21.com/v1/`.
- `AI21_TIMEOUT_SEC` - The timeout for API requests.
- `AI21_NUM_RETRIES` - The maximum number of retries for API requests. Defaults to `3` retries.
- `AI21_AWS_REGION` - The AWS region to use for AWS clients. Defaults to `us-east-1`.
## Error Handling
---
```python
from ai21 import errors as ai21_errors
from ai21 import AI21Client, AI21APIError
from ai21.models import ChatMessage
client = AI21Client()
system = "You're a support engineer in a SaaS company"
messages = [
# Notice the given role does not exist and will be the reason for the raised error
ChatMessage(text="Hello, I need help with a signup process.", role="Non-Existent-Role"),
]
try:
chat_completion = client.chat.create(
messages=messages,
model="j2-ultra",
system=system
)
except ai21_errors.AI21ServerError as e:
print("Server error and could not be reached")
print(e.details)
except ai21_errors.TooManyRequestsError as e:
print("A 429 status code was returned. Slow down on the requests")
except AI21APIError as e:
print("A non 200 status code error. For more error types see ai21.errors")
```
## Cloud Providers
---
### AWS
AI21 Library provides convenient ways to interact with two AWS clients for use with [AWS Bedrock](https://aws.amazon.com/bedrock/ai21/) and AWS SageMaker.
### Installation
---
```bash
pip install -U "ai21[AWS]"
```
This will make sure you have the required dependencies installed, including `boto3 >= 1.28.82`.
### Usage
---
### Bedrock
```python
from ai21 import AI21BedrockClient, BedrockModelID
from ai21.models.chat import ChatMessage
client = AI21BedrockClient(region='us-east-1') # region is optional, as you can use the env variable instead
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
response = client.chat.completions.create(
messages=messages,
model_id=BedrockModelID.JAMBA_1_5_LARGE,
)
```
#### Stream
```python
from ai21 import AI21BedrockClient, BedrockModelID
from ai21.models.chat import ChatMessage
system = "You're a support engineer in a SaaS company"
messages = [
ChatMessage(content=system, role="system"),
ChatMessage(content="Hello, I need help with a signup process.", role="user"),
ChatMessage(content="Hi Alice, I can help you with that. What seems to be the problem?", role="assistant"),
ChatMessage(content="I am having trouble signing up for your product with my Google account.", role="user"),
]
client = AI21BedrockClient()
response = client.chat.completions.create(
messages=messages,
model=BedrockModelID.JAMBA_1_5_LARGE,
stream=True,
)
for chunk in response:
print(chunk.choices[0].message.content, end="")
```
#### Async
```python
import asyncio
from ai21 import AsyncAI21BedrockClient, BedrockModelID
from ai21.models.chat import ChatMessage
client = AsyncAI21BedrockClient(region='us-east-1') # region is optional, as you can use the env variable instead
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
async def main():
response = await client.chat.completions.create(
messages=messages,
model_id=BedrockModelID.JAMBA_1_5_LARGE,
)
asyncio.run(main())
```
### With Boto3 Session
```python
import boto3
from ai21 import AI21BedrockClient, BedrockModelID
from ai21.models.chat import ChatMessage
boto_session = boto3.Session(region_name="us-east-1")
client = AI21BedrockClient(session=boto_session)
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
response = client.chat.completions.create(
messages=messages,
model_id=BedrockModelID.JAMBA_1_5_LARGE,
)
```
### Async
```python
import boto3
import asyncio
from ai21 import AsyncAI21BedrockClient, BedrockModelID
from ai21.models.chat import ChatMessage
boto_session = boto3.Session(region_name="us-east-1")
client = AsyncAI21BedrockClient(session=boto_session)
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
async def main():
response = await client.chat.completions.create(
messages=messages,
model_id=BedrockModelID.JAMBA_1_5_LARGE,
)
asyncio.run(main())
```
### SageMaker
```python
from ai21 import AI21SageMakerClient
client = AI21SageMakerClient(endpoint_name="j2-endpoint-name")
response = client.summarize.create(
source="Text to summarize",
source_type="TEXT",
)
print(response.summary)
```
#### Async
```python
import asyncio
from ai21 import AsyncAI21SageMakerClient
client = AsyncAI21SageMakerClient(endpoint_name="j2-endpoint-name")
async def main():
response = await client.summarize.create(
source="Text to summarize",
source_type="TEXT",
)
print(response.summary)
asyncio.run(main())
```
### With Boto3 Session
```python
from ai21 import AI21SageMakerClient
import boto3
boto_session = boto3.Session(region_name="us-east-1")
client = AI21SageMakerClient(
session=boto_session,
endpoint_name="j2-endpoint-name",
)
```
### Azure
If you wish to interact with your Azure endpoint on Azure AI Studio, use the `AI21AzureClient`
and `AsyncAI21AzureClient` clients.
The following models are supported on Azure:
- `jamba-instruct`
```python
from ai21 import AI21AzureClient
from ai21.models.chat import ChatMessage
client = AI21AzureClient(
base_url="https://<YOUR-ENDPOINT>.inference.ai.azure.com",
api_key="<your Azure api key>",
)
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
response = client.chat.completions.create(
model="jamba-1.5-mini",
messages=messages,
)
```
#### Async
```python
import asyncio
from ai21 import AsyncAI21AzureClient
from ai21.models.chat import ChatMessage
client = AsyncAI21AzureClient(
base_url="https://<YOUR-ENDPOINT>.inference.ai.azure.com/v1/chat/completions",
api_key="<your Azure api key>",
)
messages = [
ChatMessage(content="You are a helpful assistant", role="system"),
ChatMessage(content="What is the meaning of life?", role="user")
]
async def main():
response = await client.chat.completions.create(
model="jamba-instruct",
messages=messages,
)
asyncio.run(main())
```
### Vertex
If you wish to interact with your Vertex AI endpoint on GCP, use the `AI21VertexClient`
and `AsyncAI21VertexClient` clients.
The following models are supported on Vertex:
- `jamba-1.5-mini`
- `jamba-1.5-large`
```python
from ai21 import AI21VertexClient
from ai21.models.chat import ChatMessage
# You can also set the project_id, region, access_token and Google credentials in the constructor
client = AI21VertexClient()
messages = ChatMessage(content="What is the meaning of life?", role="user")
response = client.chat.completions.create(
model="jamba-1.5-mini",
messages=[messages],
)
```
#### Async
```python
import asyncio
from ai21 import AsyncAI21VertexClient
from ai21.models.chat import ChatMessage
# You can also set the project_id, region, access_token and Google credentials in the constructor
client = AsyncAI21VertexClient()
async def main():
messages = ChatMessage(content="What is the meaning of life?", role="user")
response = await client.chat.completions.create(
model="jamba-1.5-mini",
messages=[messages],
)
asyncio.run(main())
```
Happy prompting! 🚀
Raw data
{
"_id": null,
"home_page": null,
"name": "ai21",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "AI21 Labs",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/d5/8e/57a594e6e7eb8ecd612d6c40a9c8a2d438fcce9fb357f91860bf3739d8f4/ai21-3.0.0.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n <a href=\"https://github.com/AI21Labs/ai21-python\">AI21 Labs Python SDK</a>\n</h1>\n\n[//]: # \"Add when public\"\n[//]: # '<a href=\"https://github.com/AI21Labs/ai21/actions?query=workflow%3ATest+event%3Apush+branch%3Amain\"><img src=\"https://github.com/AI21Labs/ai21/actions/workflows/test.yaml/badge.svg\" alt=\"Test\"></a>'\n[//]: # '<a href=\"https://pypi.org/project/ai21\" target=\"_blank\"><img src=\"https://img.shields.io/pypi/pyversions/ai21?color=%2334D058\" alt=\"Supported Python versions\"></a>'\n\n<p align=\"center\">\n<a href=\"https://github.com/AI21Labs/ai21-python/actions/workflows/test.yaml\"><img src=\"https://github.com/AI21Labs/ai21-python/actions/workflows/test.yaml/badge.svg?branch=main\" alt=\"Test\"></a>\n<a href=\"https://github.com/AI21Labs/ai21-python/actions/workflows/integration-tests.yaml\"><img src=\"https://github.com/AI21Labs/ai21-python/actions/workflows/integration-tests.yaml/badge.svg?branch=main\" alt=\"Integration Tests\"></a>\n<a href=\"https://pypi.org/project/ai21\" target=\"_blank\"><img src=\"https://img.shields.io/pypi/v/ai21?color=%2334D058&label=pypi%20package\" alt=\"Package version\"></a>\n<a href=\"https://python-poetry.org/\" target=\"_blank\"><img src=\"https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json\" alt=\"Poetry\"></a>\n<a href=\"https://pypi.org/project/ai21\" target=\"_blank\"><img src=\"https://img.shields.io/pypi/pyversions/ai21?color=%2334D058\" alt=\"Supported Python versions\"></a>\n<a href=\"https://github.com/semantic-release/semantic-release\" target=\"_blank\"><img src=\"https://img.shields.io/badge/semantic--release-python-e10079?logo=semantic-release\" alt=\"Semantic Release Support\"></a>\n<a href=\"https://opensource.org/licenses/Apache-2.0\" target=\"_blank\"><img src=\"https://img.shields.io/badge/License-Apache_2.0-blue.svg\" alt=\"License\"></a>\n</p>\n\n---\n\n## Table of Contents\n\n- [Examples](#examples-tldr) \ud83d\uddc2\ufe0f\n- [Migration from v1.3.4 and below](#migration-from-v134-and-below)\n- [AI21 Official Documentation](#Documentation)\n- [Installation](#Installation) \ud83d\udcbf\n- [Usage - Chat Completions](#Usage)\n- [Conversational RAG (Beta)](#Conversational-RAG-Beta)\n- [Older Models Support Usage](#Older-Models-Support-Usage)\n- [More Models](#More-Models)\n - [Streaming](#Streaming)\n - [TSMs](#TSMs)\n- [Token Counting](#Token-Counting)\n- [Environment Variables](#Environment-Variables)\n- [Error Handling](#Error-Handling)\n- [Cloud Providers](#Cloud-Providers) \u2601\ufe0f\n - [AWS](#AWS)\n - [Bedrock](#Bedrock)\n - [SageMaker](#SageMaker)\n - [Azure](#Azure)\n - [Vertex](#Vertex)\n\n## Examples (tl;dr)\n\nIf you want to quickly get a glance how to use the AI21 Python SDK and jump straight to business, you can check out the examples. Take a look at our models and see them in action! Several examples and demonstrations have been put together to show our models' functionality and capabilities.\n\n### [Check out the Examples](examples/)\n\nFeel free to dive in, experiment, and adapt these examples to suit your needs. We believe they'll help you get up and running quickly.\n\n## Migration from v1.3.4 and below\n\nIn `v2.0.0` we introduced a new SDK that is not backwards compatible with the previous version.\nThis version allows for non-static instances of the client, defined parameters to each resource, modelized responses and\nmore.\n\n<details>\n<summary>Migration Examples</summary>\n\n### Instance creation (not available in v1.3.4 and below)\n\n```python\nfrom ai21 import AI21Client\n\nclient = AI21Client(api_key='my_api_key')\n\n# or set api_key in environment variable - AI21_API_KEY and then\nclient = AI21Client()\n```\n\nWe No longer support static methods for each resource, instead we have a client instance that has a method for each\nallowing for more flexibility and better control.\n\n### Completion before/after\n\n```diff\nprompt = \"some prompt\"\n\n- import ai21\n- response = ai21.Completion.execute(model=\"j2-light\", prompt=prompt, maxTokens=2)\n\n+ from ai21 import AI21Client\n+ client = ai21.AI21Client()\n+ response = client.completion.create(model=\"j2-light\", prompt=prompt, max_tokens=2)\n```\n\nThis applies to all resources. You would now need to create a client instance and use it to call the resource method.\n\n### Tokenization and Token counting before/after\n\n```diff\n- response = ai21.Tokenization.execute(text=prompt)\n- print(len(response)) # number of tokens\n\n+ from ai21 import AI21Client\n+ client = AI21Client()\n+ token_count = client.count_tokens(text=prompt)\n```\n\n### Key Access in Response Objects before/after\n\nIt is no longer possible to access the response object as a dictionary. Instead, you can access the response object as an object with attributes.\n\n```diff\n- import ai21\n- response = ai21.Summarize.execute(source=\"some text\", sourceType=\"TEXT\")\n- response[\"summary\"]\n\n+ from ai21 import AI21Client\n+ from ai21.models import DocumentType\n+ client = AI21Client()\n+ response = client.summarize.create(source=\"some text\", source_type=DocumentType.TEXT)\n+ response.summary\n```\n\n---\n\n### AWS Client Creations\n\n### Bedrock Client creation before/after\n\n```diff\n- import ai21\n- destination = ai21.BedrockDestination(model_id=ai21.BedrockModelID.J2_MID_V1)\n- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)\n\n+ from ai21 import AI21BedrockClient, BedrockModelID\n+ client = AI21BedrockClient()\n+ response = client.completion.create(prompt=prompt, max_tokens=1000, model_id=BedrockModelID.J2_MID_V1)\n```\n\n### SageMaker Client creation before/after\n\n```diff\n- import ai21\n- destination = ai21.SageMakerDestination(\"j2-mid-test-endpoint\")\n- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)\n\n+ from ai21 import AI21SageMakerClient\n+ client = AI21SageMakerClient(endpoint_name=\"j2-mid-test-endpoint\")\n+ response = client.completion.create(prompt=prompt, max_tokens=1000)\n```\n\n</details>\n\n## Documentation\n\n---\n\nThe full documentation for the REST API can be found on [docs.ai21.com](https://docs.ai21.com/).\n\n## Installation\n\n---\n\n```bash\npip install ai21\n```\n\n## Usage\n\n---\n\n```python\nfrom ai21 import AI21Client\nfrom ai21.models.chat import ChatMessage\n\nclient = AI21Client(\n # defaults to os.enviorn.get('AI21_API_KEY')\n api_key='my_api_key',\n)\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n ChatMessage(content=system, role=\"system\"),\n ChatMessage(content=\"Hello, I need help with a signup process.\", role=\"user\"),\n]\n\nchat_completions = client.chat.completions.create(\n messages=messages,\n model=\"jamba-1.5-mini\",\n)\n```\n\n### Async Usage\n\nYou can use the `AsyncAI21Client` to make asynchronous requests.\nThere is no difference between the sync and the async client in terms of usage.\n\n```python\nimport asyncio\n\nfrom ai21 import AsyncAI21Client\nfrom ai21.models.chat import ChatMessage\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n ChatMessage(content=system, role=\"system\"),\n ChatMessage(content=\"Hello, I need help with a signup process.\", role=\"user\"),\n]\n\nclient = AsyncAI21Client(\n # defaults to os.enviorn.get('AI21_API_KEY')\n api_key='my_api_key',\n)\n\n\nasync def main():\n response = await client.chat.completions.create(\n messages=messages,\n model=\"jamba-1.5-mini\",\n )\n\n print(response)\n\n\nasyncio.run(main())\n\n```\n\nA more detailed example can be found [here](examples/studio/chat/chat_completions.py).\n\n## Older Models Support Usage\n\n<details>\n<summary>Examples</summary>\n\n### Supported Models:\n\n- j2-light\n- [j2-ultra](#Chat)\n- [j2-mid](#Completion)\n- [jamba-instruct](#Chat-Completion)\n\nyou can read more about the models [here](https://docs.ai21.com/reference/j2-complete-api-ref#jurassic-2-models).\n\n### Chat\n\n```python\nfrom ai21 import AI21Client\nfrom ai21.models import RoleType\nfrom ai21.models import ChatMessage\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n ChatMessage(text=\"Hello, I need help with a signup process.\", role=RoleType.USER),\n ChatMessage(text=\"Hi Alice, I can help you with that. What seems to be the problem?\", role=RoleType.ASSISTANT),\n ChatMessage(text=\"I am having trouble signing up for your product with my Google account.\", role=RoleType.USER),\n]\n\n\nclient = AI21Client()\nchat_response = client.chat.create(\n system=system,\n messages=messages,\n model=\"j2-ultra\",\n)\n```\n\nFor a more detailed example, see the chat [examples](examples/studio/chat.py).\n\n### Completion\n\n```python\nfrom ai21 import AI21Client\n\n\nclient = AI21Client()\ncompletion_response = client.completion.create(\n prompt=\"This is a test prompt\",\n model=\"j2-mid\",\n)\n```\n\n### Chat Completion\n\n```python\nfrom ai21 import AI21Client\nfrom ai21.models.chat import ChatMessage\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n ChatMessage(content=system, role=\"system\"),\n ChatMessage(content=\"Hello, I need help with a signup process.\", role=\"user\"),\n ChatMessage(content=\"Hi Alice, I can help you with that. What seems to be the problem?\", role=\"assistant\"),\n ChatMessage(content=\"I am having trouble signing up for your product with my Google account.\", role=\"user\"),\n]\n\nclient = AI21Client()\n\nresponse = client.chat.completions.create(\n messages=messages,\n model=\"jamba-instruct\",\n max_tokens=100,\n temperature=0.7,\n top_p=1.0,\n stop=[\"\\n\"],\n)\n\nprint(response)\n```\n\nNote that jamba-instruct supports async and streaming as well.\n\n</details>\n\nFor a more detailed example, see the completion [examples](examples/studio/chat/chat_completions.py).\n\n---\n\n## Streaming\n\nWe currently support streaming for the Chat Completions API in Jamba.\n\n```python\nfrom ai21 import AI21Client\nfrom ai21.models.chat import ChatMessage\n\nmessages = [ChatMessage(content=\"What is the meaning of life?\", role=\"user\")]\n\nclient = AI21Client()\n\nresponse = client.chat.completions.create(\n messages=messages,\n model=\"jamba-instruct\",\n stream=True,\n)\nfor chunk in response:\n print(chunk.choices[0].delta.content, end=\"\")\n\n```\n\n### Async Streaming\n\n```python\nimport asyncio\n\nfrom ai21 import AsyncAI21Client\nfrom ai21.models.chat import ChatMessage\n\nmessages = [ChatMessage(content=\"What is the meaning of life?\", role=\"user\")]\n\nclient = AsyncAI21Client()\n\n\nasync def main():\n response = await client.chat.completions.create(\n messages=messages,\n model=\"jamba-1.5-mini\",\n stream=True,\n )\n async for chunk in response:\n print(chunk.choices[0].delta.content, end=\"\")\n\n\nasyncio.run(main())\n\n```\n\n---\n\n### Conversational RAG (Beta)\n\nLike chat, but with the ability to retrieve information from your Studio library.\n\n```python\nfrom ai21 import AI21Client\nfrom ai21.models.chat import ChatMessage\n\nmessages = [\n ChatMessage(content=\"Ask a question about your files\", role=\"user\"),\n]\n\nclient = AI21Client()\n\nclient.library.files.create(\n file_path=\"path/to/file\",\n path=\"path/to/file/in/library\",\n labels=[\"my_file_label\"],\n)\nchat_response = client.beta.conversational_rag.create(\n messages=messages,\n labels=[\"my_file_label\"],\n)\n```\n\nFor a more detailed example, see the chat [sync](examples/studio/conversational_rag/conversational_rag.py) and [async](examples/studio/conversational_rag/async_conversational_rag.py) examples.\n\n---\n\n### File Upload\n\n```python\nfrom ai21 import AI21Client\n\nclient = AI21Client()\n\nfile_id = client.library.files.create(\n file_path=\"path/to/file\",\n path=\"path/to/file/in/library\",\n labels=[\"label1\", \"label2\"],\n public_url=\"www.example.com\",\n)\n\nuploaded_file = client.library.files.get(file_id)\n```\n\n## Token Counting\n\n---\n\nBy using the `count_tokens` method, you can estimate the billing for a given request.\n\n```python\nfrom ai21.tokenizers import get_tokenizer\n\ntokenizer = get_tokenizer(name=\"jamba-tokenizer\")\ntotal_tokens = tokenizer.count_tokens(text=\"some text\") # returns int\nprint(total_tokens)\n```\n\n### Async Usage\n\n```python\nfrom ai21.tokenizers import get_async_tokenizer\n\n## Your async function code\n#...\ntokenizer = await get_async_tokenizer(name=\"jamba-tokenizer\")\ntotal_tokens = await tokenizer.count_tokens(text=\"some text\") # returns int\nprint(total_tokens)\n```\n\nAvailable tokenizers are:\n\n- `jamba-tokenizer`\n- `j2-tokenizer`\n\nFor more information on AI21 Tokenizers, see the [documentation](https://github.com/AI21Labs/ai21-tokenizer).\n\n## Environment Variables\n\n---\n\nYou can set several environment variables to configure the client.\n\n### Logging\n\nWe use the standard library [`logging`](https://docs.python.org/3/library/logging.html) module.\n\nTo enable logging, set the `AI21_LOG_LEVEL` environment variable.\n\n```bash\n$ export AI21_LOG_LEVEL=debug\n```\n\n### Other Important Environment Variables\n\n- `AI21_API_KEY` - Your API key. If not set, you must pass it to the client constructor.\n- `AI21_API_VERSION` - The API version. Defaults to `v1`.\n- `AI21_API_HOST` - The API host. Defaults to `https://api.ai21.com/v1/`.\n- `AI21_TIMEOUT_SEC` - The timeout for API requests.\n- `AI21_NUM_RETRIES` - The maximum number of retries for API requests. Defaults to `3` retries.\n- `AI21_AWS_REGION` - The AWS region to use for AWS clients. Defaults to `us-east-1`.\n\n## Error Handling\n\n---\n\n```python\nfrom ai21 import errors as ai21_errors\nfrom ai21 import AI21Client, AI21APIError\nfrom ai21.models import ChatMessage\n\nclient = AI21Client()\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n # Notice the given role does not exist and will be the reason for the raised error\n ChatMessage(text=\"Hello, I need help with a signup process.\", role=\"Non-Existent-Role\"),\n ]\n\ntry:\n chat_completion = client.chat.create(\n messages=messages,\n model=\"j2-ultra\",\n system=system\n )\nexcept ai21_errors.AI21ServerError as e:\n print(\"Server error and could not be reached\")\n print(e.details)\nexcept ai21_errors.TooManyRequestsError as e:\n print(\"A 429 status code was returned. Slow down on the requests\")\nexcept AI21APIError as e:\n print(\"A non 200 status code error. For more error types see ai21.errors\")\n\n```\n\n## Cloud Providers\n\n---\n\n### AWS\n\nAI21 Library provides convenient ways to interact with two AWS clients for use with [AWS Bedrock](https://aws.amazon.com/bedrock/ai21/) and AWS SageMaker.\n\n### Installation\n\n---\n\n```bash\npip install -U \"ai21[AWS]\"\n```\n\nThis will make sure you have the required dependencies installed, including `boto3 >= 1.28.82`.\n\n### Usage\n\n---\n\n### Bedrock\n\n```python\nfrom ai21 import AI21BedrockClient, BedrockModelID\nfrom ai21.models.chat import ChatMessage\n\nclient = AI21BedrockClient(region='us-east-1') # region is optional, as you can use the env variable instead\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nresponse = client.chat.completions.create(\n messages=messages,\n model_id=BedrockModelID.JAMBA_1_5_LARGE,\n)\n```\n\n#### Stream\n\n```python\nfrom ai21 import AI21BedrockClient, BedrockModelID\nfrom ai21.models.chat import ChatMessage\n\nsystem = \"You're a support engineer in a SaaS company\"\nmessages = [\n ChatMessage(content=system, role=\"system\"),\n ChatMessage(content=\"Hello, I need help with a signup process.\", role=\"user\"),\n ChatMessage(content=\"Hi Alice, I can help you with that. What seems to be the problem?\", role=\"assistant\"),\n ChatMessage(content=\"I am having trouble signing up for your product with my Google account.\", role=\"user\"),\n]\n\nclient = AI21BedrockClient()\n\nresponse = client.chat.completions.create(\n messages=messages,\n model=BedrockModelID.JAMBA_1_5_LARGE,\n stream=True,\n)\n\nfor chunk in response:\n print(chunk.choices[0].message.content, end=\"\")\n```\n\n#### Async\n\n```python\nimport asyncio\nfrom ai21 import AsyncAI21BedrockClient, BedrockModelID\nfrom ai21.models.chat import ChatMessage\n\nclient = AsyncAI21BedrockClient(region='us-east-1') # region is optional, as you can use the env variable instead\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nasync def main():\n response = await client.chat.completions.create(\n messages=messages,\n model_id=BedrockModelID.JAMBA_1_5_LARGE,\n )\n\n\nasyncio.run(main())\n```\n\n### With Boto3 Session\n\n```python\nimport boto3\n\nfrom ai21 import AI21BedrockClient, BedrockModelID\nfrom ai21.models.chat import ChatMessage\n\nboto_session = boto3.Session(region_name=\"us-east-1\")\n\nclient = AI21BedrockClient(session=boto_session)\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nresponse = client.chat.completions.create(\n messages=messages,\n model_id=BedrockModelID.JAMBA_1_5_LARGE,\n)\n```\n\n### Async\n\n```python\nimport boto3\nimport asyncio\n\nfrom ai21 import AsyncAI21BedrockClient, BedrockModelID\nfrom ai21.models.chat import ChatMessage\n\nboto_session = boto3.Session(region_name=\"us-east-1\")\n\nclient = AsyncAI21BedrockClient(session=boto_session)\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nasync def main():\n response = await client.chat.completions.create(\n messages=messages,\n model_id=BedrockModelID.JAMBA_1_5_LARGE,\n )\n\nasyncio.run(main())\n```\n\n### SageMaker\n\n```python\nfrom ai21 import AI21SageMakerClient\n\nclient = AI21SageMakerClient(endpoint_name=\"j2-endpoint-name\")\nresponse = client.summarize.create(\n source=\"Text to summarize\",\n source_type=\"TEXT\",\n)\nprint(response.summary)\n```\n\n#### Async\n\n```python\nimport asyncio\nfrom ai21 import AsyncAI21SageMakerClient\n\nclient = AsyncAI21SageMakerClient(endpoint_name=\"j2-endpoint-name\")\n\nasync def main():\n response = await client.summarize.create(\n source=\"Text to summarize\",\n source_type=\"TEXT\",\n )\n print(response.summary)\n\nasyncio.run(main())\n```\n\n### With Boto3 Session\n\n```python\nfrom ai21 import AI21SageMakerClient\nimport boto3\nboto_session = boto3.Session(region_name=\"us-east-1\")\n\nclient = AI21SageMakerClient(\n session=boto_session,\n endpoint_name=\"j2-endpoint-name\",\n)\n```\n\n### Azure\n\nIf you wish to interact with your Azure endpoint on Azure AI Studio, use the `AI21AzureClient`\nand `AsyncAI21AzureClient` clients.\n\nThe following models are supported on Azure:\n\n- `jamba-instruct`\n\n```python\nfrom ai21 import AI21AzureClient\nfrom ai21.models.chat import ChatMessage\n\nclient = AI21AzureClient(\n base_url=\"https://<YOUR-ENDPOINT>.inference.ai.azure.com\",\n api_key=\"<your Azure api key>\",\n)\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nresponse = client.chat.completions.create(\n model=\"jamba-1.5-mini\",\n messages=messages,\n)\n```\n\n#### Async\n\n```python\nimport asyncio\nfrom ai21 import AsyncAI21AzureClient\nfrom ai21.models.chat import ChatMessage\n\nclient = AsyncAI21AzureClient(\n base_url=\"https://<YOUR-ENDPOINT>.inference.ai.azure.com/v1/chat/completions\",\n api_key=\"<your Azure api key>\",\n)\n\nmessages = [\n ChatMessage(content=\"You are a helpful assistant\", role=\"system\"),\n ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n]\n\nasync def main():\n response = await client.chat.completions.create(\n model=\"jamba-instruct\",\n messages=messages,\n )\n\nasyncio.run(main())\n```\n\n### Vertex\n\nIf you wish to interact with your Vertex AI endpoint on GCP, use the `AI21VertexClient`\nand `AsyncAI21VertexClient` clients.\n\nThe following models are supported on Vertex:\n\n- `jamba-1.5-mini`\n- `jamba-1.5-large`\n\n```python\nfrom ai21 import AI21VertexClient\n\nfrom ai21.models.chat import ChatMessage\n\n# You can also set the project_id, region, access_token and Google credentials in the constructor\nclient = AI21VertexClient()\n\nmessages = ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n\nresponse = client.chat.completions.create(\n model=\"jamba-1.5-mini\",\n messages=[messages],\n)\n```\n\n#### Async\n\n```python\nimport asyncio\n\nfrom ai21 import AsyncAI21VertexClient\nfrom ai21.models.chat import ChatMessage\n\n# You can also set the project_id, region, access_token and Google credentials in the constructor\nclient = AsyncAI21VertexClient()\n\n\nasync def main():\n messages = ChatMessage(content=\"What is the meaning of life?\", role=\"user\")\n\n response = await client.chat.completions.create(\n model=\"jamba-1.5-mini\",\n messages=[messages],\n )\n\nasyncio.run(main())\n```\n\nHappy prompting! \ud83d\ude80\n\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "3.0.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2afd2254f7583460a35000bea620b08e5295400c0865ce1357d39aca22f06ceb",
"md5": "fd9de47960ca99c5ad41711b79799235",
"sha256": "b075a7ad5adc27da1ed929c258476ea009905649a41cf66bc718aea3791aaaa3"
},
"downloads": -1,
"filename": "ai21-3.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fd9de47960ca99c5ad41711b79799235",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 59802,
"upload_time": "2024-11-11T12:49:43",
"upload_time_iso_8601": "2024-11-11T12:49:43.683883Z",
"url": "https://files.pythonhosted.org/packages/2a/fd/2254f7583460a35000bea620b08e5295400c0865ce1357d39aca22f06ceb/ai21-3.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d58e57a594e6e7eb8ecd612d6c40a9c8a2d438fcce9fb357f91860bf3739d8f4",
"md5": "fdef56fec85b88b6f416ae2fd1d51657",
"sha256": "c0587c5d39e36c792eff852707f9654fcfb29a0839edf1998392fddf595baa02"
},
"downloads": -1,
"filename": "ai21-3.0.0.tar.gz",
"has_sig": false,
"md5_digest": "fdef56fec85b88b6f416ae2fd1d51657",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 39263,
"upload_time": "2024-11-11T12:49:45",
"upload_time_iso_8601": "2024-11-11T12:49:45.318776Z",
"url": "https://files.pythonhosted.org/packages/d5/8e/57a594e6e7eb8ecd612d6c40a9c8a2d438fcce9fb357f91860bf3739d8f4/ai21-3.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 12:49:45",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ai21"
}
