<div align="center">
<a href="https://www.together.ai/">
<img alt="together.ai" height="100px" src="https://assets-global.website-files.com/64f6f2c0e3f4c5a91c1e823a/654693d569494912cfc0c0d4_favicon.svg">
</a>
</div>
# Together Python API library
[](https://pypi.org/project/together/)
[](https://discord.com/invite/9Rk6sSeWEG)
[](https://twitter.com/togethercompute)
The [Together Python API Library](https://pypi.org/project/together/) is the official Python client for Together's API platform, providing a convenient way for interacting with the REST APIs and enables easy integrations with Python 3.8+ applications with easy to use synchronous and asynchronous clients.
## Installation
> 🚧
> The Library was rewritten in v1.0.0 released in April of 2024. There were significant changes made.
To install Together Python Library from PyPI, simply run:
```shell Shell
pip install --upgrade together
```
### Setting up API Key
> 🚧 You will need to create an account with [Together.ai](https://api.together.xyz/) to obtain a Together API Key.
Once logged in to the Together Playground, you can find available API keys in [this settings page](https://api.together.xyz/settings/api-keys).
#### Setting environment variable
```shell
export TOGETHER_API_KEY=xxxxx
```
#### Using the client
```python
from together import Together
client = Together(api_key="xxxxx")
```
This repo contains both a Python Library and a CLI. We'll demonstrate how to use both below.
## Usage – Python Client
### Chat Completions
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
response = client.chat.completions.create(
model="mistralai/Mixtral-8x7B-Instruct-v0.1",
messages=[{"role": "user", "content": "tell me about new york"}],
)
print(response.choices[0].message.content)
```
#### Streaming
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
stream = client.chat.completions.create(
model="mistralai/Mixtral-8x7B-Instruct-v0.1",
messages=[{"role": "user", "content": "tell me about new york"}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="", flush=True)
```
#### Async usage
```python
import os, asyncio
from together import AsyncTogether
async_client = AsyncTogether(api_key=os.environ.get("TOGETHER_API_KEY"))
messages = [
"What are the top things to do in San Francisco?",
"What country is Paris in?",
]
async def async_chat_completion(messages):
async_client = AsyncTogether(api_key=os.environ.get("TOGETHER_API_KEY"))
tasks = [
async_client.chat.completions.create(
model="mistralai/Mixtral-8x7B-Instruct-v0.1",
messages=[{"role": "user", "content": message}],
)
for message in messages
]
responses = await asyncio.gather(*tasks)
for response in responses:
print(response.choices[0].message.content)
asyncio.run(async_chat_completion(messages))
```
### Completions
Completions are for code and language models shown [here](https://docs.together.ai/docs/inference-models). Below, a code model example is shown.
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
response = client.completions.create(
model="codellama/CodeLlama-34b-Python-hf",
prompt="Write a Next.js component with TailwindCSS for a header component.",
max_tokens=200,
)
print(response.choices[0].text)
```
#### Streaming
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
stream = client.completions.create(
model="codellama/CodeLlama-34b-Python-hf",
prompt="Write a Next.js component with TailwindCSS for a header component.",
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="", flush=True)
```
#### Async usage
```python
import os, asyncio
from together import AsyncTogether
async_client = AsyncTogether(api_key=os.environ.get("TOGETHER_API_KEY"))
prompts = [
"Write a Next.js component with TailwindCSS for a header component.",
"Write a python function for the fibonacci sequence",
]
async def async_chat_completion(prompts):
tasks = [
async_client.completions.create(
model="codellama/CodeLlama-34b-Python-hf",
prompt=prompt,
)
for prompt in prompts
]
responses = await asyncio.gather(*tasks)
for response in responses:
print(response.choices[0].text)
asyncio.run(async_chat_completion(prompts))
```
### Image generation
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
response = client.images.generate(
prompt="space robots",
model="stabilityai/stable-diffusion-xl-base-1.0",
steps=10,
n=4,
)
print(response.data[0].b64_json)
```
### Embeddings
```python
from typing import List
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
def get_embeddings(texts: List[str], model: str) -> List[List[float]]:
texts = [text.replace("\n", " ") for text in texts]
outputs = client.embeddings.create(model=model, input = texts)
return [outputs.data[i].embedding for i in range(len(texts))]
input_texts = ['Our solar system orbits the Milky Way galaxy at about 515,000 mph']
embeddings = get_embeddings(input_texts, model='togethercomputer/m2-bert-80M-8k-retrieval')
print(embeddings)
```
### Files
The files API is used for fine-tuning and allows developers to upload data to fine-tune on. It also has several methods to list all files, retrive files, and delete files. Please refer to our fine-tuning docs [here](https://docs.together.ai/docs/fine-tuning-python).
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
client.files.upload(file="somedata.jsonl") # uploads a file
client.files.list() # lists all uploaded files
client.files.retrieve(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # retrieves a specific file
client.files.retrieve_content(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # retrieves content of a specific file
client.files.delete(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # deletes a file
```
### Fine-tunes
The finetune API is used for fine-tuning and allows developers to create finetuning jobs. It also has several methods to list all jobs, retrive statuses and get checkpoints. Please refer to our fine-tuning docs [here](https://docs.together.ai/docs/fine-tuning-python).
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
client.fine_tuning.create(
training_file = 'file-d0d318cb-b7d9-493a-bd70-1cfe089d3815',
model = 'mistralai/Mixtral-8x7B-Instruct-v0.1',
n_epochs = 3,
n_checkpoints = 1,
batch_size = "max",
learning_rate = 1e-5,
suffix = 'my-demo-finetune',
wandb_api_key = '1a2b3c4d5e.......',
)
client.fine_tuning.list() # lists all fine-tuned jobs
client.fine_tuning.retrieve(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # retrieves information on finetune event
client.fine_tuning.cancel(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # Cancels a fine-tuning job
client.fine_tuning.list_events(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # Lists events of a fine-tune job
client.fine_tuning.download(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # downloads compressed fine-tuned model or checkpoint to local disk
```
### Models
This lists all the models that Together supports.
```python
import os
from together import Together
client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
models = client.models.list()
for model in models:
print(model)
```
## Usage – CLI
### Chat Completions
```bash
together chat.completions \
--message "system" "You are a helpful assistant named Together" \
--message "user" "What is your name?" \
--model mistralai/Mixtral-8x7B-Instruct-v0.1
```
The Chat Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.
### Completions
```bash
together completions \
"Large language models are " \
--model mistralai/Mixtral-8x7B-v0.1 \
--max-tokens 512 \
--stop "."
```
The Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.
### Image Generations
```bash
together images generate \
"space robots" \
--model stabilityai/stable-diffusion-xl-base-1.0 \
--n 4
```
The image is opened in the default image viewer by default. To disable this, use `--no-show`.
### Files
```bash
# Help
together files --help
# Check file
together files check example.jsonl
# Upload file
together files upload example.jsonl
# List files
together files list
# Retrieve file metadata
together files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894
# Retrieve file content
together files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894
# Delete remote file
together files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894
```
### Fine-tuning
```bash
# Help
together fine-tuning --help
# Create fine-tune job
together fine-tuning create \
--model togethercomputer/llama-2-7b-chat \
--training-file file-711d8724-b3e3-4ae2-b516-94841958117d
# List fine-tune jobs
together fine-tuning list
# Retrieve fine-tune job details
together fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
# List fine-tune job events
together fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
# Cancel running job
together fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
# Download fine-tuned model weights
together fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
```
### Models
```bash
# Help
together models --help
# List models
together models list
```
## Contributing
Refer to the [Contributing Guide](CONTRIBUTING.md)
Raw data
{
"_id": null,
"home_page": "https://github.com/togethercomputer/together-python",
"name": "together",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Together AI",
"author_email": "support@together.ai",
"download_url": "https://files.pythonhosted.org/packages/8c/9f/f79e3f4ad1cee6ec2d569437e7b231853c39fafa55b07e7aec8c6ebe199c/together-1.3.5.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <a href=\"https://www.together.ai/\">\n <img alt=\"together.ai\" height=\"100px\" src=\"https://assets-global.website-files.com/64f6f2c0e3f4c5a91c1e823a/654693d569494912cfc0c0d4_favicon.svg\">\n </a>\n</div>\n\n# Together Python API library\n\n[](https://pypi.org/project/together/)\n[](https://discord.com/invite/9Rk6sSeWEG)\n[](https://twitter.com/togethercompute)\n\nThe [Together Python API Library](https://pypi.org/project/together/) is the official Python client for Together's API platform, providing a convenient way for interacting with the REST APIs and enables easy integrations with Python 3.8+ applications with easy to use synchronous and asynchronous clients.\n\n\n\n## Installation\n\n> \ud83d\udea7\n> The Library was rewritten in v1.0.0 released in April of 2024. There were significant changes made.\n\nTo install Together Python Library from PyPI, simply run:\n\n```shell Shell\npip install --upgrade together\n```\n\n### Setting up API Key\n\n> \ud83d\udea7 You will need to create an account with [Together.ai](https://api.together.xyz/) to obtain a Together API Key.\n\nOnce logged in to the Together Playground, you can find available API keys in [this settings page](https://api.together.xyz/settings/api-keys).\n\n#### Setting environment variable\n\n```shell\nexport TOGETHER_API_KEY=xxxxx\n```\n\n#### Using the client\n\n```python\nfrom together import Together\n\nclient = Together(api_key=\"xxxxx\")\n```\n\nThis repo contains both a Python Library and a CLI. We'll demonstrate how to use both below.\n\n## Usage \u2013 Python Client\n\n### Chat Completions\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nresponse = client.chat.completions.create(\n model=\"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n messages=[{\"role\": \"user\", \"content\": \"tell me about new york\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n#### Streaming\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\nstream = client.chat.completions.create(\n model=\"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n messages=[{\"role\": \"user\", \"content\": \"tell me about new york\"}],\n stream=True,\n)\n\nfor chunk in stream:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\n```\n\n#### Async usage\n\n```python\nimport os, asyncio\nfrom together import AsyncTogether\n\nasync_client = AsyncTogether(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\nmessages = [\n \"What are the top things to do in San Francisco?\",\n \"What country is Paris in?\",\n]\n\nasync def async_chat_completion(messages):\n async_client = AsyncTogether(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n tasks = [\n async_client.chat.completions.create(\n model=\"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n messages=[{\"role\": \"user\", \"content\": message}],\n )\n for message in messages\n ]\n responses = await asyncio.gather(*tasks)\n\n for response in responses:\n print(response.choices[0].message.content)\n\nasyncio.run(async_chat_completion(messages))\n```\n\n### Completions\n\nCompletions are for code and language models shown [here](https://docs.together.ai/docs/inference-models). Below, a code model example is shown.\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nresponse = client.completions.create(\n model=\"codellama/CodeLlama-34b-Python-hf\",\n prompt=\"Write a Next.js component with TailwindCSS for a header component.\",\n max_tokens=200,\n)\nprint(response.choices[0].text)\n```\n\n#### Streaming\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\nstream = client.completions.create(\n model=\"codellama/CodeLlama-34b-Python-hf\",\n prompt=\"Write a Next.js component with TailwindCSS for a header component.\",\n stream=True,\n)\n\nfor chunk in stream:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\n```\n\n#### Async usage\n\n```python\nimport os, asyncio\nfrom together import AsyncTogether\n\nasync_client = AsyncTogether(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\nprompts = [\n \"Write a Next.js component with TailwindCSS for a header component.\",\n \"Write a python function for the fibonacci sequence\",\n]\n\nasync def async_chat_completion(prompts):\n tasks = [\n async_client.completions.create(\n model=\"codellama/CodeLlama-34b-Python-hf\",\n prompt=prompt,\n )\n for prompt in prompts\n ]\n responses = await asyncio.gather(*tasks)\n\n for response in responses:\n print(response.choices[0].text)\n\nasyncio.run(async_chat_completion(prompts))\n```\n\n### Image generation\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nresponse = client.images.generate(\n prompt=\"space robots\",\n model=\"stabilityai/stable-diffusion-xl-base-1.0\",\n steps=10,\n n=4,\n)\nprint(response.data[0].b64_json)\n```\n\n### Embeddings\n\n```python\nfrom typing import List\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\ndef get_embeddings(texts: List[str], model: str) -> List[List[float]]:\n texts = [text.replace(\"\\n\", \" \") for text in texts]\n outputs = client.embeddings.create(model=model, input = texts)\n return [outputs.data[i].embedding for i in range(len(texts))]\n\ninput_texts = ['Our solar system orbits the Milky Way galaxy at about 515,000 mph']\nembeddings = get_embeddings(input_texts, model='togethercomputer/m2-bert-80M-8k-retrieval')\n\nprint(embeddings)\n```\n\n### Files\n\nThe files API is used for fine-tuning and allows developers to upload data to fine-tune on. It also has several methods to list all files, retrive files, and delete files. Please refer to our fine-tuning docs [here](https://docs.together.ai/docs/fine-tuning-python).\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nclient.files.upload(file=\"somedata.jsonl\") # uploads a file\nclient.files.list() # lists all uploaded files\nclient.files.retrieve(id=\"file-d0d318cb-b7d9-493a-bd70-1cfe089d3815\") # retrieves a specific file\nclient.files.retrieve_content(id=\"file-d0d318cb-b7d9-493a-bd70-1cfe089d3815\") # retrieves content of a specific file\nclient.files.delete(id=\"file-d0d318cb-b7d9-493a-bd70-1cfe089d3815\") # deletes a file\n```\n\n### Fine-tunes\n\nThe finetune API is used for fine-tuning and allows developers to create finetuning jobs. It also has several methods to list all jobs, retrive statuses and get checkpoints. Please refer to our fine-tuning docs [here](https://docs.together.ai/docs/fine-tuning-python).\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nclient.fine_tuning.create(\n training_file = 'file-d0d318cb-b7d9-493a-bd70-1cfe089d3815',\n model = 'mistralai/Mixtral-8x7B-Instruct-v0.1',\n n_epochs = 3,\n n_checkpoints = 1,\n batch_size = \"max\",\n learning_rate = 1e-5,\n suffix = 'my-demo-finetune',\n wandb_api_key = '1a2b3c4d5e.......',\n)\nclient.fine_tuning.list() # lists all fine-tuned jobs\nclient.fine_tuning.retrieve(id=\"ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\") # retrieves information on finetune event\nclient.fine_tuning.cancel(id=\"ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\") # Cancels a fine-tuning job\nclient.fine_tuning.list_events(id=\"ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\") # Lists events of a fine-tune job\nclient.fine_tuning.download(id=\"ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\") # downloads compressed fine-tuned model or checkpoint to local disk\n```\n\n### Models\n\nThis lists all the models that Together supports.\n\n```python\nimport os\nfrom together import Together\n\nclient = Together(api_key=os.environ.get(\"TOGETHER_API_KEY\"))\n\nmodels = client.models.list()\n\nfor model in models:\n print(model)\n```\n\n## Usage \u2013 CLI\n\n### Chat Completions\n\n```bash\ntogether chat.completions \\\n --message \"system\" \"You are a helpful assistant named Together\" \\\n --message \"user\" \"What is your name?\" \\\n --model mistralai/Mixtral-8x7B-Instruct-v0.1\n```\n\nThe Chat Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.\n\n### Completions\n\n```bash\ntogether completions \\\n \"Large language models are \" \\\n --model mistralai/Mixtral-8x7B-v0.1 \\\n --max-tokens 512 \\\n --stop \".\"\n```\n\nThe Completions CLI enables streaming tokens to stdout by default. To disable streaming, use `--no-stream`.\n\n### Image Generations\n\n```bash\ntogether images generate \\\n \"space robots\" \\\n --model stabilityai/stable-diffusion-xl-base-1.0 \\\n --n 4\n```\n\nThe image is opened in the default image viewer by default. To disable this, use `--no-show`.\n\n### Files\n\n```bash\n# Help\ntogether files --help\n\n# Check file\ntogether files check example.jsonl\n\n# Upload file\ntogether files upload example.jsonl\n\n# List files\ntogether files list\n\n# Retrieve file metadata\ntogether files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894\n\n# Retrieve file content\ntogether files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894\n\n# Delete remote file\ntogether files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894\n```\n\n### Fine-tuning\n\n```bash\n# Help\ntogether fine-tuning --help\n\n# Create fine-tune job\ntogether fine-tuning create \\\n --model togethercomputer/llama-2-7b-chat \\\n --training-file file-711d8724-b3e3-4ae2-b516-94841958117d\n\n# List fine-tune jobs\ntogether fine-tuning list\n\n# Retrieve fine-tune job details\ntogether fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\n\n# List fine-tune job events\ntogether fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\n\n# Cancel running job\ntogether fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\n\n# Download fine-tuned model weights\ntogether fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b\n```\n\n### Models\n\n```bash\n# Help\ntogether models --help\n\n# List models\ntogether models list\n```\n\n## Contributing\n\nRefer to the [Contributing Guide](CONTRIBUTING.md)\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Python client for Together's Cloud Platform!",
"version": "1.3.5",
"project_urls": {
"Bug Tracker": "https://github.com/togethercomputer/together-python/issues",
"Homepage": "https://github.com/togethercomputer/together-python",
"Repository": "https://github.com/togethercomputer/together-python"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f524cfcfd701007b653358703c8fda626498193f5f9f1338770906c7e3bfd386",
"md5": "e5b3f51e33616e85c873b581c3621911",
"sha256": "7c58106a10b317b76267604c23476f12fb3c1686138bf943aa90ed37260ade11"
},
"downloads": -1,
"filename": "together-1.3.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e5b3f51e33616e85c873b581c3621911",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 70298,
"upload_time": "2024-11-19T09:34:18",
"upload_time_iso_8601": "2024-11-19T09:34:18.885270Z",
"url": "https://files.pythonhosted.org/packages/f5/24/cfcfd701007b653358703c8fda626498193f5f9f1338770906c7e3bfd386/together-1.3.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8c9ff79e3f4ad1cee6ec2d569437e7b231853c39fafa55b07e7aec8c6ebe199c",
"md5": "a74827f9d4a1d57229a8074b2a26f163",
"sha256": "7dc5fec6746343d1bfcd8db86c33cc74f26d2d6daff2cbae816fcd92f2d89caf"
},
"downloads": -1,
"filename": "together-1.3.5.tar.gz",
"has_sig": false,
"md5_digest": "a74827f9d4a1d57229a8074b2a26f163",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 51349,
"upload_time": "2024-11-19T09:34:20",
"upload_time_iso_8601": "2024-11-19T09:34:20.190008Z",
"url": "https://files.pythonhosted.org/packages/8c/9f/f79e3f4ad1cee6ec2d569437e7b231853c39fafa55b07e7aec8c6ebe199c/together-1.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-19 09:34:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "togethercomputer",
"github_project": "together-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "together"
}
JSON
