# langchain_bytez
This package allows you to use the Bytez API in langchain. Note, only text-generation, chat, image-text-to-text, video-text-to-text, and audio-text-to-text are currently supported.
Fully supports streaming + native async!
Curious about what else Bytez has to offer? You can check out Bytez [here](https://bytez.com).
Want to know more about our API? Check out the [docs](https://docs.bytez.com)!
# Chat Example
```py
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage
from langchain_bytez import BytezChatModel
API_KEY = os.environ.get("API_KEY")
bytez_chat_model_phi = BytezChatModel(
model_id="microsoft/Phi-3-mini-4k-instruct",
api_key=API_KEY,
capacity={
"min": 1,
"max": 1, # up to 10 instances
},
params={"max_new_tokens": 64},
timeout=10, # minutes before expiring
streaming=True,
callbacks=[StreamingStdOutCallbackHandler()],
)
messages = [
SystemMessage(
content="You are a helpful assistant that answers questions clearly and concisely."
),
HumanMessage(content="List the phylums in the biological taxonomy"),
]
results = bytez_chat_model_phi.invoke(messages)
```
# Text generation (LLM) example
```python
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage
from langchain_bytez import BytezLLM
API_KEY = os.environ.get("API_KEY")
bytez_chat_model_phi = BytezLLM(
model_id="microsoft/phi-2",
api_key=API_KEY,
capacity={
"min": 1,
"max": 1, # up to 10 instances
},
params={"max_new_tokens": 64},
timeout=10, # minutes before expiring
streaming=True,
callbacks=[StreamingStdOutCallbackHandler()],
)
messages = [
SystemMessage(
content="You are a helpful assistant that answers questions clearly and concisely."
),
HumanMessage(content="List the phylums in the biological taxonomy"),
]
results = bytez_chat_model_phi.invoke(messages)
```
# Extending callback handlers for better observability
_NOTE_ this is experimental and we're working to enhance it. In the meantime it will help bootstrap you in doing whatever you need to do with a model's "run" lifecycle.
```py
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage
from langchain_bytez import BytezChatModel, BytezStdOutCallbackHandler
API_KEY = os.environ.get("API_KEY")
bytez_chat_model_phi = BytezChatModel(
model_id="microsoft/Phi-3-mini-4k-instruct",
api_key=API_KEY,
capacity={
"min": 1,
"max": 1, # up to 10 instances
},
params={"max_new_tokens": 64},
timeout=10, # minutes before expiring
streaming=True,
callbacks=[StreamingStdOutCallbackHandler(), BytezStdOutCallbackHandler()],
)
messages = [
SystemMessage(
content="You are a helpful assistant that answers questions clearly and concisely."
),
HumanMessage(content="List the phylums in the biological taxonomy"),
]
results = bytez_chat_model_phi.invoke(messages)
```
To roll our own implementation that better suites your needs, check out the implementation [here](https://github.com/Bytez-com/langchain_bytez/blob/main/langchain_bytez/BytezStdOutCallbackHandler.py)
# Shutdown your cluster
```py
bytez_chat_model_phi.shutdown_cluster()
```
# Update your cluster
```py
bytez_chat_model_phi.capacity = {
"min": 2, # we've increased the minimum number of instances
"max": 3, # up to 10 instances
}
bytez_chat_model_phi.update_cluster()
```
# kwargs for BytezChatModel and BytezLLM
```py
model_id: str = Field(..., description="The unique model ID for the Bytez LLM.")
api_key: str = Field(..., description="The API key for accessing the Bytez LLM.")
capacity: dict = Field(
default_factory=dict,
description="Controls the scaling behavior, contains one or all keys 'desired': int, 'min': int, and 'max': int",
)
timeout: int = Field(
None,
description="Controls how many minutes to wait after the last inference to shutdown the cluster",
)
streaming: bool = Field(
False, description="Enable streaming responses from the API."
)
params: dict = Field(
default_factory=dict, description="Parameters passed to the Bytez API."
)
headers: dict = Field(
default_factory=dict,
description="Additional headers for the Bytez API. Matching keys override the defaults.",
)
http_timeout_s: float = Field(
60 * 5.0,
description="How long to wait in seconds for a response from the model before timing out",
)
```
## API Playground
Explore our API endpoints in the documentation [here](https://docs.bytez.com/model-api/playground/overview).
## Status
Check out the status of our [API](https://status.bytez.com)
## Resources
Get to know our story, our mission, and our roadmap [here](https://docs.bytez.com/company/about).
## Feedback
We’re committed to building the best developer experience for AI builders. Have feedback? Let us know on [Discord](https://discord.com/invite/Z723PfCFWf) or open an issue on [GitHub](https://github.com/Bytez-com/docs/issues).
Raw data
{
"_id": null,
"home_page": "https://github.com/Bytez-com/langchain_bytez",
"name": "langchain-bytez-main",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "langchain, bytez, llm, ai, chatbot, artificial-intelligence",
"author": "AIMLStudent",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4d/70/9c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725/langchain_bytez_main-0.0.1.tar.gz",
"platform": null,
"description": "# langchain_bytez\r\n\r\nThis package allows you to use the Bytez API in langchain. Note, only text-generation, chat, image-text-to-text, video-text-to-text, and audio-text-to-text are currently supported.\r\n\r\nFully supports streaming + native async!\r\n\r\nCurious about what else Bytez has to offer? You can check out Bytez [here](https://bytez.com).\r\n\r\nWant to know more about our API? Check out the [docs](https://docs.bytez.com)!\r\n\r\n# Chat Example\r\n\r\n```py\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezChatModel\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezChatModel(\r\n model_id=\"microsoft/Phi-3-mini-4k-instruct\",\r\n api_key=API_KEY,\r\n capacity={\r\n \"min\": 1,\r\n \"max\": 1, # up to 10 instances\r\n },\r\n params={\"max_new_tokens\": 64},\r\n timeout=10, # minutes before expiring\r\n streaming=True,\r\n callbacks=[StreamingStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n SystemMessage(\r\n content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n ),\r\n HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n```\r\n\r\n# Text generation (LLM) example\r\n\r\n```python\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezLLM\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezLLM(\r\n model_id=\"microsoft/phi-2\",\r\n api_key=API_KEY,\r\n capacity={\r\n \"min\": 1,\r\n \"max\": 1, # up to 10 instances\r\n },\r\n params={\"max_new_tokens\": 64},\r\n timeout=10, # minutes before expiring\r\n streaming=True,\r\n callbacks=[StreamingStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n SystemMessage(\r\n content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n ),\r\n HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n```\r\n\r\n# Extending callback handlers for better observability\r\n\r\n_NOTE_ this is experimental and we're working to enhance it. In the meantime it will help bootstrap you in doing whatever you need to do with a model's \"run\" lifecycle.\r\n\r\n```py\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezChatModel, BytezStdOutCallbackHandler\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezChatModel(\r\n model_id=\"microsoft/Phi-3-mini-4k-instruct\",\r\n api_key=API_KEY,\r\n capacity={\r\n \"min\": 1,\r\n \"max\": 1, # up to 10 instances\r\n },\r\n params={\"max_new_tokens\": 64},\r\n timeout=10, # minutes before expiring\r\n streaming=True,\r\n callbacks=[StreamingStdOutCallbackHandler(), BytezStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n SystemMessage(\r\n content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n ),\r\n HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n\r\n```\r\n\r\nTo roll our own implementation that better suites your needs, check out the implementation [here](https://github.com/Bytez-com/langchain_bytez/blob/main/langchain_bytez/BytezStdOutCallbackHandler.py)\r\n\r\n# Shutdown your cluster\r\n\r\n```py\r\nbytez_chat_model_phi.shutdown_cluster()\r\n```\r\n\r\n# Update your cluster\r\n\r\n```py\r\nbytez_chat_model_phi.capacity = {\r\n \"min\": 2, # we've increased the minimum number of instances\r\n \"max\": 3, # up to 10 instances\r\n}\r\n\r\nbytez_chat_model_phi.update_cluster()\r\n```\r\n\r\n# kwargs for BytezChatModel and BytezLLM\r\n\r\n```py\r\nmodel_id: str = Field(..., description=\"The unique model ID for the Bytez LLM.\")\r\napi_key: str = Field(..., description=\"The API key for accessing the Bytez LLM.\")\r\ncapacity: dict = Field(\r\n default_factory=dict,\r\n description=\"Controls the scaling behavior, contains one or all keys 'desired': int, 'min': int, and 'max': int\",\r\n)\r\ntimeout: int = Field(\r\n None,\r\n description=\"Controls how many minutes to wait after the last inference to shutdown the cluster\",\r\n)\r\nstreaming: bool = Field(\r\n False, description=\"Enable streaming responses from the API.\"\r\n)\r\nparams: dict = Field(\r\n default_factory=dict, description=\"Parameters passed to the Bytez API.\"\r\n)\r\nheaders: dict = Field(\r\n default_factory=dict,\r\n description=\"Additional headers for the Bytez API. Matching keys override the defaults.\",\r\n)\r\nhttp_timeout_s: float = Field(\r\n 60 * 5.0,\r\n description=\"How long to wait in seconds for a response from the model before timing out\",\r\n)\r\n```\r\n\r\n## API Playground\r\n\r\nExplore our API endpoints in the documentation [here](https://docs.bytez.com/model-api/playground/overview).\r\n\r\n## Status\r\n\r\nCheck out the status of our [API](https://status.bytez.com)\r\n\r\n## Resources\r\n\r\nGet to know our story, our mission, and our roadmap [here](https://docs.bytez.com/company/about).\r\n\r\n## Feedback\r\n\r\nWe\u2019re committed to building the best developer experience for AI builders. Have feedback? Let us know on [Discord](https://discord.com/invite/Z723PfCFWf) or open an issue on [GitHub](https://github.com/Bytez-com/docs/issues).\r\n",
"bugtrack_url": null,
"license": null,
"summary": "Bytez langchain integration with advanced AI capabilities",
"version": "0.0.1",
"project_urls": {
"Bug Reports": "https://github.com/Bytez-com/langchain_bytez/issues",
"Documentation": "https://github.com/Bytez-com/langchain_bytez/blob/main/README.md",
"Homepage": "https://github.com/Bytez-com/langchain_bytez",
"Source": "https://github.com/Bytez-com/langchain_bytez"
},
"split_keywords": [
"langchain",
" bytez",
" llm",
" ai",
" chatbot",
" artificial-intelligence"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "cc07d0ff00b71c4a63bcc9769781a1ec5b9e308a254cead34c35c22e77ed3a26",
"md5": "efa1b7a7c97180f6b310260848a0cf0e",
"sha256": "c8e7968ae4a29a888bdf39f14d7e9548c3deb69913bafdd7f41c5ca9282b0d1d"
},
"downloads": -1,
"filename": "langchain_bytez_main-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "efa1b7a7c97180f6b310260848a0cf0e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 12757,
"upload_time": "2025-07-28T12:43:59",
"upload_time_iso_8601": "2025-07-28T12:43:59.546258Z",
"url": "https://files.pythonhosted.org/packages/cc/07/d0ff00b71c4a63bcc9769781a1ec5b9e308a254cead34c35c22e77ed3a26/langchain_bytez_main-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4d709c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725",
"md5": "0c40b1c8863d16183cb1585ae765287c",
"sha256": "50bc6301937f6ab9d9ed6c712f4be68e4271dce6fed26b62046bfb8bd4c74083"
},
"downloads": -1,
"filename": "langchain_bytez_main-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "0c40b1c8863d16183cb1585ae765287c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 12991,
"upload_time": "2025-07-28T12:44:00",
"upload_time_iso_8601": "2025-07-28T12:44:00.792867Z",
"url": "https://files.pythonhosted.org/packages/4d/70/9c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725/langchain_bytez_main-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-28 12:44:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Bytez-com",
"github_project": "langchain_bytez",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "langchain-bytez-main"
}