langchain-bytez-main


Namelangchain-bytez-main JSON
Version 0.0.1 PyPI version JSON
download
home_pagehttps://github.com/Bytez-com/langchain_bytez
SummaryBytez langchain integration with advanced AI capabilities
upload_time2025-07-28 12:44:00
maintainerNone
docs_urlNone
authorAIMLStudent
requires_python>=3.8
licenseNone
keywords langchain bytez llm ai chatbot artificial-intelligence
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # langchain_bytez

This package allows you to use the Bytez API in langchain. Note, only text-generation, chat, image-text-to-text, video-text-to-text, and audio-text-to-text are currently supported.

Fully supports streaming + native async!

Curious about what else Bytez has to offer? You can check out Bytez [here](https://bytez.com).

Want to know more about our API? Check out the [docs](https://docs.bytez.com)!

# Chat Example

```py
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage

from langchain_bytez import BytezChatModel

API_KEY = os.environ.get("API_KEY")


bytez_chat_model_phi = BytezChatModel(
    model_id="microsoft/Phi-3-mini-4k-instruct",
    api_key=API_KEY,
    capacity={
        "min": 1,
        "max": 1,  # up to 10 instances
    },
    params={"max_new_tokens": 64},
    timeout=10,  # minutes before expiring
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that answers questions clearly and concisely."
    ),
    HumanMessage(content="List the phylums in the biological taxonomy"),
]

results = bytez_chat_model_phi.invoke(messages)
```

# Text generation (LLM) example

```python
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage

from langchain_bytez import BytezLLM

API_KEY = os.environ.get("API_KEY")


bytez_chat_model_phi = BytezLLM(
    model_id="microsoft/phi-2",
    api_key=API_KEY,
    capacity={
        "min": 1,
        "max": 1,  # up to 10 instances
    },
    params={"max_new_tokens": 64},
    timeout=10,  # minutes before expiring
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that answers questions clearly and concisely."
    ),
    HumanMessage(content="List the phylums in the biological taxonomy"),
]

results = bytez_chat_model_phi.invoke(messages)
```

# Extending callback handlers for better observability

_NOTE_ this is experimental and we're working to enhance it. In the meantime it will help bootstrap you in doing whatever you need to do with a model's "run" lifecycle.

```py
import os
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.schema import HumanMessage, SystemMessage

from langchain_bytez import BytezChatModel, BytezStdOutCallbackHandler

API_KEY = os.environ.get("API_KEY")


bytez_chat_model_phi = BytezChatModel(
    model_id="microsoft/Phi-3-mini-4k-instruct",
    api_key=API_KEY,
    capacity={
        "min": 1,
        "max": 1,  # up to 10 instances
    },
    params={"max_new_tokens": 64},
    timeout=10,  # minutes before expiring
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler(), BytezStdOutCallbackHandler()],
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that answers questions clearly and concisely."
    ),
    HumanMessage(content="List the phylums in the biological taxonomy"),
]

results = bytez_chat_model_phi.invoke(messages)

```

To roll our own implementation that better suites your needs, check out the implementation [here](https://github.com/Bytez-com/langchain_bytez/blob/main/langchain_bytez/BytezStdOutCallbackHandler.py)

# Shutdown your cluster

```py
bytez_chat_model_phi.shutdown_cluster()
```

# Update your cluster

```py
bytez_chat_model_phi.capacity = {
    "min": 2,  # we've increased the minimum number of instances
    "max": 3,  # up to 10 instances
}

bytez_chat_model_phi.update_cluster()
```

# kwargs for BytezChatModel and BytezLLM

```py
model_id: str = Field(..., description="The unique model ID for the Bytez LLM.")
api_key: str = Field(..., description="The API key for accessing the Bytez LLM.")
capacity: dict = Field(
    default_factory=dict,
    description="Controls the scaling behavior, contains one or all keys 'desired': int, 'min': int, and 'max': int",
)
timeout: int = Field(
    None,
    description="Controls how many minutes to wait after the last inference to shutdown the cluster",
)
streaming: bool = Field(
    False, description="Enable streaming responses from the API."
)
params: dict = Field(
    default_factory=dict, description="Parameters passed to the Bytez API."
)
headers: dict = Field(
    default_factory=dict,
    description="Additional headers for the Bytez API. Matching keys override the defaults.",
)
http_timeout_s: float = Field(
    60 * 5.0,
    description="How long to wait in seconds for a response from the model before timing out",
)
```

## API Playground

Explore our API endpoints in the documentation [here](https://docs.bytez.com/model-api/playground/overview).

## Status

Check out the status of our [API](https://status.bytez.com)

## Resources

Get to know our story, our mission, and our roadmap [here](https://docs.bytez.com/company/about).

## Feedback

We’re committed to building the best developer experience for AI builders. Have feedback? Let us know on [Discord](https://discord.com/invite/Z723PfCFWf) or open an issue on [GitHub](https://github.com/Bytez-com/docs/issues).

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/Bytez-com/langchain_bytez",
    "name": "langchain-bytez-main",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "langchain, bytez, llm, ai, chatbot, artificial-intelligence",
    "author": "AIMLStudent",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/4d/70/9c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725/langchain_bytez_main-0.0.1.tar.gz",
    "platform": null,
    "description": "# langchain_bytez\r\n\r\nThis package allows you to use the Bytez API in langchain. Note, only text-generation, chat, image-text-to-text, video-text-to-text, and audio-text-to-text are currently supported.\r\n\r\nFully supports streaming + native async!\r\n\r\nCurious about what else Bytez has to offer? You can check out Bytez [here](https://bytez.com).\r\n\r\nWant to know more about our API? Check out the [docs](https://docs.bytez.com)!\r\n\r\n# Chat Example\r\n\r\n```py\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezChatModel\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezChatModel(\r\n    model_id=\"microsoft/Phi-3-mini-4k-instruct\",\r\n    api_key=API_KEY,\r\n    capacity={\r\n        \"min\": 1,\r\n        \"max\": 1,  # up to 10 instances\r\n    },\r\n    params={\"max_new_tokens\": 64},\r\n    timeout=10,  # minutes before expiring\r\n    streaming=True,\r\n    callbacks=[StreamingStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n    SystemMessage(\r\n        content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n    ),\r\n    HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n```\r\n\r\n# Text generation (LLM) example\r\n\r\n```python\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezLLM\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezLLM(\r\n    model_id=\"microsoft/phi-2\",\r\n    api_key=API_KEY,\r\n    capacity={\r\n        \"min\": 1,\r\n        \"max\": 1,  # up to 10 instances\r\n    },\r\n    params={\"max_new_tokens\": 64},\r\n    timeout=10,  # minutes before expiring\r\n    streaming=True,\r\n    callbacks=[StreamingStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n    SystemMessage(\r\n        content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n    ),\r\n    HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n```\r\n\r\n# Extending callback handlers for better observability\r\n\r\n_NOTE_ this is experimental and we're working to enhance it. In the meantime it will help bootstrap you in doing whatever you need to do with a model's \"run\" lifecycle.\r\n\r\n```py\r\nimport os\r\nfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\r\nfrom langchain.schema import HumanMessage, SystemMessage\r\n\r\nfrom langchain_bytez import BytezChatModel, BytezStdOutCallbackHandler\r\n\r\nAPI_KEY = os.environ.get(\"API_KEY\")\r\n\r\n\r\nbytez_chat_model_phi = BytezChatModel(\r\n    model_id=\"microsoft/Phi-3-mini-4k-instruct\",\r\n    api_key=API_KEY,\r\n    capacity={\r\n        \"min\": 1,\r\n        \"max\": 1,  # up to 10 instances\r\n    },\r\n    params={\"max_new_tokens\": 64},\r\n    timeout=10,  # minutes before expiring\r\n    streaming=True,\r\n    callbacks=[StreamingStdOutCallbackHandler(), BytezStdOutCallbackHandler()],\r\n)\r\n\r\nmessages = [\r\n    SystemMessage(\r\n        content=\"You are a helpful assistant that answers questions clearly and concisely.\"\r\n    ),\r\n    HumanMessage(content=\"List the phylums in the biological taxonomy\"),\r\n]\r\n\r\nresults = bytez_chat_model_phi.invoke(messages)\r\n\r\n```\r\n\r\nTo roll our own implementation that better suites your needs, check out the implementation [here](https://github.com/Bytez-com/langchain_bytez/blob/main/langchain_bytez/BytezStdOutCallbackHandler.py)\r\n\r\n# Shutdown your cluster\r\n\r\n```py\r\nbytez_chat_model_phi.shutdown_cluster()\r\n```\r\n\r\n# Update your cluster\r\n\r\n```py\r\nbytez_chat_model_phi.capacity = {\r\n    \"min\": 2,  # we've increased the minimum number of instances\r\n    \"max\": 3,  # up to 10 instances\r\n}\r\n\r\nbytez_chat_model_phi.update_cluster()\r\n```\r\n\r\n# kwargs for BytezChatModel and BytezLLM\r\n\r\n```py\r\nmodel_id: str = Field(..., description=\"The unique model ID for the Bytez LLM.\")\r\napi_key: str = Field(..., description=\"The API key for accessing the Bytez LLM.\")\r\ncapacity: dict = Field(\r\n    default_factory=dict,\r\n    description=\"Controls the scaling behavior, contains one or all keys 'desired': int, 'min': int, and 'max': int\",\r\n)\r\ntimeout: int = Field(\r\n    None,\r\n    description=\"Controls how many minutes to wait after the last inference to shutdown the cluster\",\r\n)\r\nstreaming: bool = Field(\r\n    False, description=\"Enable streaming responses from the API.\"\r\n)\r\nparams: dict = Field(\r\n    default_factory=dict, description=\"Parameters passed to the Bytez API.\"\r\n)\r\nheaders: dict = Field(\r\n    default_factory=dict,\r\n    description=\"Additional headers for the Bytez API. Matching keys override the defaults.\",\r\n)\r\nhttp_timeout_s: float = Field(\r\n    60 * 5.0,\r\n    description=\"How long to wait in seconds for a response from the model before timing out\",\r\n)\r\n```\r\n\r\n## API Playground\r\n\r\nExplore our API endpoints in the documentation [here](https://docs.bytez.com/model-api/playground/overview).\r\n\r\n## Status\r\n\r\nCheck out the status of our [API](https://status.bytez.com)\r\n\r\n## Resources\r\n\r\nGet to know our story, our mission, and our roadmap [here](https://docs.bytez.com/company/about).\r\n\r\n## Feedback\r\n\r\nWe\u2019re committed to building the best developer experience for AI builders. Have feedback? Let us know on [Discord](https://discord.com/invite/Z723PfCFWf) or open an issue on [GitHub](https://github.com/Bytez-com/docs/issues).\r\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "Bytez langchain integration with advanced AI capabilities",
    "version": "0.0.1",
    "project_urls": {
        "Bug Reports": "https://github.com/Bytez-com/langchain_bytez/issues",
        "Documentation": "https://github.com/Bytez-com/langchain_bytez/blob/main/README.md",
        "Homepage": "https://github.com/Bytez-com/langchain_bytez",
        "Source": "https://github.com/Bytez-com/langchain_bytez"
    },
    "split_keywords": [
        "langchain",
        " bytez",
        " llm",
        " ai",
        " chatbot",
        " artificial-intelligence"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "cc07d0ff00b71c4a63bcc9769781a1ec5b9e308a254cead34c35c22e77ed3a26",
                "md5": "efa1b7a7c97180f6b310260848a0cf0e",
                "sha256": "c8e7968ae4a29a888bdf39f14d7e9548c3deb69913bafdd7f41c5ca9282b0d1d"
            },
            "downloads": -1,
            "filename": "langchain_bytez_main-0.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "efa1b7a7c97180f6b310260848a0cf0e",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 12757,
            "upload_time": "2025-07-28T12:43:59",
            "upload_time_iso_8601": "2025-07-28T12:43:59.546258Z",
            "url": "https://files.pythonhosted.org/packages/cc/07/d0ff00b71c4a63bcc9769781a1ec5b9e308a254cead34c35c22e77ed3a26/langchain_bytez_main-0.0.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "4d709c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725",
                "md5": "0c40b1c8863d16183cb1585ae765287c",
                "sha256": "50bc6301937f6ab9d9ed6c712f4be68e4271dce6fed26b62046bfb8bd4c74083"
            },
            "downloads": -1,
            "filename": "langchain_bytez_main-0.0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "0c40b1c8863d16183cb1585ae765287c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 12991,
            "upload_time": "2025-07-28T12:44:00",
            "upload_time_iso_8601": "2025-07-28T12:44:00.792867Z",
            "url": "https://files.pythonhosted.org/packages/4d/70/9c80b263e1b01066c12124b3fb297d4c3c4c235043c2fee1fd9cedfc8725/langchain_bytez_main-0.0.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-07-28 12:44:00",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "Bytez-com",
    "github_project": "langchain_bytez",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "langchain-bytez-main"
}
        
Elapsed time: 1.04362s