# Py-TXI (previously Py-TGI)
[](https://badge.fury.io/py/py-txi)
[](https://pypi.org/project/py-txi/)
[](https://pypi.org/project/py-txi/)
[](https://pepy.tech/project/py-txi)
[](https://pypi.org/project/py-txi/)
[](https://github.com/IlyasMoutawwakil/py-txi/actions/workflows/tests.yaml)
Py-TXI is a Python wrapper around [Text-Generation-Inference](https://github.com/huggingface/text-generation-inference) and [Text-Embedding-Inference](https://github.com/huggingface/text-embeddings-inference) that enables creating and running TGI/TEI instances through the awesome `docker-py` in a similar style to Transformers API.
## Installation
```bash
pip install py-txi
```
Py-TXI is designed to be used in a similar way to Transformers API. We use `docker-py` (instead of a dirty `subprocess` solution) so that the containers you run are linked to the main process and are stopped automatically when your code finishes or fails.
## Advantages
- **Easy to use**: Py-TXI is designed to be used in a similar way to Transformers API.
- **Automatic cleanup**: Py-TXI stops the Docker container when your code finishes or fails.
- **Batched inference**: Py-TXI supports sending a batch of inputs to the server for inference.
- **Automatic port allocation**: Py-TXI automatically allocates a free port for the Inference server.
- **Configurable**: Py-TXI allows you to configure the Inference servers using a simple configuration object.
- **Verbose**: Py-TXI streams the logs of the underlying Docker container to the main process so you can debug easily.
## Usage
Here's an example of how to use it:
```python
from py_txi import TGI, TGIConfig
llm = TGI(config=TGIConfig(model_id="bigscience/bloom-560m", gpus="0"))
output = llm.generate(["Hi, I'm a language model", "I'm fine, how are you?"])
print("LLM:", output)
llm.close()
```
Output: ```LLM: [' student. I have a problem with the following code. I have a class that has a method that', '"\n\n"I\'m fine," said the girl, "but I don\'t want to be alone.']```
```python
from py_txi import TEI, TEIConfig
embed = TEI(config=TEIConfig(model_id="BAAI/bge-base-en-v1.5"))
output = embed.encode(["Hi, I'm an embedding model", "I'm fine, how are you?"])
print("Embed:", output)
embed.close()
```
Output: ```[array([[ 0.01058742, -0.01588806, -0.03487622, ..., -0.01613717,
0.01772875, -0.02237891]], dtype=float32), array([[ 0.02815401, -0.02892136, -0.0536355 , ..., 0.01225784,
-0.00241452, -0.02836569]], dtype=float32)]```
That's it! Now you can write your Python scripts using the power of TGI and TEI without having to worry about the underlying Docker containers.
Raw data
{
"_id": null,
"home_page": "https://github.com/IlyasMoutawwakil/py-txi",
"name": "py-txi",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "tgi, llm, tei, embedding, huggingface, docker, python",
"author": "Ilyas Moutawwakil",
"author_email": "ilyas.moutawwakil@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3d/36/901617f7901d8eb539f9f7d1dfc70b2d1dcf01a4828cde6bb7b331cfe792/py-txi-0.7.0.tar.gz",
"platform": "linux",
"description": "# Py-TXI (previously Py-TGI)\n\n[](https://badge.fury.io/py/py-txi)\n[](https://pypi.org/project/py-txi/)\n[](https://pypi.org/project/py-txi/)\n[](https://pepy.tech/project/py-txi)\n[](https://pypi.org/project/py-txi/)\n[](https://github.com/IlyasMoutawwakil/py-txi/actions/workflows/tests.yaml)\n\nPy-TXI is a Python wrapper around [Text-Generation-Inference](https://github.com/huggingface/text-generation-inference) and [Text-Embedding-Inference](https://github.com/huggingface/text-embeddings-inference) that enables creating and running TGI/TEI instances through the awesome `docker-py` in a similar style to Transformers API.\n\n## Installation\n\n```bash\npip install py-txi\n```\n\nPy-TXI is designed to be used in a similar way to Transformers API. We use `docker-py` (instead of a dirty `subprocess` solution) so that the containers you run are linked to the main process and are stopped automatically when your code finishes or fails.\n\n## Advantages\n\n- **Easy to use**: Py-TXI is designed to be used in a similar way to Transformers API.\n- **Automatic cleanup**: Py-TXI stops the Docker container when your code finishes or fails.\n- **Batched inference**: Py-TXI supports sending a batch of inputs to the server for inference.\n- **Automatic port allocation**: Py-TXI automatically allocates a free port for the Inference server.\n- **Configurable**: Py-TXI allows you to configure the Inference servers using a simple configuration object.\n- **Verbose**: Py-TXI streams the logs of the underlying Docker container to the main process so you can debug easily.\n\n## Usage\n\nHere's an example of how to use it:\n\n```python\nfrom py_txi import TGI, TGIConfig\n\nllm = TGI(config=TGIConfig(model_id=\"bigscience/bloom-560m\", gpus=\"0\"))\noutput = llm.generate([\"Hi, I'm a language model\", \"I'm fine, how are you?\"])\nprint(\"LLM:\", output)\nllm.close()\n```\n\nOutput: ```LLM: [' student. I have a problem with the following code. I have a class that has a method that', '\"\\n\\n\"I\\'m fine,\" said the girl, \"but I don\\'t want to be alone.']```\n\n```python\nfrom py_txi import TEI, TEIConfig\n\nembed = TEI(config=TEIConfig(model_id=\"BAAI/bge-base-en-v1.5\"))\noutput = embed.encode([\"Hi, I'm an embedding model\", \"I'm fine, how are you?\"])\nprint(\"Embed:\", output)\nembed.close()\n```\n\nOutput: ```[array([[ 0.01058742, -0.01588806, -0.03487622, ..., -0.01613717,\n 0.01772875, -0.02237891]], dtype=float32), array([[ 0.02815401, -0.02892136, -0.0536355 , ..., 0.01225784,\n -0.00241452, -0.02836569]], dtype=float32)]```\n\nThat's it! Now you can write your Python scripts using the power of TGI and TEI without having to worry about the underlying Docker containers.\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python wrapper around TGI and TEI servers",
"version": "0.7.0",
"project_urls": {
"Homepage": "https://github.com/IlyasMoutawwakil/py-txi"
},
"split_keywords": [
"tgi",
" llm",
" tei",
" embedding",
" huggingface",
" docker",
" python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dfa3b9de73b8f8a2858572bbae76deb734b0beac3ef261839d9fd742cb95697d",
"md5": "116417641fa0f666694f61a57e22be7a",
"sha256": "90cfb3db85f9cfc9d352380f21af0177422131fdfaa8cb64039cb051d75674f4"
},
"downloads": -1,
"filename": "py_txi-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "116417641fa0f666694f61a57e22be7a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 12496,
"upload_time": "2024-04-13T12:18:48",
"upload_time_iso_8601": "2024-04-13T12:18:48.286271Z",
"url": "https://files.pythonhosted.org/packages/df/a3/b9de73b8f8a2858572bbae76deb734b0beac3ef261839d9fd742cb95697d/py_txi-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3d36901617f7901d8eb539f9f7d1dfc70b2d1dcf01a4828cde6bb7b331cfe792",
"md5": "c035af0d1e7d1ca72a8af0b064305530",
"sha256": "7db63b25eb278a85dcdd460ecfb7f86f13bc69fddc6b067ee0c458ff708664a0"
},
"downloads": -1,
"filename": "py-txi-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "c035af0d1e7d1ca72a8af0b064305530",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11209,
"upload_time": "2024-04-13T12:18:50",
"upload_time_iso_8601": "2024-04-13T12:18:50.006568Z",
"url": "https://files.pythonhosted.org/packages/3d/36/901617f7901d8eb539f9f7d1dfc70b2d1dcf01a4828cde6bb7b331cfe792/py-txi-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-13 12:18:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "IlyasMoutawwakil",
"github_project": "py-txi",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "py-txi"
}
JSON
