# Py-TXI
[](https://badge.fury.io/py/py-txi)
[](https://pypi.org/project/py-txi/)
[](https://pypi.org/project/py-txi/)
[](https://pepy.tech/project/py-txi)
[](https://pypi.org/project/py-txi/)
[](https://github.com/IlyasMoutawwakil/py-txi/actions/workflows/tests.yaml)
Py-TXI is a Python wrapper around [Text-Generation-Inference](https://github.com/huggingface/text-generation-inference) and [Text-Embedding-Inference](https://github.com/huggingface/text-embeddings-inference) that enables creating and running TGI/TEI instances through the awesome `docker-py` in a similar style to Transformers API.
## Installation
```bash
pip install py-txi
```
Py-TXI is designed to be used in a similar way to Transformers API. We use `docker-py` (instead of a dirty `subprocess` solution) so that the containers you run are linked to the main process and are stopped automatically when your code finishes or fails.
## Advantages
- **Easy to use**: Py-TXI is designed to be used in a similar way to Transformers API.
- **Automatic cleanup**: Py-TXI stops the Docker container when your code finishes or fails.
- **Batched inference**: Py-TXI supports sending a batch of inputs to the server for inference.
- **Automatic port allocation**: Py-TXI automatically allocates a free port for the Inference server.
- **Configurable**: Py-TXI allows you to configure the Inference servers using a simple configuration object.
- **Verbose**: Py-TXI streams the logs of the underlying Docker container to the main process so you can debug easily.
## Usage
Here's an example of how to use it:
```python
from py_txi import TGI, TGIConfig
llm = TGI(config=TGIConfig(model_id="bigscience/bloom-560m", gpus="0"))
output = llm.generate(["Hi, I'm a language model", "I'm fine, how are you?"])
print("LLM:", output)
llm.close()
```
Output: ```LLM: [' student. I have a problem with the following code. I have a class that has a method that', '"\n\n"I\'m fine," said the girl, "but I don\'t want to be alone.']```
```python
from py_txi import TEI, TEIConfig
embed = TEI(config=TEIConfig(model_id="BAAI/bge-base-en-v1.5"))
output = embed.encode(["Hi, I'm an embedding model", "I'm fine, how are you?"])
print("Embed:", output)
embed.close()
```
Output: ```[array([[ 0.01058742, -0.01588806, -0.03487622, ..., -0.01613717,
0.01772875, -0.02237891]], dtype=float32), array([[ 0.02815401, -0.02892136, -0.0536355 , ..., 0.01225784,
-0.00241452, -0.02836569]], dtype=float32)]```
That's it! Now you can write your Python scripts using the power of TGI and TEI without having to worry about the underlying Docker containers.
Raw data
{
"_id": null,
"home_page": "https://github.com/IlyasMoutawwakil/py-txi",
"name": "py-txi",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "tgi, llm, tei, embedding, huggingface, docker, python",
"author": "Ilyas Moutawwakil",
"author_email": "ilyas.moutawwakil@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/10/19/e5e26026b1fad8e129ff414a4e6dbee7a7d4f5b75d96c6628102c3519b65/py-txi-0.9.0.tar.gz",
"platform": "linux",
"description": "# Py-TXI\n\n[](https://badge.fury.io/py/py-txi)\n[](https://pypi.org/project/py-txi/)\n[](https://pypi.org/project/py-txi/)\n[](https://pepy.tech/project/py-txi)\n[](https://pypi.org/project/py-txi/)\n[](https://github.com/IlyasMoutawwakil/py-txi/actions/workflows/tests.yaml)\n\nPy-TXI is a Python wrapper around [Text-Generation-Inference](https://github.com/huggingface/text-generation-inference) and [Text-Embedding-Inference](https://github.com/huggingface/text-embeddings-inference) that enables creating and running TGI/TEI instances through the awesome `docker-py` in a similar style to Transformers API.\n\n## Installation\n\n```bash\npip install py-txi\n```\n\nPy-TXI is designed to be used in a similar way to Transformers API. We use `docker-py` (instead of a dirty `subprocess` solution) so that the containers you run are linked to the main process and are stopped automatically when your code finishes or fails.\n\n## Advantages\n\n- **Easy to use**: Py-TXI is designed to be used in a similar way to Transformers API.\n- **Automatic cleanup**: Py-TXI stops the Docker container when your code finishes or fails.\n- **Batched inference**: Py-TXI supports sending a batch of inputs to the server for inference.\n- **Automatic port allocation**: Py-TXI automatically allocates a free port for the Inference server.\n- **Configurable**: Py-TXI allows you to configure the Inference servers using a simple configuration object.\n- **Verbose**: Py-TXI streams the logs of the underlying Docker container to the main process so you can debug easily.\n\n## Usage\n\nHere's an example of how to use it:\n\n```python\nfrom py_txi import TGI, TGIConfig\n\nllm = TGI(config=TGIConfig(model_id=\"bigscience/bloom-560m\", gpus=\"0\"))\noutput = llm.generate([\"Hi, I'm a language model\", \"I'm fine, how are you?\"])\nprint(\"LLM:\", output)\nllm.close()\n```\n\nOutput: ```LLM: [' student. I have a problem with the following code. I have a class that has a method that', '\"\\n\\n\"I\\'m fine,\" said the girl, \"but I don\\'t want to be alone.']```\n\n```python\nfrom py_txi import TEI, TEIConfig\n\nembed = TEI(config=TEIConfig(model_id=\"BAAI/bge-base-en-v1.5\"))\noutput = embed.encode([\"Hi, I'm an embedding model\", \"I'm fine, how are you?\"])\nprint(\"Embed:\", output)\nembed.close()\n```\n\nOutput: ```[array([[ 0.01058742, -0.01588806, -0.03487622, ..., -0.01613717,\n 0.01772875, -0.02237891]], dtype=float32), array([[ 0.02815401, -0.02892136, -0.0536355 , ..., 0.01225784,\n -0.00241452, -0.02836569]], dtype=float32)]```\n\nThat's it! Now you can write your Python scripts using the power of TGI and TEI without having to worry about the underlying Docker containers.\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python wrapper around TGI and TEI servers",
"version": "0.9.0",
"project_urls": {
"Homepage": "https://github.com/IlyasMoutawwakil/py-txi"
},
"split_keywords": [
"tgi",
" llm",
" tei",
" embedding",
" huggingface",
" docker",
" python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c8f9f2aa9f2796ebcfc73fa218ad40e95565da0c21a07b17f13a1b6acf87e83d",
"md5": "bd2687e2baff0397a8d20b0c154becff",
"sha256": "f9afa42f7b062e65d9229cd01d4401ae17db2fe006eaebe0f4e48c0f96354522"
},
"downloads": -1,
"filename": "py_txi-0.9.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bd2687e2baff0397a8d20b0c154becff",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 12502,
"upload_time": "2024-07-15T09:05:31",
"upload_time_iso_8601": "2024-07-15T09:05:31.523231Z",
"url": "https://files.pythonhosted.org/packages/c8/f9/f2aa9f2796ebcfc73fa218ad40e95565da0c21a07b17f13a1b6acf87e83d/py_txi-0.9.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1019e5e26026b1fad8e129ff414a4e6dbee7a7d4f5b75d96c6628102c3519b65",
"md5": "6011e75c64740b7cd94fbd2a81c276a6",
"sha256": "dc3c9ab2b89155b54728ddb5733f3ee745ca42858a87f11d351b24bdc858412f"
},
"downloads": -1,
"filename": "py-txi-0.9.0.tar.gz",
"has_sig": false,
"md5_digest": "6011e75c64740b7cd94fbd2a81c276a6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11208,
"upload_time": "2024-07-15T09:05:49",
"upload_time_iso_8601": "2024-07-15T09:05:49.811122Z",
"url": "https://files.pythonhosted.org/packages/10/19/e5e26026b1fad8e129ff414a4e6dbee7a7d4f5b75d96c6628102c3519b65/py-txi-0.9.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-15 09:05:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "IlyasMoutawwakil",
"github_project": "py-txi",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "py-txi"
}
JSON
