| Name | llama-index-llms-huggingface JSON |
| Version |
0.4.2
JSON |
| download |
| home_page | None |
| Summary | llama-index llms huggingface integration |
| upload_time | 2025-01-06 15:29:58 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.9 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex Llms Integration: Huggingface
## Installation
1. Install the required Python packages:
```bash
%pip install llama-index-llms-huggingface
%pip install llama-index-llms-huggingface-api
!pip install "transformers[torch]" "huggingface_hub[inference]"
!pip install llama-index
```
2. Set the Hugging Face API token as an environment variable:
```bash
export HUGGING_FACE_TOKEN=your_token_here
```
## Usage
### Import Required Libraries
```python
import os
from typing import List, Optional
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
```
### Run a Model Locally
To run the model locally on your machine:
```python
locally_run = HuggingFaceLLM(model_name="HuggingFaceH4/zephyr-7b-alpha")
```
### Run a Model Remotely
To run the model remotely using Hugging Face's Inference API:
```python
HF_TOKEN: Optional[str] = os.getenv("HUGGING_FACE_TOKEN")
remotely_run = HuggingFaceInferenceAPI(
model_name="HuggingFaceH4/zephyr-7b-alpha", token=HF_TOKEN
)
```
### Anonymous Remote Execution
You can also use the Inference API anonymously without providing a token:
```python
remotely_run_anon = HuggingFaceInferenceAPI(
model_name="HuggingFaceH4/zephyr-7b-alpha"
)
```
### Use Recommended Model
If you do not provide a model name, Hugging Face's recommended model is used:
```python
remotely_run_recommended = HuggingFaceInferenceAPI(token=HF_TOKEN)
```
### Generate Text Completion
To generate a text completion using the remote model:
```python
completion_response = remotely_run_recommended.complete("To infinity, and")
print(completion_response)
```
### Set Global Tokenizer
If you modify the LLM, ensure you change the global tokenizer to match:
```python
from llama_index.core import set_global_tokenizer
from transformers import AutoTokenizer
set_global_tokenizer(
AutoTokenizer.from_pretrained("HuggingFaceH4/zephyr-7b-alpha").encode
)
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/huggingface/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-huggingface",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/8a/36/bd524e426d3840dcff060465b3277c4d6fced7c7d55ad4df842eef15d43c/llama_index_llms_huggingface-0.4.2.tar.gz",
"platform": null,
"description": "# LlamaIndex Llms Integration: Huggingface\n\n## Installation\n\n1. Install the required Python packages:\n\n ```bash\n %pip install llama-index-llms-huggingface\n %pip install llama-index-llms-huggingface-api\n !pip install \"transformers[torch]\" \"huggingface_hub[inference]\"\n !pip install llama-index\n ```\n\n2. Set the Hugging Face API token as an environment variable:\n\n ```bash\n export HUGGING_FACE_TOKEN=your_token_here\n ```\n\n## Usage\n\n### Import Required Libraries\n\n```python\nimport os\nfrom typing import List, Optional\nfrom llama_index.llms.huggingface import HuggingFaceLLM\nfrom llama_index.llms.huggingface_api import HuggingFaceInferenceAPI\n```\n\n### Run a Model Locally\n\nTo run the model locally on your machine:\n\n```python\nlocally_run = HuggingFaceLLM(model_name=\"HuggingFaceH4/zephyr-7b-alpha\")\n```\n\n### Run a Model Remotely\n\nTo run the model remotely using Hugging Face's Inference API:\n\n```python\nHF_TOKEN: Optional[str] = os.getenv(\"HUGGING_FACE_TOKEN\")\nremotely_run = HuggingFaceInferenceAPI(\n model_name=\"HuggingFaceH4/zephyr-7b-alpha\", token=HF_TOKEN\n)\n```\n\n### Anonymous Remote Execution\n\nYou can also use the Inference API anonymously without providing a token:\n\n```python\nremotely_run_anon = HuggingFaceInferenceAPI(\n model_name=\"HuggingFaceH4/zephyr-7b-alpha\"\n)\n```\n\n### Use Recommended Model\n\nIf you do not provide a model name, Hugging Face's recommended model is used:\n\n```python\nremotely_run_recommended = HuggingFaceInferenceAPI(token=HF_TOKEN)\n```\n\n### Generate Text Completion\n\nTo generate a text completion using the remote model:\n\n```python\ncompletion_response = remotely_run_recommended.complete(\"To infinity, and\")\nprint(completion_response)\n```\n\n### Set Global Tokenizer\n\nIf you modify the LLM, ensure you change the global tokenizer to match:\n\n```python\nfrom llama_index.core import set_global_tokenizer\nfrom transformers import AutoTokenizer\n\nset_global_tokenizer(\n AutoTokenizer.from_pretrained(\"HuggingFaceH4/zephyr-7b-alpha\").encode\n)\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/huggingface/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms huggingface integration",
"version": "0.4.2",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3c700d01d39056d1ee472b5621c065e3cdba25ef888b044290d0373b86a68f60",
"md5": "8434dde7267401827e4ae5ff03abdb0d",
"sha256": "6e9126ebcfa649882e2e6c19d356aaad855d52a9b485ba710b0926fd30a73a36"
},
"downloads": -1,
"filename": "llama_index_llms_huggingface-0.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8434dde7267401827e4ae5ff03abdb0d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 11554,
"upload_time": "2025-01-06T15:29:54",
"upload_time_iso_8601": "2025-01-06T15:29:54.211011Z",
"url": "https://files.pythonhosted.org/packages/3c/70/0d01d39056d1ee472b5621c065e3cdba25ef888b044290d0373b86a68f60/llama_index_llms_huggingface-0.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8a36bd524e426d3840dcff060465b3277c4d6fced7c7d55ad4df842eef15d43c",
"md5": "8614ce02a188d4d03f30af67ffac74db",
"sha256": "b4e5381d7f8cf115f766a8c4d22d35b774b9b01a741c69047b91def08e7da6e2"
},
"downloads": -1,
"filename": "llama_index_llms_huggingface-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "8614ce02a188d4d03f30af67ffac74db",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 11405,
"upload_time": "2025-01-06T15:29:58",
"upload_time_iso_8601": "2025-01-06T15:29:58.342475Z",
"url": "https://files.pythonhosted.org/packages/8a/36/bd524e426d3840dcff060465b3277c4d6fced7c7d55ad4df842eef15d43c/llama_index_llms_huggingface-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-06 15:29:58",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-huggingface"
}
