# Hugging Face FS Loader
```bash
pip install llama-index-readers-huggingface-fs
```
This loader uses Hugging Face Hub's Filesystem API (> 0.14) to
load datasets.
Besides the existing `load_data` function, you may also choose to use
`load_dicts` and `load_df`.
## Usage
To use this loader, you need to pass in a path to a Hugging Face dataset.
```python
from pathlib import Path
from llama_index.readers.huggingface_fs import HuggingFaceFSReader
# load documents
loader = HuggingFaceFSReader()
documents = loader.load_data("datasets/dair-ai/emotion/data/data.jsonl.gz")
# load dicts
dicts = loader.load_dicts("datasets/dair-ai/emotion/data/data.jsonl.gz")
# load df
df = loader.load_df("datasets/dair-ai/emotion/data/data.jsonl.gz")
```
This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-readers-huggingface-fs",
"maintainer": "jerryjliu",
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "face, filesystem, fs, hugging, huggingface",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/56/d0/af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625/llama_index_readers_huggingface_fs-0.3.1.tar.gz",
"platform": null,
"description": "# Hugging Face FS Loader\n\n```bash\npip install llama-index-readers-huggingface-fs\n```\n\nThis loader uses Hugging Face Hub's Filesystem API (> 0.14) to\nload datasets.\n\nBesides the existing `load_data` function, you may also choose to use\n`load_dicts` and `load_df`.\n\n## Usage\n\nTo use this loader, you need to pass in a path to a Hugging Face dataset.\n\n```python\nfrom pathlib import Path\n\nfrom llama_index.readers.huggingface_fs import HuggingFaceFSReader\n\n# load documents\nloader = HuggingFaceFSReader()\ndocuments = loader.load_data(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n\n# load dicts\ndicts = loader.load_dicts(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n\n# load df\ndf = loader.load_df(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index readers huggingface fs integration",
"version": "0.3.1",
"project_urls": null,
"split_keywords": [
"face",
" filesystem",
" fs",
" hugging",
" huggingface"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6936e9ecb7e03bd3f3e82ee1b8261c82626e673faded91f635e8297bd3fd7f86",
"md5": "41a48f620f7b4c5e9c9142977fd31bbc",
"sha256": "cead56b53b9da4f7850a40de0eb40d5bdad6565bef319ad056444765502eb57d"
},
"downloads": -1,
"filename": "llama_index_readers_huggingface_fs-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "41a48f620f7b4c5e9c9142977fd31bbc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 2776,
"upload_time": "2025-01-06T15:30:10",
"upload_time_iso_8601": "2025-01-06T15:30:10.779834Z",
"url": "https://files.pythonhosted.org/packages/69/36/e9ecb7e03bd3f3e82ee1b8261c82626e673faded91f635e8297bd3fd7f86/llama_index_readers_huggingface_fs-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "56d0af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625",
"md5": "abc01bf55a5e52e7a229dfd8cf9275cd",
"sha256": "9883f3e7b4f6b3d41ff58e9f7095fb4e32d1317e1619b38c988a1d67f8297f01"
},
"downloads": -1,
"filename": "llama_index_readers_huggingface_fs-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "abc01bf55a5e52e7a229dfd8cf9275cd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 2434,
"upload_time": "2025-01-06T15:30:11",
"upload_time_iso_8601": "2025-01-06T15:30:11.599619Z",
"url": "https://files.pythonhosted.org/packages/56/d0/af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625/llama_index_readers_huggingface_fs-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-06 15:30:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-readers-huggingface-fs"
}
JSON