llama-index-readers-huggingface-fs


Namellama-index-readers-huggingface-fs JSON
Version 0.3.1 PyPI version JSON
download
home_pageNone
Summaryllama-index readers huggingface fs integration
upload_time2025-01-06 15:30:11
maintainerjerryjliu
docs_urlNone
authorYour Name
requires_python<4.0,>=3.9
licenseMIT
keywords face filesystem fs hugging huggingface
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Hugging Face FS Loader

```bash
pip install llama-index-readers-huggingface-fs
```

This loader uses Hugging Face Hub's Filesystem API (> 0.14) to
load datasets.

Besides the existing `load_data` function, you may also choose to use
`load_dicts` and `load_df`.

## Usage

To use this loader, you need to pass in a path to a Hugging Face dataset.

```python
from pathlib import Path

from llama_index.readers.huggingface_fs import HuggingFaceFSReader

# load documents
loader = HuggingFaceFSReader()
documents = loader.load_data("datasets/dair-ai/emotion/data/data.jsonl.gz")

# load dicts
dicts = loader.load_dicts("datasets/dair-ai/emotion/data/data.jsonl.gz")

# load df
df = loader.load_df("datasets/dair-ai/emotion/data/data.jsonl.gz")
```

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "llama-index-readers-huggingface-fs",
    "maintainer": "jerryjliu",
    "docs_url": null,
    "requires_python": "<4.0,>=3.9",
    "maintainer_email": null,
    "keywords": "face, filesystem, fs, hugging, huggingface",
    "author": "Your Name",
    "author_email": "you@example.com",
    "download_url": "https://files.pythonhosted.org/packages/56/d0/af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625/llama_index_readers_huggingface_fs-0.3.1.tar.gz",
    "platform": null,
    "description": "# Hugging Face FS Loader\n\n```bash\npip install llama-index-readers-huggingface-fs\n```\n\nThis loader uses Hugging Face Hub's Filesystem API (> 0.14) to\nload datasets.\n\nBesides the existing `load_data` function, you may also choose to use\n`load_dicts` and `load_df`.\n\n## Usage\n\nTo use this loader, you need to pass in a path to a Hugging Face dataset.\n\n```python\nfrom pathlib import Path\n\nfrom llama_index.readers.huggingface_fs import HuggingFaceFSReader\n\n# load documents\nloader = HuggingFaceFSReader()\ndocuments = loader.load_data(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n\n# load dicts\ndicts = loader.load_dicts(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n\n# load df\ndf = loader.load_df(\"datasets/dair-ai/emotion/data/data.jsonl.gz\")\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "llama-index readers huggingface fs integration",
    "version": "0.3.1",
    "project_urls": null,
    "split_keywords": [
        "face",
        " filesystem",
        " fs",
        " hugging",
        " huggingface"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "6936e9ecb7e03bd3f3e82ee1b8261c82626e673faded91f635e8297bd3fd7f86",
                "md5": "41a48f620f7b4c5e9c9142977fd31bbc",
                "sha256": "cead56b53b9da4f7850a40de0eb40d5bdad6565bef319ad056444765502eb57d"
            },
            "downloads": -1,
            "filename": "llama_index_readers_huggingface_fs-0.3.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "41a48f620f7b4c5e9c9142977fd31bbc",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.9",
            "size": 2776,
            "upload_time": "2025-01-06T15:30:10",
            "upload_time_iso_8601": "2025-01-06T15:30:10.779834Z",
            "url": "https://files.pythonhosted.org/packages/69/36/e9ecb7e03bd3f3e82ee1b8261c82626e673faded91f635e8297bd3fd7f86/llama_index_readers_huggingface_fs-0.3.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "56d0af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625",
                "md5": "abc01bf55a5e52e7a229dfd8cf9275cd",
                "sha256": "9883f3e7b4f6b3d41ff58e9f7095fb4e32d1317e1619b38c988a1d67f8297f01"
            },
            "downloads": -1,
            "filename": "llama_index_readers_huggingface_fs-0.3.1.tar.gz",
            "has_sig": false,
            "md5_digest": "abc01bf55a5e52e7a229dfd8cf9275cd",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.9",
            "size": 2434,
            "upload_time": "2025-01-06T15:30:11",
            "upload_time_iso_8601": "2025-01-06T15:30:11.599619Z",
            "url": "https://files.pythonhosted.org/packages/56/d0/af725946f9cd52df06da7546eefe5692ad9bf166281e1670d9229692e625/llama_index_readers_huggingface_fs-0.3.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-06 15:30:11",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "llama-index-readers-huggingface-fs"
}
        
Elapsed time: 0.37702s