# S3 File or Directory Loader
This loader parses any file stored on S3, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `S3Reader`, you may pass in your [AWS Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). If none are found, the loader assumes they are stored in `~/.aws/credentials`.
All files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!
## Installation
```bash
pip install llama-index-readers-s3
```
## Usage
To use this loader, you need to pass in the name of your S3 Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.
Otherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. AWS Access Key credentials may either be passed in during initialization or stored locally (see above).
```python
loader = S3Reader(
bucket="scrabble-dictionary",
key="dictionary.txt",
aws_access_id="[ACCESS_KEY_ID]",
aws_access_secret="[ACCESS_KEY_SECRET]",
)
documents = loader.load_data()
```
This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-readers-s3",
"maintainer": "thejessezhang",
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "amazon web services, aws s3, bucket",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/97/60/535badd68985cf42c73892d0bcfb6523987766398b4e64e947ceac20b16d/llama_index_readers_s3-0.4.0.tar.gz",
"platform": null,
"description": "# S3 File or Directory Loader\n\nThis loader parses any file stored on S3, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `S3Reader`, you may pass in your [AWS Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). If none are found, the loader assumes they are stored in `~/.aws/credentials`.\n\nAll files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!\n\n## Installation\n\n```bash\npip install llama-index-readers-s3\n```\n\n## Usage\n\nTo use this loader, you need to pass in the name of your S3 Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.\n\nOtherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. AWS Access Key credentials may either be passed in during initialization or stored locally (see above).\n\n```python\nloader = S3Reader(\n bucket=\"scrabble-dictionary\",\n key=\"dictionary.txt\",\n aws_access_id=\"[ACCESS_KEY_ID]\",\n aws_access_secret=\"[ACCESS_KEY_SECRET]\",\n)\ndocuments = loader.load_data()\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index readers s3 integration",
"version": "0.4.0",
"project_urls": null,
"split_keywords": [
"amazon web services",
" aws s3",
" bucket"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cfc2b77c91fcf7aca89fe9d16e2a3f80cd261a1037eb99814fea64e7fe1a305e",
"md5": "385c707aad5911560f5c3af4bd2ae437",
"sha256": "e5a69b1a335019a7067e52defc0c0a0c7a03a8758315352cc032a3afff8a72c1"
},
"downloads": -1,
"filename": "llama_index_readers_s3-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "385c707aad5911560f5c3af4bd2ae437",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 4448,
"upload_time": "2024-11-18T01:01:53",
"upload_time_iso_8601": "2024-11-18T01:01:53.751884Z",
"url": "https://files.pythonhosted.org/packages/cf/c2/b77c91fcf7aca89fe9d16e2a3f80cd261a1037eb99814fea64e7fe1a305e/llama_index_readers_s3-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9760535badd68985cf42c73892d0bcfb6523987766398b4e64e947ceac20b16d",
"md5": "0898246eb54de75104a4af059a381c86",
"sha256": "a025e92d9c88ad928d34383c864121e71c5712acfb07584b8c4e4e530108a338"
},
"downloads": -1,
"filename": "llama_index_readers_s3-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "0898246eb54de75104a4af059a381c86",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 4330,
"upload_time": "2024-11-18T01:01:54",
"upload_time_iso_8601": "2024-11-18T01:01:54.632367Z",
"url": "https://files.pythonhosted.org/packages/97/60/535badd68985cf42c73892d0bcfb6523987766398b4e64e947ceac20b16d/llama_index_readers_s3-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 01:01:54",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-readers-s3"
}
JSON