# S3 File or Directory Loader
This loader parses any file stored on S3, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `S3Reader`, you may pass in your [AWS Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). If none are found, the loader assumes they are stored in `~/.aws/credentials`.
All files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!
## Installation
```bash
pip install llama-index-readers-s3
```
## Usage
To use this loader, you need to pass in the name of your S3 Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.
Otherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. AWS Access Key credentials may either be passed in during initialization or stored locally (see above).
```python
loader = S3Reader(
bucket="scrabble-dictionary",
key="dictionary.txt",
aws_access_id="[ACCESS_KEY_ID]",
aws_access_secret="[ACCESS_KEY_SECRET]",
)
documents = loader.load_data()
```
This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-readers-s3",
"maintainer": "thejessezhang",
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "amazon web services, aws s3, bucket",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/c1/6c/7479830f9942a6199d09323969d59373306504fb5ec0ede2fab5d0d97a27/llama_index_readers_s3-0.2.0.tar.gz",
"platform": null,
"description": "# S3 File or Directory Loader\n\nThis loader parses any file stored on S3, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `S3Reader`, you may pass in your [AWS Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). If none are found, the loader assumes they are stored in `~/.aws/credentials`.\n\nAll files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!\n\n## Installation\n\n```bash\npip install llama-index-readers-s3\n```\n\n## Usage\n\nTo use this loader, you need to pass in the name of your S3 Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.\n\nOtherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. AWS Access Key credentials may either be passed in during initialization or stored locally (see above).\n\n```python\nloader = S3Reader(\n bucket=\"scrabble-dictionary\",\n key=\"dictionary.txt\",\n aws_access_id=\"[ACCESS_KEY_ID]\",\n aws_access_secret=\"[ACCESS_KEY_SECRET]\",\n)\ndocuments = loader.load_data()\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index readers s3 integration",
"version": "0.2.0",
"project_urls": null,
"split_keywords": [
"amazon web services",
" aws s3",
" bucket"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "aa81e410212a67db3c1e4b3d15d9a7f8f63eed5169c87a669e93368d7cc71444",
"md5": "22621123d002d1f90810fb8c4cff3716",
"sha256": "c656ba34fee2b2c1b1cfffe746d7cc04f064973d38391cc79633409df1475b43"
},
"downloads": -1,
"filename": "llama_index_readers_s3-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "22621123d002d1f90810fb8c4cff3716",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 4450,
"upload_time": "2024-08-22T14:33:14",
"upload_time_iso_8601": "2024-08-22T14:33:14.871893Z",
"url": "https://files.pythonhosted.org/packages/aa/81/e410212a67db3c1e4b3d15d9a7f8f63eed5169c87a669e93368d7cc71444/llama_index_readers_s3-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c16c7479830f9942a6199d09323969d59373306504fb5ec0ede2fab5d0d97a27",
"md5": "9b8194abdd006a6b96f814429b679b1d",
"sha256": "05795dd3e8f8529c1416d9239267e7dc2c31310a026458fce03e11dd1af68eca"
},
"downloads": -1,
"filename": "llama_index_readers_s3-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "9b8194abdd006a6b96f814429b679b1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 4356,
"upload_time": "2024-08-22T14:33:16",
"upload_time_iso_8601": "2024-08-22T14:33:16.163480Z",
"url": "https://files.pythonhosted.org/packages/c1/6c/7479830f9942a6199d09323969d59373306504fb5ec0ede2fab5d0d97a27/llama_index_readers_s3-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-22 14:33:16",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-readers-s3"
}
JSON