# GCS File or Directory Loader
This loader parses any file stored on GCS, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `GCSReader`, you may pass in your [GCP Service Account Key](https://cloud.google.com/iam/docs/keys-create-delete). If none are found, the loader assumes they are stored in `~/.gcp/credentials`.
All files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!
## Usage
To use this loader, you need to pass in the name of your GCS Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.
Otherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. GCP Service Account Key credentials may either be passed in during initialization or stored locally (see above).
```python
loader = GCSReader(
bucket="scrabble-dictionary",
key="dictionary.txt",
service_account_key_json="[SERVICE_ACCOUNT_KEY_JSON]",
)
documents = loader.load_data()
```
This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent. See [here](https://github.com/emptycrown/llama-hub/tree/main) for examples.
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-readers-gcs",
"maintainer": "nfiacco",
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "bucket, gcp, gcs, google cloud storage",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/3e/5b/8f07356d0f40693e641100d7560ceb081248a76ce723a21078ba58fd6fdf/llama_index_readers_gcs-0.1.6.tar.gz",
"platform": null,
"description": "# GCS File or Directory Loader\n\nThis loader parses any file stored on GCS, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `GCSReader`, you may pass in your [GCP Service Account Key](https://cloud.google.com/iam/docs/keys-create-delete). If none are found, the loader assumes they are stored in `~/.gcp/credentials`.\n\nAll files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!\n\n## Usage\n\nTo use this loader, you need to pass in the name of your GCS Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.\n\nOtherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. GCP Service Account Key credentials may either be passed in during initialization or stored locally (see above).\n\n```python\nloader = GCSReader(\n bucket=\"scrabble-dictionary\",\n key=\"dictionary.txt\",\n service_account_key_json=\"[SERVICE_ACCOUNT_KEY_JSON]\",\n)\ndocuments = loader.load_data()\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent. See [here](https://github.com/emptycrown/llama-hub/tree/main) for examples.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index readers gcs integration",
"version": "0.1.6",
"project_urls": null,
"split_keywords": [
"bucket",
" gcp",
" gcs",
" google cloud storage"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ffbf72e9ad901cbeff1f917330fd0e3e2ad07ca7685ac87ecea19e1fcec15a0d",
"md5": "c82de46991fc444c6d839019c16d8190",
"sha256": "d322c43adcb3a3754533d8864dc0afa09f0960414f37c1fcc65a60f63f1cab44"
},
"downloads": -1,
"filename": "llama_index_readers_gcs-0.1.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c82de46991fc444c6d839019c16d8190",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 3777,
"upload_time": "2024-04-01T21:35:08",
"upload_time_iso_8601": "2024-04-01T21:35:08.680192Z",
"url": "https://files.pythonhosted.org/packages/ff/bf/72e9ad901cbeff1f917330fd0e3e2ad07ca7685ac87ecea19e1fcec15a0d/llama_index_readers_gcs-0.1.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3e5b8f07356d0f40693e641100d7560ceb081248a76ce723a21078ba58fd6fdf",
"md5": "d7e140990f85bc3651167d09f19d0743",
"sha256": "7c82ac2c2dce89bcf67f8e1795c2d951d3a267edd218416d24e835aeaeb5c4c1"
},
"downloads": -1,
"filename": "llama_index_readers_gcs-0.1.6.tar.gz",
"has_sig": false,
"md5_digest": "d7e140990f85bc3651167d09f19d0743",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 3514,
"upload_time": "2024-04-01T21:35:10",
"upload_time_iso_8601": "2024-04-01T21:35:10.217860Z",
"url": "https://files.pythonhosted.org/packages/3e/5b/8f07356d0f40693e641100d7560ceb081248a76ce723a21078ba58fd6fdf/llama_index_readers_gcs-0.1.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-01 21:35:10",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-readers-gcs"
}
JSON