# Local Emulator for Google Cloud Storage
[](https://github.com/oittaa/gcp-storage-emulator/actions/workflows/main.yml)
[](https://pypi.org/project/gcp-storage-emulator/)
[](https://codecov.io/gh/oittaa/gcp-storage-emulator)
[](https://github.com/psf/black)
Google doesn't (yet) ship an emulator for the Cloud Storage API like they do for
Cloud Datastore.
This is a stub emulator so you can run your tests and do local development without
having to connect to the production Storage APIs.
**THIS IS A WORK IN PROGRESS AND ONLY SUPPORTS A LIMITED SUBSET OF THE API**
---
## Installation
`pip install gcp-storage-emulator`
## CLI Usage
### Starting the emulator
Start the emulator with:
```bash
gcp-storage-emulator start
```
By default, the server will listen on `http://localhost:9023` and data is stored under `./.cloudstorage`. You can configure the folder using the env variables `STORAGE_BASE` (default `./`) and `STORAGE_DIR` (default `.cloudstorage`).
If you wish to run the emulator in a testing environment or if you don't want to persist any data, you can use the `--in-memory` parameter. For tests, you might want to consider starting up the server from your code (see the [Python APIs](#python-apis))
If you're using the Google client library (e.g. `google-cloud-storage` for Python) then you can set the `STORAGE_EMULATOR_HOST` environment variable to tell the library to connect to your emulator endpoint rather than the standard `https://storage.googleapis.com`, e.g.:
```bash
export STORAGE_EMULATOR_HOST=http://localhost:9023
```
### Wiping data
You can wipe the data by running
```bash
gcp-storage-emulator wipe
```
You can pass `--keep-buckets` to wipe the data while keeping the buckets.
#### Example
Use in-memory storage and automatically create default storage bucket `my-bucket`.
```bash
gcp-storage-emulator start --host=localhost --port=9023 --in-memory --default-bucket=my-bucket
```
## Python APIs
To start a server from your code you can do
```python
from gcp_storage_emulator.server import create_server
server = create_server("localhost", 9023, in_memory=False)
server.start()
# ........
server.stop()
```
You can wipe the data by calling `server.wipe()`
This can also be achieved (e.g. during tests) by hitting the `/wipe` HTTP endpoint
#### Example
```python
import os
from google.cloud import storage
from gcp_storage_emulator.server import create_server
HOST = "localhost"
PORT = 9023
BUCKET = "test-bucket"
# default_bucket parameter creates the bucket automatically
server = create_server(HOST, PORT, in_memory=True, default_bucket=BUCKET)
server.start()
os.environ["STORAGE_EMULATOR_HOST"] = f"http://{HOST}:{PORT}"
client = storage.Client()
bucket = client.bucket(BUCKET)
blob = bucket.blob("blob1")
blob.upload_from_string("test1")
blob = bucket.blob("blob2")
blob.upload_from_string("test2")
for blob in bucket.list_blobs():
content = blob.download_as_bytes()
print(f"Blob [{blob.name}]: {content}")
server.stop()
```
## Docker
Pull the Docker image.
```bash
docker pull oittaa/gcp-storage-emulator
```
Inside the container instance, the value of the `PORT` environment variable always reflects the port to which requests are sent. It defaults to `8080`. The directory used for the emulated storage is located under `/storage` in the container. In the following example the host's directory `$(pwd)/cloudstorage` will be bound to the emulated storage.
```bash
docker run -d \
-e PORT=9023 \
-p 9023:9023 \
--name gcp-storage-emulator \
-v "$(pwd)/cloudstorage":/storage \
oittaa/gcp-storage-emulator
```
```python
import os
from google.cloud import exceptions, storage
HOST = "localhost"
PORT = 9023
BUCKET = "test-bucket"
os.environ["STORAGE_EMULATOR_HOST"] = f"http://{HOST}:{PORT}"
client = storage.Client()
try:
bucket = client.create_bucket(BUCKET)
except exceptions.Conflict:
bucket = client.bucket(BUCKET)
blob = bucket.blob("blob1")
blob.upload_from_string("test1")
print(blob.download_as_bytes())
```
Raw data
{
"_id": null,
"home_page": "https://github.com/oittaa/gcp-storage-emulator",
"name": "gcp-storage-emulator",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Google Cloud Storage, Google App Engine, Google Cloud Platform, GCS, GAE, GCP",
"author": "Eero Vuojolahti",
"author_email": "contact@oittaa.com",
"download_url": "https://files.pythonhosted.org/packages/17/c2/a0b0e1e54fdd9453603d90939faf652e2488b617c8752edc4ebcd89f1686/gcp_storage_emulator-2024.8.3.tar.gz",
"platform": null,
"description": "# Local Emulator for Google Cloud Storage\n\n[](https://github.com/oittaa/gcp-storage-emulator/actions/workflows/main.yml)\n[](https://pypi.org/project/gcp-storage-emulator/)\n[](https://codecov.io/gh/oittaa/gcp-storage-emulator)\n[](https://github.com/psf/black)\n\nGoogle doesn't (yet) ship an emulator for the Cloud Storage API like they do for\nCloud Datastore.\n\nThis is a stub emulator so you can run your tests and do local development without\nhaving to connect to the production Storage APIs.\n\n\n**THIS IS A WORK IN PROGRESS AND ONLY SUPPORTS A LIMITED SUBSET OF THE API**\n\n---\n\n## Installation\n\n`pip install gcp-storage-emulator`\n\n\n## CLI Usage\n\n\n### Starting the emulator\nStart the emulator with:\n\n```bash\ngcp-storage-emulator start\n```\n\nBy default, the server will listen on `http://localhost:9023` and data is stored under `./.cloudstorage`. You can configure the folder using the env variables `STORAGE_BASE` (default `./`) and `STORAGE_DIR` (default `.cloudstorage`).\n\nIf you wish to run the emulator in a testing environment or if you don't want to persist any data, you can use the `--in-memory` parameter. For tests, you might want to consider starting up the server from your code (see the [Python APIs](#python-apis))\n\nIf you're using the Google client library (e.g. `google-cloud-storage` for Python) then you can set the `STORAGE_EMULATOR_HOST` environment variable to tell the library to connect to your emulator endpoint rather than the standard `https://storage.googleapis.com`, e.g.:\n\n```bash\nexport STORAGE_EMULATOR_HOST=http://localhost:9023\n```\n\n\n### Wiping data\n\nYou can wipe the data by running\n\n```bash\ngcp-storage-emulator wipe\n```\n\nYou can pass `--keep-buckets` to wipe the data while keeping the buckets.\n\n#### Example\n\nUse in-memory storage and automatically create default storage bucket `my-bucket`.\n\n```bash\ngcp-storage-emulator start --host=localhost --port=9023 --in-memory --default-bucket=my-bucket\n```\n\n## Python APIs\n\nTo start a server from your code you can do\n\n```python\nfrom gcp_storage_emulator.server import create_server\n\nserver = create_server(\"localhost\", 9023, in_memory=False)\n\nserver.start()\n# ........\nserver.stop()\n```\n\nYou can wipe the data by calling `server.wipe()`\n\nThis can also be achieved (e.g. during tests) by hitting the `/wipe` HTTP endpoint\n\n#### Example\n\n```python\nimport os\n\nfrom google.cloud import storage\nfrom gcp_storage_emulator.server import create_server\n\nHOST = \"localhost\"\nPORT = 9023\nBUCKET = \"test-bucket\"\n\n# default_bucket parameter creates the bucket automatically\nserver = create_server(HOST, PORT, in_memory=True, default_bucket=BUCKET)\nserver.start()\n\nos.environ[\"STORAGE_EMULATOR_HOST\"] = f\"http://{HOST}:{PORT}\"\nclient = storage.Client()\n\nbucket = client.bucket(BUCKET)\nblob = bucket.blob(\"blob1\")\nblob.upload_from_string(\"test1\")\nblob = bucket.blob(\"blob2\")\nblob.upload_from_string(\"test2\")\nfor blob in bucket.list_blobs():\n content = blob.download_as_bytes()\n print(f\"Blob [{blob.name}]: {content}\")\n\nserver.stop()\n```\n\n## Docker\n\nPull the Docker image.\n\n```bash\ndocker pull oittaa/gcp-storage-emulator\n```\n\nInside the container instance, the value of the `PORT` environment variable always reflects the port to which requests are sent. It defaults to `8080`. The directory used for the emulated storage is located under `/storage` in the container. In the following example the host's directory `$(pwd)/cloudstorage` will be bound to the emulated storage.\n\n```bash\ndocker run -d \\\n -e PORT=9023 \\\n -p 9023:9023 \\\n --name gcp-storage-emulator \\\n -v \"$(pwd)/cloudstorage\":/storage \\\n oittaa/gcp-storage-emulator\n```\n\n```python\nimport os\n\nfrom google.cloud import exceptions, storage\n\nHOST = \"localhost\"\nPORT = 9023\nBUCKET = \"test-bucket\"\n\nos.environ[\"STORAGE_EMULATOR_HOST\"] = f\"http://{HOST}:{PORT}\"\nclient = storage.Client()\n\ntry:\n bucket = client.create_bucket(BUCKET)\nexcept exceptions.Conflict:\n bucket = client.bucket(BUCKET)\n\nblob = bucket.blob(\"blob1\")\nblob.upload_from_string(\"test1\")\nprint(blob.download_as_bytes())\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "A stub emulator for the Google Cloud Storage API",
"version": "2024.8.3",
"project_urls": {
"Homepage": "https://github.com/oittaa/gcp-storage-emulator"
},
"split_keywords": [
"google cloud storage",
" google app engine",
" google cloud platform",
" gcs",
" gae",
" gcp"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "53bfb6c717bd7a5b59244388057d36789e72c28bfaa5e51f3a494563a4e3028e",
"md5": "7f9f695f9ecdd62ce4c39705fe720b27",
"sha256": "1dc4ea56a0caf50fc6092898b9461d08b494824bfbbcca168e2af5da89c053ce"
},
"downloads": -1,
"filename": "gcp_storage_emulator-2024.8.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f9f695f9ecdd62ce4c39705fe720b27",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 19385,
"upload_time": "2024-08-03T19:13:56",
"upload_time_iso_8601": "2024-08-03T19:13:56.296990Z",
"url": "https://files.pythonhosted.org/packages/53/bf/b6c717bd7a5b59244388057d36789e72c28bfaa5e51f3a494563a4e3028e/gcp_storage_emulator-2024.8.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "17c2a0b0e1e54fdd9453603d90939faf652e2488b617c8752edc4ebcd89f1686",
"md5": "542ed535cf5ad7cc820e643739c93b02",
"sha256": "e5d45e5c23a0344c1c4c44b8f8c36f7e8975ca1fcc5134cab608b96ddccd9225"
},
"downloads": -1,
"filename": "gcp_storage_emulator-2024.8.3.tar.gz",
"has_sig": false,
"md5_digest": "542ed535cf5ad7cc820e643739c93b02",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 24928,
"upload_time": "2024-08-03T19:13:58",
"upload_time_iso_8601": "2024-08-03T19:13:58.073907Z",
"url": "https://files.pythonhosted.org/packages/17/c2/a0b0e1e54fdd9453603d90939faf652e2488b617c8752edc4ebcd89f1686/gcp_storage_emulator-2024.8.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-03 19:13:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "oittaa",
"github_project": "gcp-storage-emulator",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "gcp-storage-emulator"
}
JSON
