Python Client for Google Cloud Storage
======================================
|stable| |pypi| |versions|
`Google Cloud Storage`_ is a managed service for storing unstructured data. Cloud Storage
allows world-wide storage and retrieval of any amount of data at any time. You can use
Cloud Storage for a range of scenarios including serving website content, storing data
for archival and disaster recovery, or distributing large data objects to users via direct download.
A comprehensive list of changes in each version may be found in the `CHANGELOG`_.
- `Product Documentation`_
- `Client Library Documentation`_
- `github.com/googleapis/python-storage`_
Certain control plane and long-running operations for Cloud Storage (including Folder
and Managed Folder operations) are supported via the `Storage Control Client`_.
The `Storage Control API`_ creates one space to perform metadata-specific, control plane,
and long-running operations apart from the Storage API.
Read more about the client libraries for Cloud APIs, including the older
Google APIs Client Libraries, in `Client Libraries Explained`_.
.. |stable| image:: https://img.shields.io/badge/support-stable-gold.svg
:target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels
.. |pypi| image:: https://img.shields.io/pypi/v/google-cloud-storage.svg
:target: https://pypi.org/project/google-cloud-storage/
.. |versions| image:: https://img.shields.io/pypi/pyversions/google-cloud-storage.svg
:target: https://pypi.org/project/google-cloud-storage/
.. _Google Cloud Storage: https://cloud.google.com/storage
.. _Client Library Documentation: https://cloud.google.com/python/docs/reference/storage/latest
.. _Product Documentation: https://cloud.google.com/storage
.. _CHANGELOG: https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md
.. _github.com/googleapis/python-storage: https://github.com/googleapis/python-storage
.. _Storage Control Client: https://cloud.google.com/python/docs/reference/google-cloud-storage-control/latest
.. _Storage Control API: https://cloud.google.com/storage/docs/reference/rpc/google.storage.control.v2
.. _Client Libraries Explained: https://cloud.google.com/apis/docs/client-libraries-explained
3.0 Major Version Notes
-----------------------
Feedback Welcome
~~~~~~~~~~~~~~~~
If you experience that backwards compatibility for your application is broken
with this major version release, please let us know through the Github issues
system. While some breaks of backwards compatibility may be unavoidable due to
new features in the major version release, we will do our best to minimize
them. Thank you.
Exception Handling
~~~~~~~~~~~~~~~~~~
In Python Storage 3.0, the dependency ``google-resumable-media`` was integrated.
The ``google-resumable-media`` dependency included exceptions
``google.resumable_media.common.InvalidResponse`` and
``google.resumable_media.common.DataCorruption``, which were often imported
directly in user application code. The replacements for these exceptions are
``google.cloud.storage.exceptions.InvalidResponse`` and
``google.cloud.storage.exceptions.DataCorruption``. Please update application code
to import and use these exceptions instead.
For backwards compatibility, if ``google-resumable-media`` is installed, the new
exceptions will be defined as subclasses of the old exceptions, so applications
should continue to work without modification. This backwards compatibility
feature may be removed in a future major version update.
Some users may be using the original exception classes from the
``google-resumable-media`` library without explicitly installing that library. So
as not to break user applications following this pattern,
``google-resumable-media`` is still in the list of dependencies in this package's
setup.py file. Applications which do not import directly from
``google-resumable-media`` can safely disregard this dependency.
This backwards compatibility feature **will be removed** in a future major
version update. Please migrate to using the ``google.cloud.storage.exceptions``
classes as above.
Checksum Defaults
~~~~~~~~~~~~~~~~~
In Python Storage 3.0, uploads and downloads now have a default of "auto" where
applicable. "Auto" will use crc32c checksums, except for unusual cases where the
fast (C extension) crc32c implementation is not available, in which case it will
use md5 instead. Before Python Storage 3.0, the default was md5 for most
downloads and None for most uploads. Note that ranged downloads ("start" or
"end" set) still do not support any checksumming, and some features in
``transfer_manager.py`` still support crc32c only.
Note: The method ``Blob.upload_from_file()`` requires a file in bytes mode, but
when checksum is set to None, as was the previous default, would not throw an
error if passed a file in string mode under some circumstances. With the new
defaults, it will now raise a TypeError. Please use a file opened in bytes
reading mode as required.
Miscellaneous
~~~~~~~~~~~~~
- The ``BlobWriter`` class now attempts to terminate an ongoing resumable upload if
the writer exits with an exception.
- Retry behavior is now identical between media operations (uploads and
downloads) and other operations, and custom predicates are now supported for
media operations as well.
- ``Blob.download_as_filename()`` will now delete the empty file if it results in a
google.cloud.exceptions.NotFound exception (HTTP 404).
- Previously, object upload, metadata update, and delete methods had retries
disabled by default unless the generation or metageneration was specified in
the request. This has now changed so that retries are enabled by default.
Quick Start
-----------
In order to use this library, you first need to go through the following steps.
A step-by-step guide may also be found in `Get Started with Client Libraries`_.
1. `Select or create a Cloud Platform project.`_
2. `Enable billing for your project.`_
3. `Enable the Google Cloud Storage API.`_
4. `Setup Authentication.`_
.. _Get Started with Client Libraries: https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-python
.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project
.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project
.. _Enable the Google Cloud Storage API.: https://console.cloud.google.com/flows/enableapi?apiid=storage-api.googleapis.com
.. _Setup Authentication.: https://cloud.google.com/docs/authentication/client-libraries
Installation
~~~~~~~~~~~~
Install this library in a virtual environment using `venv`_. `venv`_ is a tool that
creates isolated Python environments. These isolated environments can have separate
versions of Python packages, which allows you to isolate one project's dependencies
from the dependencies of other projects.
With `venv`_, it's possible to install this library without needing system
install permissions, and without clashing with the installed system
dependencies.
.. _`venv`: https://docs.python.org/3/library/venv.html
Code samples and snippets
~~~~~~~~~~~~~~~~~~~~~~~~~
Code samples and snippets live in the `samples/`_ folder.
.. _`samples/`: https://github.com/googleapis/python-storage/tree/main/samples
Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of
Python.
Python >= 3.7
.. _active: https://devguide.python.org/devcycle/#in-development-main-branch
.. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches
Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.6
If you are using an `end-of-life`_
version of Python, we recommend that you update as soon as possible to an actively supported version.
.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches
Mac/Linux
^^^^^^^^^
.. code-block:: console
python3 -m venv <your-env>
source <your-env>/bin/activate
pip install google-cloud-storage
Windows
^^^^^^^
.. code-block:: console
py -m venv <your-env>
.\<your-env>\Scripts\activate
pip install google-cloud-storage
Tracing With OpenTelemetry
~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a PREVIEW FEATURE: Coverage and functionality are still in development and subject to change.
This library can be configured to use `OpenTelemetry`_ to generate traces on calls to Google Cloud Storage.
For information on the benefits and utility of tracing, read the `Cloud Trace Overview <https://cloud.google.com/trace/docs/overview>`_.
To enable OpenTelemetry tracing in the Cloud Storage client, first install OpenTelemetry:
.. code-block:: console
pip install google-cloud-storage[tracing]
Set the ``ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES`` environment variable to selectively opt-in tracing for the Cloud Storage client:
.. code-block:: console
export ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES=True
You will also need to tell OpenTelemetry which exporter to use. An example to export traces to Google Cloud Trace can be found below.
.. code-block:: console
# Install the Google Cloud Trace exporter and propagator, however you can use any exporter of your choice.
pip install opentelemetry-exporter-gcp-trace opentelemetry-propagator-gcp
# [Optional] Install the OpenTelemetry Requests Instrumentation to trace the underlying HTTP requests.
pip install opentelemetry-instrumentation-requests
.. code-block:: python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(CloudTraceSpanExporter()))
trace.set_tracer_provider(tracer_provider)
# Optional yet recommended to instrument the requests HTTP library
from opentelemetry.instrumentation.requests import RequestsInstrumentor
RequestsInstrumentor().instrument(tracer_provider=tracer_provider)
In this example, tracing data will be published to the `Google Cloud Trace`_ console.
Tracing is most effective when many libraries are instrumented to provide insight over the entire lifespan of a request.
For a list of libraries that can be instrumented, refer to the `OpenTelemetry Registry`_.
.. _OpenTelemetry: https://opentelemetry.io
.. _OpenTelemetry Registry: https://opentelemetry.io/ecosystem/registry
.. _Google Cloud Trace: https://cloud.google.com/trace
Next Steps
~~~~~~~~~~
- Read the `Google Cloud Storage Product documentation`_ to learn
more about the product and see How-to Guides.
- Read the `Client Library Documentation`_ for Google Cloud Storage API
to see other available methods on the client.
- View this `README`_ to see the full list of Cloud
APIs that we cover.
.. _Google Cloud Storage Product documentation: https://cloud.google.com/storage
.. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst
Raw data
{
"_id": null,
"home_page": "https://github.com/googleapis/python-storage",
"name": "google-cloud-storage",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Google LLC",
"author_email": "googleapis-packages@google.com",
"download_url": "https://files.pythonhosted.org/packages/f3/08/52143124415a889bbab60a8ecede1e31ea0e8d992ca078317886f26dc3be/google_cloud_storage-3.1.0.tar.gz",
"platform": "Posix; MacOS X; Windows",
"description": "Python Client for Google Cloud Storage\n======================================\n\n|stable| |pypi| |versions|\n\n`Google Cloud Storage`_ is a managed service for storing unstructured data. Cloud Storage\nallows world-wide storage and retrieval of any amount of data at any time. You can use\nCloud Storage for a range of scenarios including serving website content, storing data\nfor archival and disaster recovery, or distributing large data objects to users via direct download.\n\nA comprehensive list of changes in each version may be found in the `CHANGELOG`_.\n\n- `Product Documentation`_\n- `Client Library Documentation`_\n- `github.com/googleapis/python-storage`_\n\nCertain control plane and long-running operations for Cloud Storage (including Folder\nand Managed Folder operations) are supported via the `Storage Control Client`_.\nThe `Storage Control API`_ creates one space to perform metadata-specific, control plane,\nand long-running operations apart from the Storage API.\n\nRead more about the client libraries for Cloud APIs, including the older\nGoogle APIs Client Libraries, in `Client Libraries Explained`_.\n\n.. |stable| image:: https://img.shields.io/badge/support-stable-gold.svg\n :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels\n.. |pypi| image:: https://img.shields.io/pypi/v/google-cloud-storage.svg\n :target: https://pypi.org/project/google-cloud-storage/\n.. |versions| image:: https://img.shields.io/pypi/pyversions/google-cloud-storage.svg\n :target: https://pypi.org/project/google-cloud-storage/\n.. _Google Cloud Storage: https://cloud.google.com/storage\n.. _Client Library Documentation: https://cloud.google.com/python/docs/reference/storage/latest\n.. _Product Documentation: https://cloud.google.com/storage\n.. _CHANGELOG: https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md\n.. _github.com/googleapis/python-storage: https://github.com/googleapis/python-storage\n.. _Storage Control Client: https://cloud.google.com/python/docs/reference/google-cloud-storage-control/latest\n.. _Storage Control API: https://cloud.google.com/storage/docs/reference/rpc/google.storage.control.v2\n.. _Client Libraries Explained: https://cloud.google.com/apis/docs/client-libraries-explained\n\n3.0 Major Version Notes\n-----------------------\n\nFeedback Welcome\n~~~~~~~~~~~~~~~~\n\nIf you experience that backwards compatibility for your application is broken\nwith this major version release, please let us know through the Github issues\nsystem. While some breaks of backwards compatibility may be unavoidable due to\nnew features in the major version release, we will do our best to minimize\nthem. Thank you.\n\nException Handling\n~~~~~~~~~~~~~~~~~~\n\nIn Python Storage 3.0, the dependency ``google-resumable-media`` was integrated.\nThe ``google-resumable-media`` dependency included exceptions\n``google.resumable_media.common.InvalidResponse`` and\n``google.resumable_media.common.DataCorruption``, which were often imported\ndirectly in user application code. The replacements for these exceptions are\n``google.cloud.storage.exceptions.InvalidResponse`` and\n``google.cloud.storage.exceptions.DataCorruption``. Please update application code\nto import and use these exceptions instead.\n\nFor backwards compatibility, if ``google-resumable-media`` is installed, the new\nexceptions will be defined as subclasses of the old exceptions, so applications\nshould continue to work without modification. This backwards compatibility\nfeature may be removed in a future major version update.\n\nSome users may be using the original exception classes from the\n``google-resumable-media`` library without explicitly installing that library. So\nas not to break user applications following this pattern,\n``google-resumable-media`` is still in the list of dependencies in this package's\nsetup.py file. Applications which do not import directly from\n``google-resumable-media`` can safely disregard this dependency.\nThis backwards compatibility feature **will be removed** in a future major\nversion update. Please migrate to using the ``google.cloud.storage.exceptions``\nclasses as above.\n\nChecksum Defaults\n~~~~~~~~~~~~~~~~~\n\nIn Python Storage 3.0, uploads and downloads now have a default of \"auto\" where\napplicable. \"Auto\" will use crc32c checksums, except for unusual cases where the\nfast (C extension) crc32c implementation is not available, in which case it will\nuse md5 instead. Before Python Storage 3.0, the default was md5 for most\ndownloads and None for most uploads. Note that ranged downloads (\"start\" or\n\"end\" set) still do not support any checksumming, and some features in\n``transfer_manager.py`` still support crc32c only.\n\nNote: The method ``Blob.upload_from_file()`` requires a file in bytes mode, but\nwhen checksum is set to None, as was the previous default, would not throw an\nerror if passed a file in string mode under some circumstances. With the new\ndefaults, it will now raise a TypeError. Please use a file opened in bytes\nreading mode as required.\n\nMiscellaneous\n~~~~~~~~~~~~~\n\n- The ``BlobWriter`` class now attempts to terminate an ongoing resumable upload if\n the writer exits with an exception.\n- Retry behavior is now identical between media operations (uploads and\n downloads) and other operations, and custom predicates are now supported for\n media operations as well.\n- ``Blob.download_as_filename()`` will now delete the empty file if it results in a\n google.cloud.exceptions.NotFound exception (HTTP 404).\n- Previously, object upload, metadata update, and delete methods had retries\n disabled by default unless the generation or metageneration was specified in\n the request. This has now changed so that retries are enabled by default.\n\nQuick Start\n-----------\n\nIn order to use this library, you first need to go through the following steps.\nA step-by-step guide may also be found in `Get Started with Client Libraries`_.\n\n1. `Select or create a Cloud Platform project.`_\n2. `Enable billing for your project.`_\n3. `Enable the Google Cloud Storage API.`_\n4. `Setup Authentication.`_\n\n.. _Get Started with Client Libraries: https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-python\n.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project\n.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project\n.. _Enable the Google Cloud Storage API.: https://console.cloud.google.com/flows/enableapi?apiid=storage-api.googleapis.com\n.. _Setup Authentication.: https://cloud.google.com/docs/authentication/client-libraries\n\nInstallation\n~~~~~~~~~~~~\n\nInstall this library in a virtual environment using `venv`_. `venv`_ is a tool that\ncreates isolated Python environments. These isolated environments can have separate\nversions of Python packages, which allows you to isolate one project's dependencies\nfrom the dependencies of other projects.\n\nWith `venv`_, it's possible to install this library without needing system\ninstall permissions, and without clashing with the installed system\ndependencies.\n\n.. _`venv`: https://docs.python.org/3/library/venv.html\n\n\nCode samples and snippets\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nCode samples and snippets live in the `samples/`_ folder.\n\n.. _`samples/`: https://github.com/googleapis/python-storage/tree/main/samples\n\n\nSupported Python Versions\n^^^^^^^^^^^^^^^^^^^^^^^^^\nOur client libraries are compatible with all current `active`_ and `maintenance`_ versions of\nPython.\n\nPython >= 3.7\n\n.. _active: https://devguide.python.org/devcycle/#in-development-main-branch\n.. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches\n\nUnsupported Python Versions\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\nPython <= 3.6\n\nIf you are using an `end-of-life`_\nversion of Python, we recommend that you update as soon as possible to an actively supported version.\n\n.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches\n\nMac/Linux\n^^^^^^^^^\n\n.. code-block:: console\n\n python3 -m venv <your-env>\n source <your-env>/bin/activate\n pip install google-cloud-storage\n\n\nWindows\n^^^^^^^\n\n.. code-block:: console\n\n py -m venv <your-env>\n .\\<your-env>\\Scripts\\activate\n pip install google-cloud-storage\n\n\nTracing With OpenTelemetry\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThis is a PREVIEW FEATURE: Coverage and functionality are still in development and subject to change.\n\nThis library can be configured to use `OpenTelemetry`_ to generate traces on calls to Google Cloud Storage.\nFor information on the benefits and utility of tracing, read the `Cloud Trace Overview <https://cloud.google.com/trace/docs/overview>`_.\n\nTo enable OpenTelemetry tracing in the Cloud Storage client, first install OpenTelemetry:\n\n.. code-block:: console\n\n pip install google-cloud-storage[tracing]\n\nSet the ``ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES`` environment variable to selectively opt-in tracing for the Cloud Storage client:\n\n.. code-block:: console\n\n export ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES=True\n\nYou will also need to tell OpenTelemetry which exporter to use. An example to export traces to Google Cloud Trace can be found below.\n\n.. code-block:: console\n\n # Install the Google Cloud Trace exporter and propagator, however you can use any exporter of your choice.\n pip install opentelemetry-exporter-gcp-trace opentelemetry-propagator-gcp\n\n # [Optional] Install the OpenTelemetry Requests Instrumentation to trace the underlying HTTP requests.\n pip install opentelemetry-instrumentation-requests\n\n.. code-block:: python\n\n from opentelemetry import trace\n from opentelemetry.sdk.trace import TracerProvider\n from opentelemetry.sdk.trace.export import BatchSpanProcessor\n from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter\n\n tracer_provider = TracerProvider()\n tracer_provider.add_span_processor(BatchSpanProcessor(CloudTraceSpanExporter()))\n trace.set_tracer_provider(tracer_provider)\n\n # Optional yet recommended to instrument the requests HTTP library\n from opentelemetry.instrumentation.requests import RequestsInstrumentor\n RequestsInstrumentor().instrument(tracer_provider=tracer_provider)\n\nIn this example, tracing data will be published to the `Google Cloud Trace`_ console.\nTracing is most effective when many libraries are instrumented to provide insight over the entire lifespan of a request.\nFor a list of libraries that can be instrumented, refer to the `OpenTelemetry Registry`_.\n\n.. _OpenTelemetry: https://opentelemetry.io\n.. _OpenTelemetry Registry: https://opentelemetry.io/ecosystem/registry\n.. _Google Cloud Trace: https://cloud.google.com/trace\n\n\nNext Steps\n~~~~~~~~~~\n\n- Read the `Google Cloud Storage Product documentation`_ to learn\n more about the product and see How-to Guides.\n- Read the `Client Library Documentation`_ for Google Cloud Storage API\n to see other available methods on the client.\n- View this `README`_ to see the full list of Cloud\n APIs that we cover.\n\n.. _Google Cloud Storage Product documentation: https://cloud.google.com/storage\n.. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Google Cloud Storage API client library",
"version": "3.1.0",
"project_urls": {
"Homepage": "https://github.com/googleapis/python-storage"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13b8c99c965659f45efa73080477c49ffddf7b9aecb00806be8422560bb5b824",
"md5": "562ab660f891f08aab8d907613a1b2d9",
"sha256": "eaf36966b68660a9633f03b067e4a10ce09f1377cae3ff9f2c699f69a81c66c6"
},
"downloads": -1,
"filename": "google_cloud_storage-3.1.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "562ab660f891f08aab8d907613a1b2d9",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.7",
"size": 174861,
"upload_time": "2025-02-28T00:19:59",
"upload_time_iso_8601": "2025-02-28T00:19:59.459370Z",
"url": "https://files.pythonhosted.org/packages/13/b8/c99c965659f45efa73080477c49ffddf7b9aecb00806be8422560bb5b824/google_cloud_storage-3.1.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f30852143124415a889bbab60a8ecede1e31ea0e8d992ca078317886f26dc3be",
"md5": "077f0d9bdba40f870a4d36deb174b89b",
"sha256": "944273179897c7c8a07ee15f2e6466a02da0c7c4b9ecceac2a26017cb2972049"
},
"downloads": -1,
"filename": "google_cloud_storage-3.1.0.tar.gz",
"has_sig": false,
"md5_digest": "077f0d9bdba40f870a4d36deb174b89b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 7666527,
"upload_time": "2025-02-28T00:20:02",
"upload_time_iso_8601": "2025-02-28T00:20:02.383343Z",
"url": "https://files.pythonhosted.org/packages/f3/08/52143124415a889bbab60a8ecede1e31ea0e8d992ca078317886f26dc3be/google_cloud_storage-3.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-28 00:20:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "googleapis",
"github_project": "python-storage",
"travis_ci": false,
"coveralls": true,
"github_actions": false,
"lcname": "google-cloud-storage"
}
JSON
