AWS S3 Backend for Flask-Cache
==============================
When installed and configured, this package will allow you to utilize an AWS S3 bucket as the backend for `Flask-Cache <https://flask-caching.readthedocs.io/en/latest/>`_.
Yes, S3 is not a very good cache. But if millisecond performance is not a concern, S3 is a low-effort and simple to implement alternative that is far less costly than Redis or Memcache.
Installation
------------
Install via ``pip`` directly, or whatever your preferred dependency/package environment manager may be (``poetry``, ``pipenv``, ...):
.. code-block:: shell
pip install flask-cache-s3
Configuration
-------------
Configuration is straightforward, and is identical to the usual process for configuring ``Flask-Cache``. Note that the `mechanism by which you choose to configure <https://flask.palletsprojects.com/en/3.0.x/config/>`_ is up to you; the example below is simply for illustrative purposes:
.. code-block:: python
from flask import Flask
from flask_caching import Cache
config = {
# The import path of the S3 cache backend
"CACHE_TYPE": "flask_caching_s3.S3Cache",
# The bucket name where you'd like cache artifacts to be stored/queried
"CACHE_S3_BUCKET": "your-unique-bucket-name",
# Optional key prefix
"CACHE_KEY_PREFIX": "cache_",
}
app = Flask(__name__)
app.config.from_mapping(config)
cache = Cache(app)
Of course, an application-factory is also supported:
.. code-block:: python
from flask import Flask
from flask_caching import Cache
configuration = {
# The import path of the S3 cache backend
"CACHE_TYPE": "flask_caching_s3.S3Cache",
# The bucket name where you'd like cache artifacts to be stored/queried
"CACHE_S3_BUCKET": "your-unique-bucket-name",
# Optional key prefix
"CACHE_KEY_PREFIX": "cache_",
}
# Initialize the extension without the application object present,
# as is typical in the Flask application factory pattern
cache = Cache()
def create_app():
"""Create and configure an application instance."""
app = Flask("factory-app")
app.config.from_mapping(configuration)
cache.init_app(app)
return app
And in situations where you may want to have _multiple_ caching strategies with different backends, a per-object configuration is supported:
.. code-block:: python
from flask import Flask
from flask_caching import Cache
s3_cache = Cache()
redis_cache = Cache()
def create_app():
"""Create and configure an application instance."""
app = Flask("factory-app")
# Setup our s3 cache
s3_cache.init_app(
app,
config={
"CACHE_TYPE": "flask_caching_s3.S3Cache",
"CACHE_S3_BUCKET": "the-tholian-initiative"
}
)
# And now, separately, we can setup our redis cache,
# with redis-specific configuration options.
redis_cache.init_app(
app,
config={
"CACHE_TYPE": "RedisCache",
"CACHE_REDIS_HOST": "example.com"
}
)
return app
Configuration Options
---------------------
Required
~~~~~~~~
- ``CACHE_S3_BUCKET``: There's only one required configuration, and that's the S3 bucket name. Your bucket must already exist in S3, and you must set the correct ACLs/permissions for your application to read and write from it. This backend will do none of that work for you.
Optional
~~~~~~~~
The following options can be provided to the S3Cache, but are entirely optional:
- ``CACHE_KEY_PREFIX``: A string that will be prepended to /every/ cache key, for both reads and writes. Useful if you want to use the same bucket for non-cache related things and avoid disaster when you call ``cache.clear()`` and wonder where all your S3 bucket contents have gone.
- ``CACHE_DEFAULT_TIMEOUT``: The number of seconds that an item in the cache is valid for. After this time has elapsed, the item is considered expired, and even if the item is still in the S3 bucket, a cache miss will occur.
- ``CACHE_S3_ENDPOINT_URL``: The endpoint for the S3 service. Typically this is only utilized when using something like `localstack <https://localstack.cloud/>`_ for local development/testing.
- ``CACHE_OPTIONS``: A dictionary of key/value pairs for more fine-grained configuration of how the cache will behave.
.. code-block:: python
s3_cache.init_app(
app,
config={
"CACHE_TYPE": "flask_caching_s3.S3Cache",
"CACHE_S3_BUCKET": "the-tholian-initiative",
"CACHE_OPTIONS": {"purge_expired_on_read": True}
}
)
The only key currently supported in ``CACHE_OPTIONS`` is the boolean ``purge_expired_on_read``, which defaults to ``False``. If set to ``True``, items will be evicted from S3 if ``Flask-Cache```` attempts to read them (via ``cache.get()`` or ``cache.has()``, for example) and they have expired.
S3 Object Lifecycle Management
-------------------------------
The use of ``purge_expired_on_read`` *does* incur a performance penalty since the eviction/deletion is performed in the same operation, and it also means that if some items are never accessed, they will continue to exist in the bucket far beyond their expiration.
The proper solution to this is to create an `S3 object lifecycle rule
<https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html>`_
which can clean up objects for you. Care must be taken here, though, since you cannot rely on the ``CACHE_DEFAULT_TIMEOUT`` value as the boundary value for object lifetimes; users of ``Flask-Cache`` can always override the default timeout at call-time with e.g.:
.. code-block:: python
@app.route("/")
@cache.cached(timeout=50)
def index():
return render_template('index.html')
Where ``cache.cached(timeout=3600)`` indicates that the cached object is valid for 3600 seconds, even though our default may be set to 300.
Thus, if you do choose to go with an Object Lifecycle Management rule, pick an Expiration policy that is beyond whatever maximum timeout value that you would conceivably apply.
Testing
~~~~~~~
0. Have Docker installed and running
1. Clone this repository
2. Ensure you have `poetry` available on your system
3. `poetry run pytest`
The test suite will spin up an ephemeral Docker container; it may take a few seconds for it to load. The relevant test fixtures will handle creating objects and their values in the Localstack S3 service.
Raw data
{
"_id": null,
"home_page": "https://github.com/fictivekin/flask-caching-s3",
"name": "flask-caching-s3",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "flask,flask-caching,aws,s3,caching",
"author": "Jo\u00ebl Perras",
"author_email": "joel@fictivekin.com",
"download_url": "https://files.pythonhosted.org/packages/eb/96/e2e98f69b8511f35bf8304ca658281061d58310d68d9195a3828bd0b3dec/flask_caching_s3-0.1.5.tar.gz",
"platform": null,
"description": "AWS S3 Backend for Flask-Cache\n==============================\n\nWhen installed and configured, this package will allow you to utilize an AWS S3 bucket as the backend for `Flask-Cache <https://flask-caching.readthedocs.io/en/latest/>`_.\n\nYes, S3 is not a very good cache. But if millisecond performance is not a concern, S3 is a low-effort and simple to implement alternative that is far less costly than Redis or Memcache.\n\nInstallation\n------------\n\nInstall via ``pip`` directly, or whatever your preferred dependency/package environment manager may be (``poetry``, ``pipenv``, ...):\n\n.. code-block:: shell\n\n pip install flask-cache-s3\n\nConfiguration\n-------------\n\nConfiguration is straightforward, and is identical to the usual process for configuring ``Flask-Cache``. Note that the `mechanism by which you choose to configure <https://flask.palletsprojects.com/en/3.0.x/config/>`_ is up to you; the example below is simply for illustrative purposes:\n\n.. code-block:: python\n\n from flask import Flask\n from flask_caching import Cache\n\n config = {\n # The import path of the S3 cache backend\n \"CACHE_TYPE\": \"flask_caching_s3.S3Cache\",\n # The bucket name where you'd like cache artifacts to be stored/queried\n \"CACHE_S3_BUCKET\": \"your-unique-bucket-name\",\n # Optional key prefix\n \"CACHE_KEY_PREFIX\": \"cache_\",\n }\n\n app = Flask(__name__)\n app.config.from_mapping(config)\n cache = Cache(app)\n\nOf course, an application-factory is also supported:\n\n.. code-block:: python\n\n from flask import Flask\n from flask_caching import Cache\n\n configuration = {\n # The import path of the S3 cache backend\n \"CACHE_TYPE\": \"flask_caching_s3.S3Cache\",\n # The bucket name where you'd like cache artifacts to be stored/queried\n \"CACHE_S3_BUCKET\": \"your-unique-bucket-name\",\n # Optional key prefix\n \"CACHE_KEY_PREFIX\": \"cache_\",\n }\n\n # Initialize the extension without the application object present,\n # as is typical in the Flask application factory pattern\n cache = Cache()\n\n def create_app():\n \"\"\"Create and configure an application instance.\"\"\"\n\n app = Flask(\"factory-app\")\n app.config.from_mapping(configuration)\n\n cache.init_app(app)\n\n return app\n\nAnd in situations where you may want to have _multiple_ caching strategies with different backends, a per-object configuration is supported:\n\n.. code-block:: python\n\n from flask import Flask\n from flask_caching import Cache\n\n s3_cache = Cache()\n redis_cache = Cache()\n\n def create_app():\n \"\"\"Create and configure an application instance.\"\"\"\n\n app = Flask(\"factory-app\")\n\n # Setup our s3 cache\n s3_cache.init_app(\n app,\n config={\n \"CACHE_TYPE\": \"flask_caching_s3.S3Cache\",\n \"CACHE_S3_BUCKET\": \"the-tholian-initiative\"\n }\n )\n\n # And now, separately, we can setup our redis cache,\n # with redis-specific configuration options.\n redis_cache.init_app(\n app,\n config={\n \"CACHE_TYPE\": \"RedisCache\",\n \"CACHE_REDIS_HOST\": \"example.com\"\n }\n )\n\n return app\n\nConfiguration Options\n---------------------\n\nRequired\n~~~~~~~~\n\n- ``CACHE_S3_BUCKET``: There's only one required configuration, and that's the S3 bucket name. Your bucket must already exist in S3, and you must set the correct ACLs/permissions for your application to read and write from it. This backend will do none of that work for you.\n\nOptional\n~~~~~~~~\n\nThe following options can be provided to the S3Cache, but are entirely optional:\n\n- ``CACHE_KEY_PREFIX``: A string that will be prepended to /every/ cache key, for both reads and writes. Useful if you want to use the same bucket for non-cache related things and avoid disaster when you call ``cache.clear()`` and wonder where all your S3 bucket contents have gone.\n- ``CACHE_DEFAULT_TIMEOUT``: The number of seconds that an item in the cache is valid for. After this time has elapsed, the item is considered expired, and even if the item is still in the S3 bucket, a cache miss will occur.\n- ``CACHE_S3_ENDPOINT_URL``: The endpoint for the S3 service. Typically this is only utilized when using something like `localstack <https://localstack.cloud/>`_ for local development/testing.\n- ``CACHE_OPTIONS``: A dictionary of key/value pairs for more fine-grained configuration of how the cache will behave.\n\n .. code-block:: python\n\n s3_cache.init_app(\n app,\n config={\n \"CACHE_TYPE\": \"flask_caching_s3.S3Cache\",\n \"CACHE_S3_BUCKET\": \"the-tholian-initiative\",\n \"CACHE_OPTIONS\": {\"purge_expired_on_read\": True}\n }\n )\n\n The only key currently supported in ``CACHE_OPTIONS`` is the boolean ``purge_expired_on_read``, which defaults to ``False``. If set to ``True``, items will be evicted from S3 if ``Flask-Cache```` attempts to read them (via ``cache.get()`` or ``cache.has()``, for example) and they have expired.\n\nS3 Object Lifecycle Management\n-------------------------------\n\nThe use of ``purge_expired_on_read`` *does* incur a performance penalty since the eviction/deletion is performed in the same operation, and it also means that if some items are never accessed, they will continue to exist in the bucket far beyond their expiration.\n\nThe proper solution to this is to create an `S3 object lifecycle rule\n<https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html>`_\nwhich can clean up objects for you. Care must be taken here, though, since you cannot rely on the ``CACHE_DEFAULT_TIMEOUT`` value as the boundary value for object lifetimes; users of ``Flask-Cache`` can always override the default timeout at call-time with e.g.:\n\n.. code-block:: python\n\n @app.route(\"/\")\n @cache.cached(timeout=50)\n def index():\n return render_template('index.html')\n\nWhere ``cache.cached(timeout=3600)`` indicates that the cached object is valid for 3600 seconds, even though our default may be set to 300.\n\nThus, if you do choose to go with an Object Lifecycle Management rule, pick an Expiration policy that is beyond whatever maximum timeout value that you would conceivably apply.\n\n\nTesting\n~~~~~~~\n\n0. Have Docker installed and running\n1. Clone this repository\n2. Ensure you have `poetry` available on your system\n3. `poetry run pytest`\n\nThe test suite will spin up an ephemeral Docker container; it may take a few seconds for it to load. The relevant test fixtures will handle creating objects and their values in the Localstack S3 service.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An S3 adapter for the Flask-Caching framework",
"version": "0.1.5",
"project_urls": {
"Homepage": "https://github.com/fictivekin/flask-caching-s3",
"Repository": "https://github.com/fictivekin/flask-caching-s3"
},
"split_keywords": [
"flask",
"flask-caching",
"aws",
"s3",
"caching"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "78ac2f363245afbcb25fed69244827c9eb4258ce23850dadb02025d5ec3a5d5d",
"md5": "2e279d61881ac20bf06f2575b281269a",
"sha256": "e8b823fd9239554f681dfda1837afe6f5d842f30fba2991861a21b86abf00cfe"
},
"downloads": -1,
"filename": "flask_caching_s3-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2e279d61881ac20bf06f2575b281269a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 8748,
"upload_time": "2023-10-19T03:35:19",
"upload_time_iso_8601": "2023-10-19T03:35:19.658574Z",
"url": "https://files.pythonhosted.org/packages/78/ac/2f363245afbcb25fed69244827c9eb4258ce23850dadb02025d5ec3a5d5d/flask_caching_s3-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "eb96e2e98f69b8511f35bf8304ca658281061d58310d68d9195a3828bd0b3dec",
"md5": "76075ee38b34a840b2886312911c0850",
"sha256": "b6217370f68073707ba85c692d965913677995dc8b07da94c9e753773ac3c4ad"
},
"downloads": -1,
"filename": "flask_caching_s3-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "76075ee38b34a840b2886312911c0850",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 7396,
"upload_time": "2023-10-19T03:35:21",
"upload_time_iso_8601": "2023-10-19T03:35:21.190732Z",
"url": "https://files.pythonhosted.org/packages/eb/96/e2e98f69b8511f35bf8304ca658281061d58310d68d9195a3828bd0b3dec/flask_caching_s3-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-19 03:35:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fictivekin",
"github_project": "flask-caching-s3",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "flask-caching-s3"
}
JSON
