Summary
-------
A Cubicweb Storage that stores the data on S3.
It is a standard Cubicweb_ Storage, similar to the BytesFileSystemStorage_. For
example, if you want to store File.data payloads in S3, you should create an
S3Storage instance, typically in a startup hook, as::
from cubicweb.server.hook import Hook
from cubicweb_s3storage.storages import S3Storage
class S3StorageStartupHook(Hook):
__regid__ = 'mycube.server-startup-hook'
events = ('server_startup', 'server_maintenance')
def __call__(self):
storage = S3Storage('s3_bucket')
self.repo.system_source.set_storage('File', 'data', storage)
By default, S3 object keys (ie. S3 object identifiers) are random UUID
generated at entity creation time. If you want to implement another key
generation algorithm, you must write overload ``S3Storage.new_s3_key()``
method.
Communications with AWS are handled by boto3_.
.. Warning::
There is no cache, for now, so every time you read an entity
attribute that is stored on S3, it will be downloaded from bucket provider.
Configuration
-------------
The S3Storage class constructor needs an S3 bucket name.
If you want to put your aws credentials in an other file than the default
(``~/.aws/credentials``), you must set the ``AWS_CONFIG_FILE`` environment
variable.
If you want to use another profile name than ``default``, set the
``AWS_PROFILE`` environment variable.
If you prefer to configure AWS credentials by environment variables
(eg. when using docker), you can use the following variables:
- AWS_ACCESS_KEY_ID: the access key for your AWS account,
- AWS_SECRET_ACCESS_KEY: the secret key for your AWS account.
Please read `boto3 documentation`_ for more details.
Further configuration
---------------------
In some cases you might want to use compatible S3 storage which is not on
AWS.
For this you have the following variables :
- AWS_S3_ENDPOINT_URL: endpoint url for your alternative S3 storage
This enables the use of a `minio server <https://min.io>`_, see
`their documentation about endpoints
<https://docs.min.io/docs/aws-cli-with-minio>`_.
Note : this type of variable `has been requested upstream
<https://github.com/boto/boto3/issues/2099>`_
(and related `aws-cli <https://github.com/aws/aws-cli/issues/4454>`_).
.. _`boto3 documentation`: https://boto3.readthedocs.io/en/stable/guide/configuration.html#shared-credentials-file
.. _boto3: https://github.com/boto/boto3
.. _Cubicweb: https://www.cubicweb.org/project/cubicweb
.. _BytesFileSystemStorage: http://cubicweb.readthedocs.io/en/3.26.2/tutorials/advanced/part03_bfss
Raw data
{
"_id": null,
"home_page": "https://forge.extranet.logilab.fr/cubicweb/cubes/cubicweb-s3storage",
"name": "cubicweb-s3storage",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "LOGILAB S.A. (Paris, FRANCE)",
"author_email": "contact@logilab.fr",
"download_url": "https://files.pythonhosted.org/packages/c2/42/0456ce77766cb62ead1f860e4918f0586eccc5353c2daa89be470472fa19/cubicweb-s3storage-4.0.1.tar.gz",
"platform": null,
"description": "Summary\n-------\n\nA Cubicweb Storage that stores the data on S3.\n\nIt is a standard Cubicweb_ Storage, similar to the BytesFileSystemStorage_. For\nexample, if you want to store File.data payloads in S3, you should create an\nS3Storage instance, typically in a startup hook, as::\n\n\n from cubicweb.server.hook import Hook\n from cubicweb_s3storage.storages import S3Storage\n\n class S3StorageStartupHook(Hook):\n __regid__ = 'mycube.server-startup-hook'\n events = ('server_startup', 'server_maintenance')\n\n def __call__(self):\n storage = S3Storage('s3_bucket')\n self.repo.system_source.set_storage('File', 'data', storage)\n\nBy default, S3 object keys (ie. S3 object identifiers) are random UUID\ngenerated at entity creation time. If you want to implement another key\ngeneration algorithm, you must write overload ``S3Storage.new_s3_key()``\nmethod.\n\n\nCommunications with AWS are handled by boto3_.\n\n\n.. Warning:: \n\n There is no cache, for now, so every time you read an entity\n attribute that is stored on S3, it will be downloaded from bucket provider.\n\nConfiguration\n-------------\n\nThe S3Storage class constructor needs an S3 bucket name.\n\nIf you want to put your aws credentials in an other file than the default\n(``~/.aws/credentials``), you must set the ``AWS_CONFIG_FILE`` environment\nvariable.\n\nIf you want to use another profile name than ``default``, set the\n``AWS_PROFILE`` environment variable.\n\nIf you prefer to configure AWS credentials by environment variables\n(eg. when using docker), you can use the following variables:\n\n- AWS_ACCESS_KEY_ID: the access key for your AWS account,\n\n- AWS_SECRET_ACCESS_KEY: the secret key for your AWS account.\n\nPlease read `boto3 documentation`_ for more details.\n\nFurther configuration\n---------------------\n\nIn some cases you might want to use compatible S3 storage which is not on\nAWS.\n\nFor this you have the following variables :\n\n- AWS_S3_ENDPOINT_URL: endpoint url for your alternative S3 storage\n\nThis enables the use of a `minio server <https://min.io>`_, see\n`their documentation about endpoints\n<https://docs.min.io/docs/aws-cli-with-minio>`_.\n\nNote : this type of variable `has been requested upstream\n<https://github.com/boto/boto3/issues/2099>`_\n(and related `aws-cli <https://github.com/aws/aws-cli/issues/4454>`_).\n\n\n.. _`boto3 documentation`: https://boto3.readthedocs.io/en/stable/guide/configuration.html#shared-credentials-file\n.. _boto3: https://github.com/boto/boto3\n.. _Cubicweb: https://www.cubicweb.org/project/cubicweb\n.. _BytesFileSystemStorage: http://cubicweb.readthedocs.io/en/3.26.2/tutorials/advanced/part03_bfss\n\n",
"bugtrack_url": null,
"license": "LGPL",
"summary": "A Cubicweb Storage that stores the data on S3",
"version": "4.0.1",
"project_urls": {
"Homepage": "https://forge.extranet.logilab.fr/cubicweb/cubes/cubicweb-s3storage"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1e1875e1421abb27ca58546444dd7930c06065684fa766528c8bbd39b80fa3ea",
"md5": "d75beb124381c2215597f27efc1d4b5a",
"sha256": "db4f3fb453cf8f2fa7a960735ef4de6d5fa21f7ba491011e805418337c28e487"
},
"downloads": -1,
"filename": "cubicweb_s3storage-4.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d75beb124381c2215597f27efc1d4b5a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 10510,
"upload_time": "2024-02-26T15:19:06",
"upload_time_iso_8601": "2024-02-26T15:19:06.635487Z",
"url": "https://files.pythonhosted.org/packages/1e/18/75e1421abb27ca58546444dd7930c06065684fa766528c8bbd39b80fa3ea/cubicweb_s3storage-4.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c2420456ce77766cb62ead1f860e4918f0586eccc5353c2daa89be470472fa19",
"md5": "aa39b1a1cfa2944d90e0ec8bfbdc0f19",
"sha256": "264b4514f0b3d1d43162919a27e20d621a49c6d8ad6ff00cf76b7060081f70ef"
},
"downloads": -1,
"filename": "cubicweb-s3storage-4.0.1.tar.gz",
"has_sig": false,
"md5_digest": "aa39b1a1cfa2944d90e0ec8bfbdc0f19",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10680,
"upload_time": "2024-02-26T15:19:08",
"upload_time_iso_8601": "2024-02-26T15:19:08.946245Z",
"url": "https://files.pythonhosted.org/packages/c2/42/0456ce77766cb62ead1f860e4918f0586eccc5353c2daa89be470472fa19/cubicweb-s3storage-4.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-26 15:19:08",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "cubicweb-s3storage"
}
JSON