GUILLOTINA_GCLOUDSTORAGE
========================
GCloud blob storage for guillotina.
Example config.json entry:
.. code-block:: json
...
"cloud_storage": "guillotina_gcloudstorage.interfaces.IGCloudFileField",
"cloud_datamanager": "redis",
"load_utilities": {
"gcloud": {
"provides": "guillotina_gcloudstorage.interfaces.IGCloudBlobStore",
"factory": "guillotina_gcloudstorage.storage.GCloudBlobStore",
"settings": {
"uniform_bucket_level_access": True,
"json_credentials": "/path/to/credentials.json",
"bucket": "name-of-bucket",
"bucket_name_format": "{container}-foobar{delimiter}{base}",
"bucket_labels": {
"foo": "bar"
}
}
}
}
...
Getting started with development
--------------------------------
Using pip (requires Python > 3.7):
.. code-block:: shell
python3.7 -m venv .
./bin/pip install -e .[test]
pre-commit install
6.0.3 (2023-03-21)
------------------
- Add delete method
6.0.2 (2021-05-05)
------------------
- Allow to delete multiattachment fields
[bloodbare]
6.0.1 (2021-02-04)
------------------
- Get get access token in executor
[vangheem]
- Use IFileNameGenerator adapter to generate file name
- black, isort reformat
[qiwn]
6.0.0 (2020-07-29)
------------------
- Moving to Guillotina 6
- Added github actions
- Linting: black and isort
[lferran]
5.0.14 (2020-06-08)
-------------------
- Handle Conflict error when creating bucket. This can happen when multiple pods are attempting to
create bucket at the same time.
[vangheem]
5.0.13 (2020-02-20)
-------------------
- Allow configuring uniform bucket level access by setting uniform_bucket_level_access
[vangheem]
- Do not update bucket labels if they have not changed
[vangheem]
5.0.12 (2020-01-02)
-------------------
- Add missing `range_supported`
[vangheem]
5.0.11 (2020-01-02)
-------------------
- Add range support
[vangheem]
- Apply black to code
5.0.10 (2019-12-20)
-------------------
- Handle 404 on delete
[vangheem]
5.0.9 (2019-12-20)
------------------
- Handle 401 error
[vangheem]
5.0.8 (2019-12-20)
------------------
- Handle 410 errors from google on upload
[vangheem]
5.0.7 (2019-12-19)
------------------
- More logging info
[vangheem]
5.0.6 (2019-12-19)
------------------
- More error handling and retries
[vangheem]
5.0.5 (2019-12-18)
------------------
- No backoff on iter_data
[vangheem]
5.0.4 (2019-12-18)
------------------
- More backoff decorators to address intermittent API issues
[vangheem]
5.0.3 (2019-11-08)
------------------
- Fix `GCloudFileManager.append()`
[qiwn]
5.0.2 (2019-11-01)
------------------
- Be able to import types
[vangheem]
5.0.1 (2019-08-30)
------------------
- Fix get_client to not be called in executor because context vars do not work in task vars
[vangheem]
5.0.0 (2019-06-23)
------------------
- Upgrade to work with latest guillotina >= 5
[vangheem]
2.0.12 (2019-06-18)
-------------------
- restrict guillotina version
2.0.11 (2019-06-06)
-------------------
- fix release
2.0.10 (2019-06-06)
-------------------
- Reuse aiohttp client session
[vangheem]
2.0.9 (2019-05-07)
------------------
- Fix creating buckets with different versions of google cloud storage
[vangheem]
2.0.8 (2019-04-12)
------------------
- Fix bug creating buckets
[vangheem]
2.0.7 (2019-04-09)
------------------
- Add `get_client` and `_create_bucket` methods to `GCloudBlobStore`
[vangheem]
2.0.6 (2019-03-21)
------------------
- Adding location parameter [bloodbare]
2.0.5 (2019-03-08)
------------------
- Add `bucket_name_format` and `bucket_labels` settings
[vangheem]
2.0.4 (2019-01-17)
------------------
- Credentials compatiblity [bloodbare]
2.0.3 (2019-01-15)
------------------
- Raise 404 if object no longer available
[vangheem]
2.0.2 (2019-01-15)
------------------
- Implement exists for head requests
[vangheem]
2.0.1 (2018-12-07)
------------------
- Use quote_plus when starting multi part upload. Fixes issues
with `+` in content ids not working.
- No need to run get_access_token in executor
[lferran]
2.0.0 (2018-06-07)
------------------
- Upgrade to guillotina 4
[vangheem]
[vangheem]
1.1.7 (2018-06-07)
------------------
- Pin version of guillotina
[vangheem]
1.1.6 (2018-06-07)
------------------
- Handle 404 when copying files
[vangheem]
1.1.5 (2018-05-12)
------------------
- bump
1.1.4 (2018-05-12)
------------------
- More strict object checks
[vangheem]
1.1.3 (2018-03-20)
------------------
- Another logging fix
[vangheem]
1.1.2 (2018-03-20)
------------------
- Fix logging issue
[vangheem]
1.1.1 (2018-03-19)
------------------
- Be able to use `iter_data` with custom uri
[vangheem]
1.1.0 (2018-03-19)
------------------
- Upgrade to latest guillotina file management to simplify code-base
[vangheem]
1.0.36 (2018-03-09)
-------------------
- Fix saving previous file
[vangheem]
1.0.35 (2018-03-01)
-------------------
- Change when we store previous file info
[vangheem]
1.0.34 (2018-02-22)
-------------------
- Customize more of the download
[vangheem]
1.0.33 (2018-02-22)
-------------------
- Be able to specify uri to download
[vangheem]
1.0.32 (2018-02-21)
-------------------
- Tweak IFileCleanup
[vangheem]
1.0.31 (2018-02-20)
-------------------
- Implement IFileCleanup
[vangheem]
1.0.30 (2018-01-02)
-------------------
- Retry google cloud exceptions
[vangheem]
1.0.29 (2017-10-30)
-------------------
- Handle file size being zero for download reporting
[vangheem]
1.0.28 (2017-10-12)
-------------------
- Make sure to register write on object for behavior files
[vangheem]
1.0.27 (2017-10-11)
-------------------
- Return NotFound response when no file found on context
[vangheem]
1.0.26 (2017-10-04)
-------------------
- Handle google cloud error when deleting existing files
[vangheem]
1.0.25 (2017-10-03)
-------------------
- Check type instead of None for existing value
[vangheem]
1.0.24 (2017-10-02)
-------------------
- Use latest guillotina base classes
[vangheem]
- Use field context if set
[vangheem]
1.0.23 (2017-10-02)
-------------------
- Add copy_cloud_file method
[vangheem]
1.0.22 (2017-09-29)
-------------------
- Limit request limit cache size to a max of the CHUNK_SIZE
[vangheem]
1.0.21 (2017-09-29)
-------------------
- Cache data on request object in case of request conflict errors
[vangheem]
1.0.20 (2017-09-27)
-------------------
- Do not timeout when downloading for gcloud
[vangheem]
- Make sure to use async with syntax with aiohttp requests
[vangheem]
1.0.19 (2017-09-11)
-------------------
- Make sure CORS headers are applied before we start sending a download result
[vangheem]
1.0.18 (2017-09-11)
-------------------
- Be able to override disposition of download
[vangheem]
1.0.17 (2017-09-01)
-------------------
- Implement save_file method
[vangheem]
1.0.16 (2017-08-15)
-------------------
- Provide iter_data method
[vangheem]
1.0.15 (2017-06-15)
-------------------
- Guess content type if none provided when downloading file
[vangheem]
1.0.14 (2017-06-14)
-------------------
- Be able to customize content disposition header of file download
[vangheem]
1.0.13 (2017-06-12)
-------------------
- Remove GCloudBlobStore._service property
[vangheem]
- Change GCloudBlobStore._bucket to GCloudBlobStore._bucket_name
[vangheem]
- Remove GCloudBlobStore._client property
[vangheem]
- Rename GCloudBlobStore.bucket property to coroutine:GCloudBlobStore.get_bucket_name()
[vangheem]
- Make everything async and use executor if necessary so we don't block
[vangheem]
1.0.12 (2017-05-19)
-------------------
- Provide iterate_bucket method
[vangheem]
1.0.11 (2017-05-19)
-------------------
- provide method to rename object
[vangheem]
- Use keys that use the object's oid
[vangheem]
- Make delete async
[vangheem]
1.0.10 (2017-05-02)
-------------------
- Convert bytes to string for content_type
[vangheem]
1.0.9 (2017-05-02)
------------------
- contentType was renamed to content_type
[vangheem]
1.0.8 (2017-05-02)
------------------
- Make sure to register the object for writing to the database
[vangheem]
1.0.7 (2017-05-01)
------------------
- Fix reference to _md5hash instead of _md5 so serializing works
[vangheem]
1.0.6 (2017-05-01)
------------------
- Fix bytes serialization issue
[vangheem]
1.0.5 (2017-05-01)
------------------
- Fix import error
[vangheem]
1.0.4 (2017-05-01)
------------------
- Do not inherit from BaseObject
[vangheem]
1.0.3 (2017-05-01)
------------------
- Allow GCloudFile to take all arguments
[vangheem]
1.0.2 (2017-04-26)
------------------
- Need to be able to provide loop param in constructor of utility
[vangheem]
1.0.1 (2017-04-25)
------------------
- Compatibility fixes with aiohttp 2
[vangheem]
1.0.0 (2017-04-24)
------------------
- initial release
Raw data
{
"_id": null,
"home_page": "https://pypi.python.org/pypi/guillotina_gcloudstorage",
"name": "guillotina-gcloudstorage",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "guillotina async cloud storage",
"author": "Ramon Navarro Bosch",
"author_email": "ramon@plone.org",
"download_url": "https://files.pythonhosted.org/packages/74/c3/fe17950e7546c983b236114ad18c8320ee21eaf77e532a572b8b7ad0d420/guillotina_gcloudstorage-6.0.3.tar.gz",
"platform": null,
"description": "GUILLOTINA_GCLOUDSTORAGE\n========================\n\nGCloud blob storage for guillotina.\n\n\nExample config.json entry:\n\n.. code-block:: json\n\n ...\n \"cloud_storage\": \"guillotina_gcloudstorage.interfaces.IGCloudFileField\",\n \"cloud_datamanager\": \"redis\",\n \"load_utilities\": {\n \"gcloud\": {\n \"provides\": \"guillotina_gcloudstorage.interfaces.IGCloudBlobStore\",\n \"factory\": \"guillotina_gcloudstorage.storage.GCloudBlobStore\",\n \"settings\": {\n \"uniform_bucket_level_access\": True,\n \"json_credentials\": \"/path/to/credentials.json\",\n \"bucket\": \"name-of-bucket\",\n \"bucket_name_format\": \"{container}-foobar{delimiter}{base}\",\n \"bucket_labels\": {\n \"foo\": \"bar\"\n }\n }\n }\n }\n ...\n\n\nGetting started with development\n--------------------------------\n\nUsing pip (requires Python > 3.7):\n\n.. code-block:: shell\n\n python3.7 -m venv .\n ./bin/pip install -e .[test]\n pre-commit install\n\n\n6.0.3 (2023-03-21)\n------------------\n\n- Add delete method\n\n\n6.0.2 (2021-05-05)\n------------------\n\n- Allow to delete multiattachment fields\n [bloodbare]\n\n\n6.0.1 (2021-02-04)\n------------------\n\n- Get get access token in executor\n [vangheem]\n\n- Use IFileNameGenerator adapter to generate file name\n- black, isort reformat\n [qiwn]\n\n\n6.0.0 (2020-07-29)\n------------------\n\n- Moving to Guillotina 6\n- Added github actions\n- Linting: black and isort\n [lferran]\n\n\n5.0.14 (2020-06-08)\n-------------------\n\n- Handle Conflict error when creating bucket. This can happen when multiple pods are attempting to\n create bucket at the same time.\n [vangheem]\n\n5.0.13 (2020-02-20)\n-------------------\n\n- Allow configuring uniform bucket level access by setting uniform_bucket_level_access\n [vangheem]\n\n- Do not update bucket labels if they have not changed\n [vangheem]\n\n\n5.0.12 (2020-01-02)\n-------------------\n\n- Add missing `range_supported`\n [vangheem]\n\n\n5.0.11 (2020-01-02)\n-------------------\n\n- Add range support\n [vangheem]\n\n- Apply black to code\n\n\n5.0.10 (2019-12-20)\n-------------------\n\n- Handle 404 on delete\n [vangheem]\n\n\n5.0.9 (2019-12-20)\n------------------\n\n- Handle 401 error\n [vangheem]\n\n\n5.0.8 (2019-12-20)\n------------------\n\n- Handle 410 errors from google on upload\n [vangheem]\n\n\n5.0.7 (2019-12-19)\n------------------\n\n- More logging info\n [vangheem]\n\n\n5.0.6 (2019-12-19)\n------------------\n\n- More error handling and retries\n [vangheem]\n\n5.0.5 (2019-12-18)\n------------------\n\n- No backoff on iter_data\n [vangheem]\n\n\n5.0.4 (2019-12-18)\n------------------\n\n- More backoff decorators to address intermittent API issues\n [vangheem]\n\n\n5.0.3 (2019-11-08)\n------------------\n\n- Fix `GCloudFileManager.append()`\n [qiwn]\n\n\n5.0.2 (2019-11-01)\n------------------\n\n- Be able to import types\n [vangheem]\n\n\n5.0.1 (2019-08-30)\n------------------\n\n- Fix get_client to not be called in executor because context vars do not work in task vars\n [vangheem]\n\n\n5.0.0 (2019-06-23)\n------------------\n\n- Upgrade to work with latest guillotina >= 5\n [vangheem]\n\n\n2.0.12 (2019-06-18)\n-------------------\n\n- restrict guillotina version\n\n\n2.0.11 (2019-06-06)\n-------------------\n\n- fix release\n\n\n2.0.10 (2019-06-06)\n-------------------\n\n- Reuse aiohttp client session\n [vangheem]\n\n\n2.0.9 (2019-05-07)\n------------------\n\n- Fix creating buckets with different versions of google cloud storage\n [vangheem]\n\n\n2.0.8 (2019-04-12)\n------------------\n\n- Fix bug creating buckets\n [vangheem]\n\n\n2.0.7 (2019-04-09)\n------------------\n\n- Add `get_client` and `_create_bucket` methods to `GCloudBlobStore`\n [vangheem]\n\n\n2.0.6 (2019-03-21)\n------------------\n\n- Adding location parameter [bloodbare]\n\n\n2.0.5 (2019-03-08)\n------------------\n\n- Add `bucket_name_format` and `bucket_labels` settings\n [vangheem]\n\n\n2.0.4 (2019-01-17)\n------------------\n\n- Credentials compatiblity [bloodbare]\n\n\n2.0.3 (2019-01-15)\n------------------\n\n- Raise 404 if object no longer available\n [vangheem]\n\n\n2.0.2 (2019-01-15)\n------------------\n\n- Implement exists for head requests\n [vangheem]\n\n\n2.0.1 (2018-12-07)\n------------------\n\n- Use quote_plus when starting multi part upload. Fixes issues\n with `+` in content ids not working.\n\n- No need to run get_access_token in executor\n [lferran]\n\n2.0.0 (2018-06-07)\n------------------\n\n- Upgrade to guillotina 4\n [vangheem]\n [vangheem]\n\n\n1.1.7 (2018-06-07)\n------------------\n\n- Pin version of guillotina\n [vangheem]\n\n\n1.1.6 (2018-06-07)\n------------------\n\n- Handle 404 when copying files\n [vangheem]\n\n\n1.1.5 (2018-05-12)\n------------------\n\n- bump\n\n\n1.1.4 (2018-05-12)\n------------------\n\n- More strict object checks\n [vangheem]\n\n\n1.1.3 (2018-03-20)\n------------------\n\n- Another logging fix\n [vangheem]\n\n\n1.1.2 (2018-03-20)\n------------------\n\n- Fix logging issue\n [vangheem]\n\n\n1.1.1 (2018-03-19)\n------------------\n\n- Be able to use `iter_data` with custom uri\n [vangheem]\n\n\n1.1.0 (2018-03-19)\n------------------\n\n- Upgrade to latest guillotina file management to simplify code-base\n [vangheem]\n\n\n1.0.36 (2018-03-09)\n-------------------\n\n- Fix saving previous file\n [vangheem]\n\n\n1.0.35 (2018-03-01)\n-------------------\n\n- Change when we store previous file info\n [vangheem]\n\n\n1.0.34 (2018-02-22)\n-------------------\n\n- Customize more of the download\n [vangheem]\n\n\n1.0.33 (2018-02-22)\n-------------------\n\n- Be able to specify uri to download\n [vangheem]\n\n\n1.0.32 (2018-02-21)\n-------------------\n\n- Tweak IFileCleanup\n [vangheem]\n\n\n1.0.31 (2018-02-20)\n-------------------\n\n- Implement IFileCleanup\n [vangheem]\n\n\n1.0.30 (2018-01-02)\n-------------------\n\n- Retry google cloud exceptions\n [vangheem]\n\n\n1.0.29 (2017-10-30)\n-------------------\n\n- Handle file size being zero for download reporting\n [vangheem]\n\n\n1.0.28 (2017-10-12)\n-------------------\n\n- Make sure to register write on object for behavior files\n [vangheem]\n\n\n1.0.27 (2017-10-11)\n-------------------\n\n- Return NotFound response when no file found on context\n [vangheem]\n\n\n1.0.26 (2017-10-04)\n-------------------\n\n- Handle google cloud error when deleting existing files\n [vangheem]\n\n\n1.0.25 (2017-10-03)\n-------------------\n\n- Check type instead of None for existing value\n [vangheem]\n\n\n1.0.24 (2017-10-02)\n-------------------\n\n- Use latest guillotina base classes\n [vangheem]\n\n- Use field context if set\n [vangheem]\n\n\n1.0.23 (2017-10-02)\n-------------------\n\n- Add copy_cloud_file method\n [vangheem]\n\n\n1.0.22 (2017-09-29)\n-------------------\n\n- Limit request limit cache size to a max of the CHUNK_SIZE\n [vangheem]\n\n\n1.0.21 (2017-09-29)\n-------------------\n\n- Cache data on request object in case of request conflict errors\n [vangheem]\n\n\n1.0.20 (2017-09-27)\n-------------------\n\n- Do not timeout when downloading for gcloud\n [vangheem]\n\n- Make sure to use async with syntax with aiohttp requests\n [vangheem]\n\n\n1.0.19 (2017-09-11)\n-------------------\n\n- Make sure CORS headers are applied before we start sending a download result\n [vangheem]\n\n\n1.0.18 (2017-09-11)\n-------------------\n\n- Be able to override disposition of download\n [vangheem]\n\n\n1.0.17 (2017-09-01)\n-------------------\n\n- Implement save_file method\n [vangheem]\n\n\n1.0.16 (2017-08-15)\n-------------------\n\n- Provide iter_data method\n [vangheem]\n\n\n1.0.15 (2017-06-15)\n-------------------\n\n- Guess content type if none provided when downloading file\n [vangheem]\n\n\n1.0.14 (2017-06-14)\n-------------------\n\n- Be able to customize content disposition header of file download\n [vangheem]\n\n\n1.0.13 (2017-06-12)\n-------------------\n\n- Remove GCloudBlobStore._service property\n [vangheem]\n\n- Change GCloudBlobStore._bucket to GCloudBlobStore._bucket_name\n [vangheem]\n\n- Remove GCloudBlobStore._client property\n [vangheem]\n\n- Rename GCloudBlobStore.bucket property to coroutine:GCloudBlobStore.get_bucket_name()\n [vangheem]\n\n- Make everything async and use executor if necessary so we don't block\n [vangheem]\n\n\n1.0.12 (2017-05-19)\n-------------------\n\n- Provide iterate_bucket method\n [vangheem]\n\n\n1.0.11 (2017-05-19)\n-------------------\n\n- provide method to rename object\n [vangheem]\n\n- Use keys that use the object's oid\n [vangheem]\n\n- Make delete async\n [vangheem]\n\n\n1.0.10 (2017-05-02)\n-------------------\n\n- Convert bytes to string for content_type\n [vangheem]\n\n\n1.0.9 (2017-05-02)\n------------------\n\n- contentType was renamed to content_type\n [vangheem]\n\n\n1.0.8 (2017-05-02)\n------------------\n\n- Make sure to register the object for writing to the database\n [vangheem]\n\n\n1.0.7 (2017-05-01)\n------------------\n\n- Fix reference to _md5hash instead of _md5 so serializing works\n [vangheem]\n\n\n1.0.6 (2017-05-01)\n------------------\n\n- Fix bytes serialization issue\n [vangheem]\n\n\n1.0.5 (2017-05-01)\n------------------\n\n- Fix import error\n [vangheem]\n\n\n1.0.4 (2017-05-01)\n------------------\n\n- Do not inherit from BaseObject\n [vangheem]\n\n\n1.0.3 (2017-05-01)\n------------------\n\n- Allow GCloudFile to take all arguments\n [vangheem]\n\n\n1.0.2 (2017-04-26)\n------------------\n\n- Need to be able to provide loop param in constructor of utility\n [vangheem]\n\n\n1.0.1 (2017-04-25)\n------------------\n\n- Compatibility fixes with aiohttp 2\n [vangheem]\n\n\n1.0.0 (2017-04-24)\n------------------\n\n- initial release\n\n\n",
"bugtrack_url": null,
"license": "GPL version 3",
"summary": "guillotina gcloud storage support",
"version": "6.0.3",
"split_keywords": [
"guillotina",
"async",
"cloud",
"storage"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "74c3fe17950e7546c983b236114ad18c8320ee21eaf77e532a572b8b7ad0d420",
"md5": "c2bcb1a8eed13cb3e8f7fc3b6d3e0c48",
"sha256": "1a2c4b804f95c6d45f3b89b866227cf0a80452ba7877c07bae1b2cb7eca11152"
},
"downloads": -1,
"filename": "guillotina_gcloudstorage-6.0.3.tar.gz",
"has_sig": false,
"md5_digest": "c2bcb1a8eed13cb3e8f7fc3b6d3e0c48",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18836,
"upload_time": "2023-03-21T10:01:01",
"upload_time_iso_8601": "2023-03-21T10:01:01.949809Z",
"url": "https://files.pythonhosted.org/packages/74/c3/fe17950e7546c983b236114ad18c8320ee21eaf77e532a572b8b7ad0d420/guillotina_gcloudstorage-6.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-21 10:01:01",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "guillotina-gcloudstorage"
}