[](https://github.com/DataShades/ckanext-files/actions/workflows/test.yml)
# ckanext-files
Files as first-class citizens of CKAN. Upload, manage, remove files directly
and attach them to datasets, resources, etc.
## Requirements
Compatibility with core CKAN versions:
| CKAN version | Compatible? |
|--------------|-------------|
| 2.8 | yes |
| 2.9 | yes |
| 2.10 | yes |
| master | yes |
CKAN v2.8 and v2.9 are supported by ckanext-files v0.2. Starting from v1.0 this
extension switches to CKAN support policy of two latest CKAN releases. I.e,
ckanext-files v1.0 supports only CKAN v2.10 and v2.11.
v0.2 will not receive any new features, only bug-fixes.
It's recommended to install the extension via pip, so you probably have all the
requirements pinned already. If you are using GitHub version of this extension,
stick to the vX.Y.Z tags to avoid breaking changes. Check the changelog before
upgrading the extension.
## Installation
To install ckanext-files:
1. Install the extension
```sh
# minimal installation
pip install ckanext-files
# Google Cloud Storage support
pip install 'ckanext-files[gcs]'
```
1. Add `files` to the `ckan.plugins` setting in your CKAN
config file.
1. Run DB migrations
```sh
# CKAN >= v2.9
ckan db upgrade -p files
# CKAN == v2.8
paster --plugin=ckanext-files files -c ckan.ini initdb
```
## Usage
### Configure the storage
Before uploading any file, you have to configure a named **storage**. Tell
extension which driver to use(i.e, where and how data will be stored), add few
storage specific settings and you are ready to go. Let's start from the Redis
driver, because it has minimal requirements in terms of configuration.
Add the following line to the CKAN config file:
```ini
ckanext.files.storage.my_storage.type = files:redis
```
Look at this option. The prefix `ckanext.files.storage.` will be the same for
every configuration option related to storage. Then comes `my_storage`. It's a
name of your storage. It doesn't change the behavior of the storage, but
determines how options are grouped together and helps you to use multiple
storages simultaneously. For example the following configuration adds two
different storages:`first_storage` with `prefix` set to `first:` and
`second_storage` with prefix set to `second:`:
```ini
ckanext.files.storage.first_storage.type = files:redis
ckanext.files.storage.second_storage.type = files:redis
ckanext.files.storage.first_storage.prefix = first:
ckanext.files.storage.second_storage.prefix = second:
```
Our storage is called `my_storage` and after the storage's name comes option's
name `type`. And it's value `files:redis`. `type` option tells us which driver
we are going to use for the storage. `files:redis` is the name of the driver.
The recommended format of the driver name is
`<EXTENSION_NAME>:<STORAGE_TYPE>`. But this format is not enforced, so you can
see any value there. If you accidentally make a typo in the driver's name,
starting from v2.10 CKAN will show you an error message on startup with the
list of available drivers:
```sh
Invalid configuration values provided:
ckanext.files.storage.my_storage.type: Value must be one of ['files:bq_google_cloud_storage', 'files:fs', 'files:public_fs', 'files:redis', 'files:google_cloud_storage']
Aborted!
```
Storage is configured, so we can actually upload the file. Let's use
[ckanapi](https://github.com/ckan/ckanapi) for this task. Files are created via
`files_file_create` API action and this time we have to pass 3 parameters into it:
* `name`: the name of uploaded file
* `upload`: content of the file
* `storage`: name of the storage that stores the file(we just configured `my_storage`)
The final command is here:
```sh
ckanapi action files_file_create \
name=hello.txt \
upload='hello world' \
storage=my_storage
```
And that's what you see as result:
```json
{
"atime": null,
"completed": true,
"ctime": "2024-03-12T22:08:05.185914",
"id": "7f9a7676-5177-40f0-b610-0b47918fdccc",
"mtime": null,
"name": "hello.txt",
"storage": "my_storage",
"storage_data": {
"content_type": "text/plain",
"filename": "2f51aeff-96a5-4d79-8973-7867527d7f2e",
"hash": "5eb63bbbe01eeed093cb22bb8f5acdc3",
"size": 11
}
}
```
Now go to Redis CLI and check the content of the file. Note, you cannot get the
content via CKAN API, because it's JSON-based and downloading files doesn't
suit its principles.
By default, Redis driver puts the content under the key
`<PREFIX><FILENAME>`. Pay attention to `FILENAME`. It's the value available as
`storage_data.filename` in the API response(i.e,
`2f51aeff-96a5-4d79-8973-7867527d7f2e` in our case), not the name of the real
file you just uploaded.
`PREFIX` can be configured, but we skipped this step and got the default value: `ckanext:files:default:file_content:`. So the final Redis key of our file is `ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e`
```sh
redis-cli
127.0.0.1:6379> GET ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e
"hello world"
```
And before we moved further, let's remove the file, using ID we got when it was created:
```sh
ckanapi action files_file_delete id=7f9a7676-5177-40f0-b610-0b47918fdccc
# ... response from api
redis-cli
127.0.0.1:6379> GET ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e
(nil)
```
### Usage in code
If you are writing the code and you want to interact with the storage directly,
without the API layer, you can do it via a number of public functions of the
extension.
Let's configure filesystem storage first. This time we are going to use the
storage name `default`. This is "default" storage name - if you haven't
specified explicitely the name of the storage when working with it(calling API
actions, for example), `default` storage is used. So it's better to create this
storage instead of making fancy names.
Filesystem driver has a mandatory option `path` that controls path, where files
are stored. If path does not exist, storage will raise an exception by
default. But it can also create missing path if you enable `create_path`
option. Here's our final version of settings:
```ini
ckanext.files.storage.default.type = files:fs
ckanext.files.storage.default.path = /tmp/example
ckanext.files.storage.default.create_path = true
```
Now we are going to connect to CKAN shell via `ckan shell` CLI command and
create an instance of the storage:
```python
from ckanext.files.shared import get_storage
storage = get_storage('default')
```
Because you have all configuration in place, the rest is fairly
straightforward. We will upload the file, read it's content and remove it from
the CKAN shell.
Uploading is the most challenging step. As CKAN is based on Flask, we have to
upload files as Flask does. For it we are going to create an instance of `werkzeug.datastructures.FileStorage` and pass bytes stream to its constructor. After that, we can pass the object into storage's `upload` method and specify the name of the upload.
```python
from werkzeug.datastructures import FileStorage
from io import BytesIO
upload = FileStorage(BytesIO(b'hello world'))
result = storage.upload('file.txt', upload, {})
print(result)
... {'filename': 'd65da0fd-299a-433c-850e-086fdc5ebb7e',
... 'content_type': None,
... 'size': 11,
... 'hash': '5eb63bbbe01eeed093cb22bb8f5acdc3'}
```
`result` contains minimal amount of information that is required by storage to
manage the file. In case of FS storage, it includes `filename`. If you visit
`/tmp/example` directory, that we specified as a `path` for our storage, you'll
see there a file with the name matching `filename` from result. And its content
matches the content of our upload, which is quite an expected outcome.
But let's go back to the shell and try reading file from the python's
code. We'll pass `result` to the storage's `stream` method, which produces a
readable buffer based on our result. This buffer has `read` method so we can
work with the buffer just as we usually do with IO streams:
```python
buffer = storage.stream(result)
content = buffer.read()
print(content)
... b'hello world'
```
And finally we need to remove the file
```python
storage.remove(result)
```
### Usage in browser
You can upload files using JavaScript CKAN modules. The extension extends
CKAN's Sandbox object(available as `this.sandbox` inside the JS CKAN module),
so we can use shortcut and upload file directly from the DevTools. Open any
CKAN page, switch to JS console and create the sandbox instance. Inside it we
have `files` object, which in turn contains `upload` method. This method
accepts `File` object for upload(the same object you can get from the
`input[type=file]`).
```js
sandbox = ckan.sandbox()
await sandbox.files.upload(
new File(["content"], "file.txt")
)
... {
... "id": "d6fa3096-613a-4b16-a0eb-b7a6ec91d690",
... "name": "file.txt",
... "storage": "default",
... "ctime": "2024-03-12T23:08:11.291971",
... "mtime": null,
... "atime": null,
... "storage_data": {
... "filename": "f484d116-cf6f-4238-80e5-905849873be0",
... "content_type": "application/octet-stream",
... "size": 7,
... "hash": "9a0364b9e99bb480dd25e1f0284c8555"
... },
... "completed": true
... }
```
If you are still using FS storage configured in previous section, switch to
`/tmp/example` folder and check it's content:
```sh
ls /tmp/example
... f484d116-cf6f-4238-80e5-905849873be0
cat f484d116-cf6f-4238-80e5-905849873be0
... content
```
And, as usually, let's remove file using the ID from the `upload` promise:
```js
sandbox.client.call('POST', 'files_file_delete', {
id: 'd6fa3096-613a-4b16-a0eb-b7a6ec91d690'
})
```
### UI
This extension provides a basic UI as an example and it's recommended to build
your own UI if you need it. But we can still use default functionality for
testing. Open the `/user/<YOUR USERNAME>/files` page of CKAN application in the
browser.
Choose the file
Click on play button(the one with triangle icon) and wait a bit. If nothing
changes, check JS console, it can show some errors when storage is not properly
configured.
When upload finished, reload the page to see uploaded file.
## Configuration
There are two types of config options for ckanext-files:
* Global configuration affects the common behavior of the extension
* Storage configuration changes behavior of the specific storage and never
affects anything outside of the storage
Depending on the type of the storage, available options for storage change. For
example, `files:fs` storage type requires `path` option that controls
filesystem path where uploads are stored. `files:redis` storage type accepts
`prefix` option that defines Redis' key prefix of files stored in Redis. All
storage specific options always have form
`ckanext.files.storage.<STORAGE>.<OPTION>`:
```ini
ckanext.files.storage.memory.prefix = xxx:
# or
ckanext.files.storage.my_drive.path = /tmp/hello
```
Below is the list of non-storage specific options. Details of the specific
storage type can be found in the dedicated section of the storage type.
```ini
# Default storage used for upload when no explicit storage specified
# (optional, default: default)
ckanext.files.default_storage = default
# Configuration of the named storage.
# (optional, default: )
ckanext.files.storage.<NAME>.<OPTION> =
```
Starting from CKAN v2.10 you can check all available options for the storage
type via config declarations CLI. First, add the storage type to the config
file:
```ini
ckanext.files.storage.xxx.type = files:redis
```
Now run the command that shows all available config option of the
plugin.
```sh
ckan config declaration files -d
```
Because redis storage adapter is enabled, you'll see all the options
regsitered by redis driver alongside with the global options:
```ini
## ckanext-files ###############################################################
## ...
## Storage adapter used by the storage
ckanext.files.storage.xxx.type = files:redis
## The maximum size of a single upload.
## Supports size suffixes: 42B, 2M, 24KiB, 1GB. `0` means no restrictions.
ckanext.files.storage.xxx.max_size = 0
## Descriptive name of the storage used for debugging.
ckanext.files.storage.xxx.name = xxx
## Static prefix of the Redis key generated for every upload.
ckanext.files.storage.xxx.prefix = ckanext:files:default:file_content:
```
Sometimes you will see a validation error if storage has required config
options. Let's try using `files:fs` storage instead of the redis:
```ini
ckanext.files.storage.xxx.type = files:fs
```
Now attempt to run `ckan config declaration files -d` will show an error,
because required `path` option is missing:
```sh
Invalid configuration values provided:
ckanext.files.storage.xxx.path: Missing value
Aborted!
```
Add the required option to satisfy the application
```ini
ckanext.files.storage.xxx.type = files:fs
ckanext.files.storage.xxx.path = /tmp
```
And run CLI command once again. This time you'll see the list of allowed
options:
```ini
## ckanext-files ###############################################################
## ...
## Storage adapter used by the storage
ckanext.files.storage.xxx.type = files:fs
## The maximum size of a single upload.
## Supports size suffixes: 42B, 2M, 24KiB, 1GB. `0` means no restrictions.
ckanext.files.storage.xxx.max_size = 0
## Descriptive name of the storage used for debugging.
ckanext.files.storage.xxx.name = xxx
## Path to the folder where uploaded data will be stored.
ckanext.files.storage.xxx.path =
## Create storage folder if it does not exist.
ckanext.files.storage.xxx.create_path = false
```
Raw data
{
"_id": null,
"home_page": "https://github.com/DataShades/ckanext-files",
"name": "ckanext-files",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "CKAN",
"author": "Sergey Motornyuk",
"author_email": "sergey.motornyuk@linkdigital.com.au",
"download_url": "https://files.pythonhosted.org/packages/3b/ef/8a0b600744a5e87e4cc1125d27aeaf056d8b9e35d9d383c4630793c41492/ckanext_files-0.2.6.tar.gz",
"platform": null,
"description": "[](https://github.com/DataShades/ckanext-files/actions/workflows/test.yml)\n\n# ckanext-files\n\nFiles as first-class citizens of CKAN. Upload, manage, remove files directly\nand attach them to datasets, resources, etc.\n\n## Requirements\n\nCompatibility with core CKAN versions:\n\n| CKAN version | Compatible? |\n|--------------|-------------|\n| 2.8 | yes |\n| 2.9 | yes |\n| 2.10 | yes |\n| master | yes |\n\nCKAN v2.8 and v2.9 are supported by ckanext-files v0.2. Starting from v1.0 this\nextension switches to CKAN support policy of two latest CKAN releases. I.e,\nckanext-files v1.0 supports only CKAN v2.10 and v2.11.\n\nv0.2 will not receive any new features, only bug-fixes.\n\nIt's recommended to install the extension via pip, so you probably have all the\nrequirements pinned already. If you are using GitHub version of this extension,\nstick to the vX.Y.Z tags to avoid breaking changes. Check the changelog before\nupgrading the extension.\n\n## Installation\n\nTo install ckanext-files:\n\n1. Install the extension\n ```sh\n # minimal installation\n pip install ckanext-files\n\n # Google Cloud Storage support\n pip install 'ckanext-files[gcs]'\n ```\n\n1. Add `files` to the `ckan.plugins` setting in your CKAN\n config file.\n\n1. Run DB migrations\n ```sh\n # CKAN >= v2.9\n ckan db upgrade -p files\n\n # CKAN == v2.8\n paster --plugin=ckanext-files files -c ckan.ini initdb\n ```\n\n## Usage\n\n### Configure the storage\n\nBefore uploading any file, you have to configure a named **storage**. Tell\nextension which driver to use(i.e, where and how data will be stored), add few\nstorage specific settings and you are ready to go. Let's start from the Redis\ndriver, because it has minimal requirements in terms of configuration.\n\nAdd the following line to the CKAN config file:\n\n```ini\nckanext.files.storage.my_storage.type = files:redis\n```\n\nLook at this option. The prefix `ckanext.files.storage.` will be the same for\nevery configuration option related to storage. Then comes `my_storage`. It's a\nname of your storage. It doesn't change the behavior of the storage, but\ndetermines how options are grouped together and helps you to use multiple\nstorages simultaneously. For example the following configuration adds two\ndifferent storages:`first_storage` with `prefix` set to `first:` and\n`second_storage` with prefix set to `second:`:\n\n```ini\nckanext.files.storage.first_storage.type = files:redis\nckanext.files.storage.second_storage.type = files:redis\n\nckanext.files.storage.first_storage.prefix = first:\nckanext.files.storage.second_storage.prefix = second:\n```\n\nOur storage is called `my_storage` and after the storage's name comes option's\nname `type`. And it's value `files:redis`. `type` option tells us which driver\nwe are going to use for the storage. `files:redis` is the name of the driver.\n\nThe recommended format of the driver name is\n`<EXTENSION_NAME>:<STORAGE_TYPE>`. But this format is not enforced, so you can\nsee any value there. If you accidentally make a typo in the driver's name,\nstarting from v2.10 CKAN will show you an error message on startup with the\nlist of available drivers:\n\n```sh\nInvalid configuration values provided:\nckanext.files.storage.my_storage.type: Value must be one of ['files:bq_google_cloud_storage', 'files:fs', 'files:public_fs', 'files:redis', 'files:google_cloud_storage']\nAborted!\n```\n\nStorage is configured, so we can actually upload the file. Let's use\n[ckanapi](https://github.com/ckan/ckanapi) for this task. Files are created via\n`files_file_create` API action and this time we have to pass 3 parameters into it:\n\n* `name`: the name of uploaded file\n* `upload`: content of the file\n* `storage`: name of the storage that stores the file(we just configured `my_storage`)\n\nThe final command is here:\n\n```sh\nckanapi action files_file_create \\\n name=hello.txt \\\n upload='hello world' \\\n storage=my_storage\n```\n\nAnd that's what you see as result:\n\n```json\n{\n \"atime\": null,\n \"completed\": true,\n \"ctime\": \"2024-03-12T22:08:05.185914\",\n \"id\": \"7f9a7676-5177-40f0-b610-0b47918fdccc\",\n \"mtime\": null,\n \"name\": \"hello.txt\",\n \"storage\": \"my_storage\",\n \"storage_data\": {\n \"content_type\": \"text/plain\",\n \"filename\": \"2f51aeff-96a5-4d79-8973-7867527d7f2e\",\n \"hash\": \"5eb63bbbe01eeed093cb22bb8f5acdc3\",\n \"size\": 11\n }\n}\n```\n\nNow go to Redis CLI and check the content of the file. Note, you cannot get the\ncontent via CKAN API, because it's JSON-based and downloading files doesn't\nsuit its principles.\n\nBy default, Redis driver puts the content under the key\n`<PREFIX><FILENAME>`. Pay attention to `FILENAME`. It's the value available as\n`storage_data.filename` in the API response(i.e,\n`2f51aeff-96a5-4d79-8973-7867527d7f2e` in our case), not the name of the real\nfile you just uploaded.\n\n`PREFIX` can be configured, but we skipped this step and got the default value: `ckanext:files:default:file_content:`. So the final Redis key of our file is `ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e`\n\n```sh\nredis-cli\n\n127.0.0.1:6379> GET ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e\n\"hello world\"\n\n```\n\nAnd before we moved further, let's remove the file, using ID we got when it was created:\n\n```sh\nckanapi action files_file_delete id=7f9a7676-5177-40f0-b610-0b47918fdccc\n\n# ... response from api\n\nredis-cli\n\n127.0.0.1:6379> GET ckanext:files:default:file_content:2f51aeff-96a5-4d79-8973-7867527d7f2e\n(nil)\n\n```\n\n### Usage in code\n\nIf you are writing the code and you want to interact with the storage directly,\nwithout the API layer, you can do it via a number of public functions of the\nextension.\n\nLet's configure filesystem storage first. This time we are going to use the\nstorage name `default`. This is \"default\" storage name - if you haven't\nspecified explicitely the name of the storage when working with it(calling API\nactions, for example), `default` storage is used. So it's better to create this\nstorage instead of making fancy names.\n\nFilesystem driver has a mandatory option `path` that controls path, where files\nare stored. If path does not exist, storage will raise an exception by\ndefault. But it can also create missing path if you enable `create_path`\noption. Here's our final version of settings:\n\n```ini\nckanext.files.storage.default.type = files:fs\nckanext.files.storage.default.path = /tmp/example\nckanext.files.storage.default.create_path = true\n```\n\nNow we are going to connect to CKAN shell via `ckan shell` CLI command and\ncreate an instance of the storage:\n\n```python\nfrom ckanext.files.shared import get_storage\nstorage = get_storage('default')\n```\n\nBecause you have all configuration in place, the rest is fairly\nstraightforward. We will upload the file, read it's content and remove it from\nthe CKAN shell.\n\nUploading is the most challenging step. As CKAN is based on Flask, we have to\nupload files as Flask does. For it we are going to create an instance of `werkzeug.datastructures.FileStorage` and pass bytes stream to its constructor. After that, we can pass the object into storage's `upload` method and specify the name of the upload.\n\n```python\nfrom werkzeug.datastructures import FileStorage\nfrom io import BytesIO\n\nupload = FileStorage(BytesIO(b'hello world'))\nresult = storage.upload('file.txt', upload, {})\n\nprint(result)\n\n... {'filename': 'd65da0fd-299a-433c-850e-086fdc5ebb7e',\n... 'content_type': None,\n... 'size': 11,\n... 'hash': '5eb63bbbe01eeed093cb22bb8f5acdc3'}\n\n```\n\n`result` contains minimal amount of information that is required by storage to\nmanage the file. In case of FS storage, it includes `filename`. If you visit\n`/tmp/example` directory, that we specified as a `path` for our storage, you'll\nsee there a file with the name matching `filename` from result. And its content\nmatches the content of our upload, which is quite an expected outcome.\n\nBut let's go back to the shell and try reading file from the python's\ncode. We'll pass `result` to the storage's `stream` method, which produces a\nreadable buffer based on our result. This buffer has `read` method so we can\nwork with the buffer just as we usually do with IO streams:\n\n```python\nbuffer = storage.stream(result)\ncontent = buffer.read()\nprint(content)\n\n... b'hello world'\n```\n\nAnd finally we need to remove the file\n\n```python\nstorage.remove(result)\n```\n\n### Usage in browser\n\nYou can upload files using JavaScript CKAN modules. The extension extends\nCKAN's Sandbox object(available as `this.sandbox` inside the JS CKAN module),\nso we can use shortcut and upload file directly from the DevTools. Open any\nCKAN page, switch to JS console and create the sandbox instance. Inside it we\nhave `files` object, which in turn contains `upload` method. This method\naccepts `File` object for upload(the same object you can get from the\n`input[type=file]`).\n\n```js\nsandbox = ckan.sandbox()\nawait sandbox.files.upload(\n new File([\"content\"], \"file.txt\")\n)\n\n... {\n... \"id\": \"d6fa3096-613a-4b16-a0eb-b7a6ec91d690\",\n... \"name\": \"file.txt\",\n... \"storage\": \"default\",\n... \"ctime\": \"2024-03-12T23:08:11.291971\",\n... \"mtime\": null,\n... \"atime\": null,\n... \"storage_data\": {\n... \"filename\": \"f484d116-cf6f-4238-80e5-905849873be0\",\n... \"content_type\": \"application/octet-stream\",\n... \"size\": 7,\n... \"hash\": \"9a0364b9e99bb480dd25e1f0284c8555\"\n... },\n... \"completed\": true\n... }\n```\n\nIf you are still using FS storage configured in previous section, switch to\n`/tmp/example` folder and check it's content:\n\n```sh\nls /tmp/example\n... f484d116-cf6f-4238-80e5-905849873be0\n\ncat f484d116-cf6f-4238-80e5-905849873be0\n... content\n```\n\nAnd, as usually, let's remove file using the ID from the `upload` promise:\n\n```js\nsandbox.client.call('POST', 'files_file_delete', {\n id: 'd6fa3096-613a-4b16-a0eb-b7a6ec91d690'\n})\n```\n\n### UI\n\nThis extension provides a basic UI as an example and it's recommended to build\nyour own UI if you need it. But we can still use default functionality for\ntesting. Open the `/user/<YOUR USERNAME>/files` page of CKAN application in the\nbrowser.\n\n\n\nChoose the file\n\n\n\nClick on play button(the one with triangle icon) and wait a bit. If nothing\nchanges, check JS console, it can show some errors when storage is not properly\nconfigured.\n\n\n\nWhen upload finished, reload the page to see uploaded file.\n\n\n\n## Configuration\n\nThere are two types of config options for ckanext-files:\n* Global configuration affects the common behavior of the extension\n* Storage configuration changes behavior of the specific storage and never\n affects anything outside of the storage\n\nDepending on the type of the storage, available options for storage change. For\nexample, `files:fs` storage type requires `path` option that controls\nfilesystem path where uploads are stored. `files:redis` storage type accepts\n`prefix` option that defines Redis' key prefix of files stored in Redis. All\nstorage specific options always have form\n`ckanext.files.storage.<STORAGE>.<OPTION>`:\n\n```ini\nckanext.files.storage.memory.prefix = xxx:\n# or\nckanext.files.storage.my_drive.path = /tmp/hello\n```\n\nBelow is the list of non-storage specific options. Details of the specific\nstorage type can be found in the dedicated section of the storage type.\n\n```ini\n\n# Default storage used for upload when no explicit storage specified\n# (optional, default: default)\nckanext.files.default_storage = default\n\n# Configuration of the named storage.\n# (optional, default: )\nckanext.files.storage.<NAME>.<OPTION> =\n\n```\n\nStarting from CKAN v2.10 you can check all available options for the storage\ntype via config declarations CLI. First, add the storage type to the config\nfile:\n\n```ini\nckanext.files.storage.xxx.type = files:redis\n```\n\nNow run the command that shows all available config option of the\nplugin.\n\n```sh\nckan config declaration files -d\n```\n\nBecause redis storage adapter is enabled, you'll see all the options\nregsitered by redis driver alongside with the global options:\n\n\n```ini\n## ckanext-files ###############################################################\n## ...\n## Storage adapter used by the storage\nckanext.files.storage.xxx.type = files:redis\n## The maximum size of a single upload.\n## Supports size suffixes: 42B, 2M, 24KiB, 1GB. `0` means no restrictions.\nckanext.files.storage.xxx.max_size = 0\n## Descriptive name of the storage used for debugging.\nckanext.files.storage.xxx.name = xxx\n## Static prefix of the Redis key generated for every upload.\nckanext.files.storage.xxx.prefix = ckanext:files:default:file_content:\n```\n\nSometimes you will see a validation error if storage has required config\noptions. Let's try using `files:fs` storage instead of the redis:\n\n```ini\nckanext.files.storage.xxx.type = files:fs\n```\n\nNow attempt to run `ckan config declaration files -d` will show an error,\nbecause required `path` option is missing:\n\n```sh\nInvalid configuration values provided:\nckanext.files.storage.xxx.path: Missing value\nAborted!\n```\n\nAdd the required option to satisfy the application\n\n```ini\nckanext.files.storage.xxx.type = files:fs\nckanext.files.storage.xxx.path = /tmp\n```\n\nAnd run CLI command once again. This time you'll see the list of allowed\noptions:\n\n```ini\n## ckanext-files ###############################################################\n## ...\n## Storage adapter used by the storage\nckanext.files.storage.xxx.type = files:fs\n## The maximum size of a single upload.\n## Supports size suffixes: 42B, 2M, 24KiB, 1GB. `0` means no restrictions.\nckanext.files.storage.xxx.max_size = 0\n## Descriptive name of the storage used for debugging.\nckanext.files.storage.xxx.name = xxx\n## Path to the folder where uploaded data will be stored.\nckanext.files.storage.xxx.path =\n## Create storage folder if it does not exist.\nckanext.files.storage.xxx.create_path = false\n```\n",
"bugtrack_url": null,
"license": "AGPL",
"summary": null,
"version": "0.2.6",
"project_urls": {
"Homepage": "https://github.com/DataShades/ckanext-files"
},
"split_keywords": [
"ckan"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "54da7af7a12a2b39061d048743a27561e18e3e59c9ff3f637d954a7dc5851b89",
"md5": "133496e3f11fa1de589f1d88ad685942",
"sha256": "a8ba7a3ad39232741efd6a479f4c95df2760be344682578c197253c87745726d"
},
"downloads": -1,
"filename": "ckanext_files-0.2.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "133496e3f11fa1de589f1d88ad685942",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 102016,
"upload_time": "2024-04-24T11:53:27",
"upload_time_iso_8601": "2024-04-24T11:53:27.275453Z",
"url": "https://files.pythonhosted.org/packages/54/da/7af7a12a2b39061d048743a27561e18e3e59c9ff3f637d954a7dc5851b89/ckanext_files-0.2.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3bef8a0b600744a5e87e4cc1125d27aeaf056d8b9e35d9d383c4630793c41492",
"md5": "fe99dcb4cf375d8ab8229d6a87831fa9",
"sha256": "2309e49543ee03fce74f6cc95b2a2b713fe4be1f694c4b7f5fe82f19c08f312e"
},
"downloads": -1,
"filename": "ckanext_files-0.2.6.tar.gz",
"has_sig": false,
"md5_digest": "fe99dcb4cf375d8ab8229d6a87831fa9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 88495,
"upload_time": "2024-04-24T11:53:29",
"upload_time_iso_8601": "2024-04-24T11:53:29.786922Z",
"url": "https://files.pythonhosted.org/packages/3b/ef/8a0b600744a5e87e4cc1125d27aeaf056d8b9e35d9d383c4630793c41492/ckanext_files-0.2.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-24 11:53:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DataShades",
"github_project": "ckanext-files",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "ckanext-files"
}
JSON
