ckanext-dcor_schemas
====================
|PyPI Version| |Build Status| |Coverage Status|
This module introduces/lifts restrictions (authorization) for the management
of data and meta data on DCOR. The corresponding UI elements are modified
accordingly:
- Authorization (auth.py)
- datasets: do not allow deleting datasets unless they are drafts
- datasets: allow purging of deleted datasets
- datasets: do not allow switching to a more restrictive license
- datasets: do not allow changing the name (slug)
- datasets: do not allow adding resources to non-draft datasets
- datasets: do not allow to set the visibility of a public dataset to private
- datasets: do not allow uploading datasets via the web interface
- organization: do not allow bulk_update_delete (e.g. datasets by organization admins)
- resources: do not allow deleting resources unless they are drafts
- resources: only allow changing the "description"
- resources: do not allow setting a resource id when uploading
- user: allow all logged-in users to create datasets, circles, and collections
- Validation (validate.py)
- datasets: force user to select authors
- datasets: author list "authors" is CSV
- datasets: parse DOI field (remove URL part)
- datasets: force user to select a license
- datasets: restrict to basic CC licenses
- datasets: automatically generate dataset name (slug) using random characters
if necessary (does not apply to admins)
- datasets: a dataset without resources is considered to be a draft;
it's state cannot be set to "active"
- datasets: the state of a dataset that does not contain at least one
valid .rtdc resource cannot be set to "active"
- resources: do not allow uploading resources with the same name
for a dataset (important for ckanext-dcor_depot)
- resources: make sure the resource name matches the file name of the
upload; this is actually implemented in plugin.before_create
(IResourceController) and not in validate.py
- resources: custom resource name is overridden during upload
- resources: do not allow weird characters in resource names
- resources: restrict upload data extensions to .'.rtdc', '.ini', '.csv',
'.tsv', '.pdf', '.txt', '.jpg', '.png', '.tif', '.py', '.ipynb', '.poly',
'.sof', '.so2'
- resources: configuration metadata (using `dclab.dfn.config_funcs`)
- resources: 's3_available' and 's3_url' for identifying objects that
are available on S3 and for downloading public datasets.
- IPermissionLabels (plugin.py)
- Allow a user A to see user B's private dataset if the private dataset
is in a group that user A is a member of.
- UI Dataset:
- hide "add new resource" button in ``templates/package/resources.html``
- add field ``authors`` (csv list)
- add field ``doi`` (validator parses URLs)
- add field ``references`` (parses arxiv, bioRxiv, DOI, links)
- add CC license file ``licenses.json`` (only show less restrictive licenses
when editing the dataset)
- UI Organization:
- remove "Delete" button in bulk view
- UI Resource:
- Do not show these variables (because they are redundant):
['last modified', 'revision id', 'url type', 'state', 'on same domain']
(``templates/package/resource_read.html``)
- Show DC config data via "toggle-more"
- Add supplementary resource schema via json files located in
`dcor_schemas/resource_schema_supplements`
- Background jobs:
- set the mimetype for each dataset
- populate "dc:sec:key" metadata for each DC dataset
- generates sha256 hash upon resource creation
- populate etag resource property from S3 storage upon resource creation
- Configuration keywords:
- the ``ckanext.dcor_schemas.allow_content_listing_for_anon`` boolean
parameter can be set to False to prevent anonymous users to see
circles, colletions, and other content.
- the ``ckanext.dcor_schemas.allow_public_datasets`` boolean parameter
can be used to disable the creation of public datasets (e.g. for DCOR-med).
- the ``ckanext.dcor_schemas.json_resource_schema_dir`` parameter
can be used to specify a directory containing .json files that
define the supplementary resource schema. The default is
``package`` which means that the supplementary resource schema of
this extension is used.
- These DCOR-wide configuratio options for accessing S3 object storage
- ``endpoint_url``
- ``bucket_name``
- ``access_key_id``
- ``secret_access_key``
- ``ssl_verify``
- API extensions:
- ``resource_upload_s3_urls`` returns a dictionary containing the upload
URLs (single file or multipart) required for uploading a new resource
directly to S3
- ``resource_schema_supplements`` returns a dictionary of the
current supplementary resource schema
- ``supported_resource_suffixes`` returns a list of supported
resource suffixes
- CLI:
- CKAN command `list-circles` returns the list of DCOR circles
- CKAN command `list-collections` returns the list of DCOR collections
- CKAN command `list-group-resources <NAME>` returns the list of resources in
a DCOR circle or collection
- CKAN command `list-zombie-users` for users with no datasets and
no activity for a certain amount of time
- CKAN command `run-jobs-dcor-schemas` that runs all background
jobs for all resources (if not already done)
- CKAN command `dcor-move-dataset-to-circle` for moving a dataset to
a different circle
Installation
------------
Simply run
::
pip install ckanext-dcor_schemas
In the configuration file ckan.ini:
::
ckan.plugins = [...] dcor_schemas
ckan.extra_resource_fields = etag sha256
Testing
-------
If CKAN/DCOR is installed and setup for testing, this extension can
be tested with pytest:
::
pytest ckanext
Testing can also be done via vagrant in a virtualmachine using the
`dcor-test <https://app.vagrantup.com/paulmueller/boxes/dcor-test/>` image.
Make sure that `vagrant` and `virtualbox` are installed and run the
following commands in the root of this repository:
::
# Setup virtual machine using `Vagrantfile`
vagrant up
# Run the tests
vagrant ssh -- sudo bash /testing/vagrant-run-tests.sh
.. |PyPI Version| image:: https://img.shields.io/pypi/v/ckanext.dcor_schemas.svg
:target: https://pypi.python.org/pypi/ckanext.dcor_schemas
.. |Build Status| image:: https://img.shields.io/github/actions/workflow/status/DCOR-dev/ckanext-dcor_schemas/check.yml
:target: https://github.com/DCOR-dev/ckanext-dcor_schemas/actions?query=workflow%3AChecks
.. |Coverage Status| image:: https://img.shields.io/codecov/c/github/DCOR-dev/ckanext-dcor_schemas
:target: https://codecov.io/gh/DCOR-dev/ckanext-dcor_schemas
Raw data
{
"_id": null,
"home_page": null,
"name": "ckanext-dcor-schemas",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.8",
"maintainer_email": "Paul M\u00fcller <dev@craban.de>",
"keywords": "DC, DCOR, deformability, cytometry",
"author": "Paul M\u00fcller",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/a5/a9/63e22f5dbe85159679c9ac50a77c9b86aafa394745be167fbdd155556edb/ckanext_dcor_schemas-0.20.2.tar.gz",
"platform": null,
"description": "ckanext-dcor_schemas\n====================\n\n|PyPI Version| |Build Status| |Coverage Status|\n\nThis module introduces/lifts restrictions (authorization) for the management\nof data and meta data on DCOR. The corresponding UI elements are modified\naccordingly:\n\n- Authorization (auth.py)\n\n - datasets: do not allow deleting datasets unless they are drafts\n - datasets: allow purging of deleted datasets\n - datasets: do not allow switching to a more restrictive license\n - datasets: do not allow changing the name (slug)\n - datasets: do not allow adding resources to non-draft datasets\n - datasets: do not allow to set the visibility of a public dataset to private\n - datasets: do not allow uploading datasets via the web interface\n - organization: do not allow bulk_update_delete (e.g. datasets by organization admins)\n - resources: do not allow deleting resources unless they are drafts\n - resources: only allow changing the \"description\"\n - resources: do not allow setting a resource id when uploading\n - user: allow all logged-in users to create datasets, circles, and collections\n\n- Validation (validate.py)\n\n - datasets: force user to select authors\n - datasets: author list \"authors\" is CSV\n - datasets: parse DOI field (remove URL part)\n - datasets: force user to select a license\n - datasets: restrict to basic CC licenses\n - datasets: automatically generate dataset name (slug) using random characters\n if necessary (does not apply to admins)\n - datasets: a dataset without resources is considered to be a draft;\n it's state cannot be set to \"active\"\n - datasets: the state of a dataset that does not contain at least one\n valid .rtdc resource cannot be set to \"active\"\n - resources: do not allow uploading resources with the same name\n for a dataset (important for ckanext-dcor_depot)\n - resources: make sure the resource name matches the file name of the\n upload; this is actually implemented in plugin.before_create\n (IResourceController) and not in validate.py\n - resources: custom resource name is overridden during upload\n - resources: do not allow weird characters in resource names\n - resources: restrict upload data extensions to .'.rtdc', '.ini', '.csv',\n '.tsv', '.pdf', '.txt', '.jpg', '.png', '.tif', '.py', '.ipynb', '.poly',\n '.sof', '.so2'\n - resources: configuration metadata (using `dclab.dfn.config_funcs`)\n - resources: 's3_available' and 's3_url' for identifying objects that\n are available on S3 and for downloading public datasets.\n\n- IPermissionLabels (plugin.py)\n\n - Allow a user A to see user B's private dataset if the private dataset\n is in a group that user A is a member of.\n\n- UI Dataset:\n\n - hide \"add new resource\" button in ``templates/package/resources.html``\n - add field ``authors`` (csv list)\n - add field ``doi`` (validator parses URLs)\n - add field ``references`` (parses arxiv, bioRxiv, DOI, links)\n - add CC license file ``licenses.json`` (only show less restrictive licenses\n when editing the dataset)\n\n- UI Organization:\n\n - remove \"Delete\" button in bulk view\n\n- UI Resource:\n\n - Do not show these variables (because they are redundant):\n ['last modified', 'revision id', 'url type', 'state', 'on same domain']\n (``templates/package/resource_read.html``)\n - Show DC config data via \"toggle-more\"\n - Add supplementary resource schema via json files located in\n `dcor_schemas/resource_schema_supplements`\n\n- Background jobs:\n\n - set the mimetype for each dataset\n - populate \"dc:sec:key\" metadata for each DC dataset\n - generates sha256 hash upon resource creation\n - populate etag resource property from S3 storage upon resource creation\n\n- Configuration keywords:\n\n - the ``ckanext.dcor_schemas.allow_content_listing_for_anon`` boolean\n parameter can be set to False to prevent anonymous users to see\n circles, colletions, and other content.\n - the ``ckanext.dcor_schemas.allow_public_datasets`` boolean parameter\n can be used to disable the creation of public datasets (e.g. for DCOR-med).\n - the ``ckanext.dcor_schemas.json_resource_schema_dir`` parameter\n can be used to specify a directory containing .json files that\n define the supplementary resource schema. The default is\n ``package`` which means that the supplementary resource schema of\n this extension is used.\n\n - These DCOR-wide configuratio options for accessing S3 object storage\n\n - ``endpoint_url``\n - ``bucket_name``\n - ``access_key_id``\n - ``secret_access_key``\n - ``ssl_verify``\n\n- API extensions:\n\n - ``resource_upload_s3_urls`` returns a dictionary containing the upload\n URLs (single file or multipart) required for uploading a new resource\n directly to S3\n - ``resource_schema_supplements`` returns a dictionary of the\n current supplementary resource schema\n - ``supported_resource_suffixes`` returns a list of supported\n resource suffixes\n\n- CLI:\n\n - CKAN command `list-circles` returns the list of DCOR circles\n - CKAN command `list-collections` returns the list of DCOR collections\n - CKAN command `list-group-resources <NAME>` returns the list of resources in\n a DCOR circle or collection\n - CKAN command `list-zombie-users` for users with no datasets and\n no activity for a certain amount of time\n - CKAN command `run-jobs-dcor-schemas` that runs all background\n jobs for all resources (if not already done)\n - CKAN command `dcor-move-dataset-to-circle` for moving a dataset to\n a different circle\n\n\nInstallation\n------------\nSimply run\n\n::\n\n pip install ckanext-dcor_schemas\n\nIn the configuration file ckan.ini:\n\n::\n \n ckan.plugins = [...] dcor_schemas\n ckan.extra_resource_fields = etag sha256\n\n\nTesting\n-------\nIf CKAN/DCOR is installed and setup for testing, this extension can\nbe tested with pytest:\n\n::\n\n pytest ckanext\n\nTesting can also be done via vagrant in a virtualmachine using the\n`dcor-test <https://app.vagrantup.com/paulmueller/boxes/dcor-test/>` image.\nMake sure that `vagrant` and `virtualbox` are installed and run the\nfollowing commands in the root of this repository:\n\n::\n\n # Setup virtual machine using `Vagrantfile`\n vagrant up\n # Run the tests\n vagrant ssh -- sudo bash /testing/vagrant-run-tests.sh\n\n\n.. |PyPI Version| image:: https://img.shields.io/pypi/v/ckanext.dcor_schemas.svg\n :target: https://pypi.python.org/pypi/ckanext.dcor_schemas\n.. |Build Status| image:: https://img.shields.io/github/actions/workflow/status/DCOR-dev/ckanext-dcor_schemas/check.yml\n :target: https://github.com/DCOR-dev/ckanext-dcor_schemas/actions?query=workflow%3AChecks\n.. |Coverage Status| image:: https://img.shields.io/codecov/c/github/DCOR-dev/ckanext-dcor_schemas\n :target: https://codecov.io/gh/DCOR-dev/ckanext-dcor_schemas\n",
"bugtrack_url": null,
"license": "GNU Affero General Public License v3 or later (AGPLv3+)",
"summary": "Introduces or lifts restrictions (authorization) for managing data and metadata on DCOR",
"version": "0.20.2",
"project_urls": {
"changelog": "https://github.com/DCOR-dev/ckanext-dcor_schemas/blob/main/CHANGELOG",
"source": "https://github.com/DCOR-dev/ckanext-dcor_schemas",
"tracker": "https://github.com/DCOR-dev/ckanext-dcor_schemas/issues"
},
"split_keywords": [
"dc",
" dcor",
" deformability",
" cytometry"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c622490f701795f89c3ef06ce8b5292ee36172555a6313cd8194f60c088948b2",
"md5": "db6dab9115eafd7ab221acbf30423de4",
"sha256": "a1d0980b2369c70741ad886751ab12982dd9ec22e6f1b75d63fb39c837098577"
},
"downloads": -1,
"filename": "ckanext_dcor_schemas-0.20.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "db6dab9115eafd7ab221acbf30423de4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.8",
"size": 704868,
"upload_time": "2024-08-05T10:46:11",
"upload_time_iso_8601": "2024-08-05T10:46:11.239147Z",
"url": "https://files.pythonhosted.org/packages/c6/22/490f701795f89c3ef06ce8b5292ee36172555a6313cd8194f60c088948b2/ckanext_dcor_schemas-0.20.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a5a963e22f5dbe85159679c9ac50a77c9b86aafa394745be167fbdd155556edb",
"md5": "130756e4792caa645f016a719db81bfa",
"sha256": "7ac76f1d4b25a9e06bf6a4027e1b03c8bfa4f92942f23cea4b12d13600e71488"
},
"downloads": -1,
"filename": "ckanext_dcor_schemas-0.20.2.tar.gz",
"has_sig": false,
"md5_digest": "130756e4792caa645f016a719db81bfa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.8",
"size": 696007,
"upload_time": "2024-08-05T10:46:21",
"upload_time_iso_8601": "2024-08-05T10:46:21.687763Z",
"url": "https://files.pythonhosted.org/packages/a5/a9/63e22f5dbe85159679c9ac50a77c9b86aafa394745be167fbdd155556edb/ckanext_dcor_schemas-0.20.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-05 10:46:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DCOR-dev",
"github_project": "ckanext-dcor_schemas",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ckanext-dcor-schemas"
}
JSON
