# ckanext-toolbelt
Collection of different tools for daily use.
## Requirements
| CKAN version | Compatible? |
|--------------|-------------|
| 2.9 | yes |
| 2.10 | yes |
| master | yes |
## Content
* [Decorators](#decorators)
* [Collector](#collector)
* [Cache](#cache)
* [Plugins](#plugins)
* [CLI](#cli)
* [Misc](#misc)
## Decorators (`ckanext.toolbelt.decorators`)
### `Collector`
Creates a decorator that collects functions and returns them in a
dictionary. Originally designed for actions, auth functions, validators and
helpers.
:information_source: CKAN v2.10 has `tk.blanket` module. It does the same
things in a bit different manner.
Can be used as decorator. Call `Collector.get_collection` when you need
dictionary with names of helpers mapped to helper functions
helper = Collector()
@helper
def func():
pass
###
# ITemplateHelpers
def get_helpers(self):
return helper.get_collection()
`Collector.split` allows you to visually separate decorator from the method,
that returns collection
action, get_actions = Collector().split()
@action
def func():
pass
###
# IActions
def get_actions(self):
return get_actions()
If you want your functions prefixed by the plugin name, provide this prefix as
a first argument to the `Collector`'s constructor. If particular items must
remain unprefixed, you can specify what name to use, when decorating an item
validator, get_validators = Collector("toolbelt").split()
@validator
def func():
"""I am toolbelt_func
"""
pass
@validator("custom_func")
def func():
"""I am custom_func
"""
pass
###
# IValidators
def get_validators(self):
return get_validators()
[Back to content](#content)
### `Cache`
Cache for functions.
Cache()
def func(v):
return v * v
By default, cache is based on:
* module, where function is defined
* name of the function
* positional arguments
* named arguments
That means that following two invocations cached separately:
func(10)
func(v=10)
Cached data stored in redis as a JSON serialized structure. In order to use
different serializers, you can specify `dumper` and `loader` parameters when
creating `Cache` instance. Any function that accepts single value and returns
`str` or `bytes` can be used as a `dumper`. Any function that accepts `str` or
`bytes` and returns unserialized value can be used as loader.
from pickle import dumps, loads
@Cache(dumper=dumps, loader=loads)
def func(v):
return v
As mentioned before, cache key computed using module, name of the function and
parameters. It can be changed by passing a function as `key` argument to the
`Cache` constructor. Expected signature is `key_strategy(func, *args,
**kwargs)`.
# this function will be called only once, because cache key is based on its name.
# And name will never change. Unless you change it
@Cache(key=lambda f, *a, **k: f.__name__)
def func(v):
return v
Cache duration(in seconds) can be configured via `duration` parameter of the
constructor(which can be a callable that returns comuted duration).
cache = Cache(duration=3600)
@cache
def func(v):
return v + v
[Back to content](#content)
---
## Plugins
### `toolbelt_fdt_sqlalchemy`
Adapter for
[Flask-SQLAlchemy](https://flask-sqlalchemy.palletsprojects.com/en/3.0.x/). Enables
SQLAlchemy panel on FlaskDebugToolbar. You have to install `flask-sqlalchemy`
extra to use this plugin:
```sh
pip install 'ckanext-toolbelt[flask-sqlalchemy]'
```
### `toolbelt_cascade_organization_updates`
Reindex all organization's datasets when organization updated. Requires
background worker.
### `toolbelt_composite_groups` / `toolbelt_composite_organizations`
Enable repeating subfields(ckanext-scheming) for organization and group schemas
[Back to content](#content)
---
## CLI
As soon as you've installed ckanext-toolbelt, it will register `ckan toolbelt`
route for CLI. You don't have to add `toolbelt` to the list of enabled
plugins.
Depending on the active plugins, extra subroutes may be added to the `ckan
toolbelt` route.
In addition, there is global `ctb` command that allows to use this package
without CKAN installed or without CKAN config file. But in this way some of
commands (`search-index` for example) are not available, because they use CKAN
core. `ctb` alias exists for setting up the CKAN or extensions and running
generic services, that do not rely on CKAN instance.
Global commands, available via `ctb` and `ckan toolbelt` routes:
```sh
# create a generic configuration. Supported types:
# * deps-makefile CKAN dependency manager
# * pre-commit Pre-commit
# * pyproject pyproject.toml
# * gulp-sass gulpfile.js with SASS configuration
make config <type>
# create a configuration for GitHub Action. Supported types:
# * pypi-publish Publish package to PyPI when vX.Y.Z tag added.
# * release-please Create a PR that compiles changelog and publishes GitHub release.
# * test Test workflow.
make gh-action <type>
# Generate parts of README.md
# Supported types:
# * config Print declared config options for the given plugins.
make readme <type>
# Start mail server that will catch outcomming mails.
dev mail-server
```
Commands that depends on CKAN core and available only via `ckan toolbelt`
route:
```sh
# Drop packages that are only in search index but not in DB.
search-index clear-missing
# Clean the DB, optionally keeping data in the given tables.
db clean --yes [-k user] [-k group] [-k ...]
```
[Back to content](#content)
---
## Misc
### `ckanext.toolbelt.utils.cache`
#### `DontCache`
#### `Cache`
### `ckanext.toolbelt.utils.fs`
#### StaticPath
No-op wrapper around filepath that can be used as a context manager:
```python
with StaticPath("/tmp/x.txt") as path:
with open(path) as src:
...
# nothing is changed
```
#### RemovablePath
Context manager that removes file on exit:
```python
with RemovablePath("/tmp/x.txt") as path:
with open(path) as src:
...
# /tmp/x.txt is removed
```
#### `path_to_resource(res_dict, max_size=0)`
Returns a filepath for a resource.
If resource is stored locally, return StaticPath. If resource stored remotely,
download it to /tmp and return RemovablePath. Remote resources with size
exceeding `max_size` are not downloaded and empty StaticPath returned.
Example:
```python
with path_to_resource(resource) as path:
with open(path) as src:
print(src.read())
```
### `ckanext.toolbelt.utils.scheming`
#### `get_validation_schema`
### `ckanext.toolbelt.utils.structures`
#### `Node`
### `ckanext.toolbelt.utils.hierarchy`
#### `Node`
#### `Strategy`
#### `ParentReference`
#### `package_hierarchy`
[Back to content](#content)
Raw data
{
"_id": null,
"home_page": "https://github.com/DataShades/ckanext-toolbelt",
"name": "ckanext-toolbelt",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "CKAN",
"author": "Sergey Motornyuk",
"author_email": "sergey.motornyuk@linkdigital.com.au",
"download_url": "https://files.pythonhosted.org/packages/c0/1f/bd493c3f6a7a4be090191e4ef2dbefd66d2fe9648fd7dd49e02ed171f68c/ckanext-toolbelt-0.4.14.tar.gz",
"platform": null,
"description": "# ckanext-toolbelt\n\nCollection of different tools for daily use.\n\n\n## Requirements\n\n| CKAN version | Compatible? |\n|--------------|-------------|\n| 2.9 | yes |\n| 2.10 | yes |\n| master | yes |\n\n\n## Content\n\n* [Decorators](#decorators)\n * [Collector](#collector)\n * [Cache](#cache)\n* [Plugins](#plugins)\n* [CLI](#cli)\n* [Misc](#misc)\n\n\n## Decorators (`ckanext.toolbelt.decorators`)\n\n### `Collector`\n\nCreates a decorator that collects functions and returns them in a\ndictionary. Originally designed for actions, auth functions, validators and\nhelpers.\n\n:information_source: CKAN v2.10 has `tk.blanket` module. It does the same\nthings in a bit different manner.\n\nCan be used as decorator. Call `Collector.get_collection` when you need\ndictionary with names of helpers mapped to helper functions\n\n\thelper = Collector()\n\n\t@helper\n\tdef func():\n\t\tpass\n\n\t###\n # ITemplateHelpers\n\tdef get_helpers(self):\n\t\treturn helper.get_collection()\n\n`Collector.split` allows you to visually separate decorator from the method,\nthat returns collection\n\n\taction, get_actions = Collector().split()\n\n\t@action\n\tdef func():\n\t\tpass\n\n\t###\n # IActions\n\tdef get_actions(self):\n\t\treturn get_actions()\n\nIf you want your functions prefixed by the plugin name, provide this prefix as\na first argument to the `Collector`'s constructor. If particular items must\nremain unprefixed, you can specify what name to use, when decorating an item\n\n\n\tvalidator, get_validators = Collector(\"toolbelt\").split()\n\n\t@validator\n\tdef func():\n\t\t\"\"\"I am toolbelt_func\n\t\t\"\"\"\n\t\tpass\n\n\t@validator(\"custom_func\")\n\tdef func():\n\t\t\"\"\"I am custom_func\n\t\t\"\"\"\n\t\tpass\n\n\t###\n # IValidators\n\tdef get_validators(self):\n\t\treturn get_validators()\n\n\n[Back to content](#content)\n\n### `Cache`\n\nCache for functions.\n\n\tCache()\n\tdef func(v):\n\t return v * v\n\nBy default, cache is based on:\n\n* module, where function is defined\n* name of the function\n* positional arguments\n* named arguments\n\nThat means that following two invocations cached separately:\n\n\tfunc(10)\n\tfunc(v=10)\n\nCached data stored in redis as a JSON serialized structure. In order to use\ndifferent serializers, you can specify `dumper` and `loader` parameters when\ncreating `Cache` instance. Any function that accepts single value and returns\n`str` or `bytes` can be used as a `dumper`. Any function that accepts `str` or\n`bytes` and returns unserialized value can be used as loader.\n\n\tfrom pickle import dumps, loads\n\n\t@Cache(dumper=dumps, loader=loads)\n\tdef func(v):\n\t return v\n\nAs mentioned before, cache key computed using module, name of the function and\nparameters. It can be changed by passing a function as `key` argument to the\n`Cache` constructor. Expected signature is `key_strategy(func, *args,\n**kwargs)`.\n\n\t# this function will be called only once, because cache key is based on its name.\n\t# And name will never change. Unless you change it\n\t@Cache(key=lambda f, *a, **k: f.__name__)\n\tdef func(v):\n\t return v\n\nCache duration(in seconds) can be configured via `duration` parameter of the\nconstructor(which can be a callable that returns comuted duration).\n\n\tcache = Cache(duration=3600)\n\n\t@cache\n\tdef func(v):\n\t return v + v\n\n[Back to content](#content)\n\n---\n\n## Plugins\n\n### `toolbelt_fdt_sqlalchemy`\n\nAdapter for\n[Flask-SQLAlchemy](https://flask-sqlalchemy.palletsprojects.com/en/3.0.x/). Enables\nSQLAlchemy panel on FlaskDebugToolbar. You have to install `flask-sqlalchemy`\nextra to use this plugin:\n\n```sh\npip install 'ckanext-toolbelt[flask-sqlalchemy]'\n```\n\n### `toolbelt_cascade_organization_updates`\n\nReindex all organization's datasets when organization updated. Requires\nbackground worker.\n\n### `toolbelt_composite_groups` / `toolbelt_composite_organizations`\n\nEnable repeating subfields(ckanext-scheming) for organization and group schemas\n\n\n[Back to content](#content)\n---\n\n## CLI\n\nAs soon as you've installed ckanext-toolbelt, it will register `ckan toolbelt`\nroute for CLI. You don't have to add `toolbelt` to the list of enabled\nplugins.\n\nDepending on the active plugins, extra subroutes may be added to the `ckan\ntoolbelt` route.\n\nIn addition, there is global `ctb` command that allows to use this package\nwithout CKAN installed or without CKAN config file. But in this way some of\ncommands (`search-index` for example) are not available, because they use CKAN\ncore. `ctb` alias exists for setting up the CKAN or extensions and running\ngeneric services, that do not rely on CKAN instance.\n\n\nGlobal commands, available via `ctb` and `ckan toolbelt` routes:\n\n```sh\n# create a generic configuration. Supported types:\n# * deps-makefile CKAN dependency manager\n# * pre-commit Pre-commit\n# * pyproject pyproject.toml\n# * gulp-sass gulpfile.js with SASS configuration\nmake config <type>\n\n# create a configuration for GitHub Action. Supported types:\n# * pypi-publish Publish package to PyPI when vX.Y.Z tag added.\n# * release-please Create a PR that compiles changelog and publishes GitHub release.\n# * test Test workflow.\nmake gh-action <type>\n\n# Generate parts of README.md\n# Supported types:\n# * config Print declared config options for the given plugins.\nmake readme <type>\n\n# Start mail server that will catch outcomming mails.\ndev mail-server\n```\n\nCommands that depends on CKAN core and available only via `ckan toolbelt`\nroute:\n```sh\n\n# Drop packages that are only in search index but not in DB.\nsearch-index clear-missing\n\n# Clean the DB, optionally keeping data in the given tables.\ndb clean --yes [-k user] [-k group] [-k ...]\n```\n\n[Back to content](#content)\n\n---\n\n## Misc\n\n### `ckanext.toolbelt.utils.cache`\n#### `DontCache`\n#### `Cache`\n\n### `ckanext.toolbelt.utils.fs`\n#### StaticPath\n\nNo-op wrapper around filepath that can be used as a context manager:\n```python\nwith StaticPath(\"/tmp/x.txt\") as path:\n with open(path) as src:\n ...\n# nothing is changed\n```\n\n#### RemovablePath\n\nContext manager that removes file on exit:\n```python\nwith RemovablePath(\"/tmp/x.txt\") as path:\n with open(path) as src:\n ...\n# /tmp/x.txt is removed\n```\n\n#### `path_to_resource(res_dict, max_size=0)`\n\nReturns a filepath for a resource.\n\nIf resource is stored locally, return StaticPath. If resource stored remotely,\ndownload it to /tmp and return RemovablePath. Remote resources with size\nexceeding `max_size` are not downloaded and empty StaticPath returned.\n\nExample:\n```python\nwith path_to_resource(resource) as path:\n with open(path) as src:\n print(src.read())\n```\n\n\n### `ckanext.toolbelt.utils.scheming`\n#### `get_validation_schema`\n\n### `ckanext.toolbelt.utils.structures`\n#### `Node`\n\n### `ckanext.toolbelt.utils.hierarchy`\n#### `Node`\n#### `Strategy`\n#### `ParentReference`\n#### `package_hierarchy`\n\n\n[Back to content](#content)\n",
"bugtrack_url": null,
"license": "AGPL",
"summary": "",
"version": "0.4.14",
"project_urls": {
"Homepage": "https://github.com/DataShades/ckanext-toolbelt"
},
"split_keywords": [
"ckan"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ba0b920fc73478338f874f1e8ae66df06998d09b1f37507980723bb403ef76d1",
"md5": "2a4c3f90fa6bb54f3bea2b7138e998b0",
"sha256": "197de3bdd5993cd1151bb42f4d5c430d457b39f4b757b9ff095025e2ddc19a53"
},
"downloads": -1,
"filename": "ckanext_toolbelt-0.4.14-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2a4c3f90fa6bb54f3bea2b7138e998b0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 51324,
"upload_time": "2024-03-06T13:36:15",
"upload_time_iso_8601": "2024-03-06T13:36:15.487936Z",
"url": "https://files.pythonhosted.org/packages/ba/0b/920fc73478338f874f1e8ae66df06998d09b1f37507980723bb403ef76d1/ckanext_toolbelt-0.4.14-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c01fbd493c3f6a7a4be090191e4ef2dbefd66d2fe9648fd7dd49e02ed171f68c",
"md5": "772907daff0459de3baa27be95a77518",
"sha256": "2696e252dfc6518251e3833b4b7da9833d2788281092fe49a6f0d2e177ba48d7"
},
"downloads": -1,
"filename": "ckanext-toolbelt-0.4.14.tar.gz",
"has_sig": false,
"md5_digest": "772907daff0459de3baa27be95a77518",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 45145,
"upload_time": "2024-03-06T13:36:20",
"upload_time_iso_8601": "2024-03-06T13:36:20.831788Z",
"url": "https://files.pythonhosted.org/packages/c0/1f/bd493c3f6a7a4be090191e4ef2dbefd66d2fe9648fd7dd49e02ed171f68c/ckanext-toolbelt-0.4.14.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-06 13:36:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DataShades",
"github_project": "ckanext-toolbelt",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "ckanext-toolbelt"
}
JSON
