A few caching data structures and other lossy things with capped sizes.
*Latest release 20240422.1*:
ConvCache docstring update.
## Class `CachingMapping(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)`
A caching front end for another mapping.
This is intended as a generic superclass for a proxy to a
slower mapping such as a database or remote key value store.
Note that this subclasses `MultiOpenMixin` to start/stop the worker `Thread`.
Users must enclose use of a `CachingMapping` in a `with` statement.
If subclasses also subclass `MultiOpenMixin` their `startup_shutdown`
method needs to also call our `startup_shutdown` method.
Example:
class Store:
""" A key value store with a slower backend.
"""
def __init__(self, mapping:Mapping):
self.mapping = CachingMapping(mapping)
.....
S = Store(slow_mapping)
with S:
... work with S ...
*Method `CachingMapping.__init__(self, mapping: Mapping, *, max_size=1024, queue_length=1024, delitem_bg: Optional[Callable[[Any], cs.result.Result]] = None, setitem_bg: Optional[Callable[[Any, Any], cs.result.Result]] = None, missing_fallthrough: bool = False)`*:
Initialise the cache.
Parameters:
* `mapping`: the backing store, a mapping
* `max_size`: optional maximum size for the cache, default 1024
* `queue_length`: option size for the queue to the worker, default 1024
* `delitem_bg`: optional callable to queue a delete of a
key in the backing store; if unset then deleted are
serialised in the worker thread
* `setitem_bg`: optional callable to queue setting the value
for a key in the backing store; if unset then deleted are
serialised in the worker thread
* `missing_fallthrough`: is true (default `False`) always
fall back to the backing mapping if a key is not in the cache
*Method `CachingMapping.flush(self)`*:
Wait for outstanding requests in the queue to complete.
Return the UNIX time of completion.
*Method `CachingMapping.items(self)`*:
Generator yielding `(k,v)` pairs.
*Method `CachingMapping.keys(self)`*:
Generator yielding the keys.
## Class `ConvCache(cs.fs.HasFSPath)`
A cache for conversions of file contents such as thumbnails
or transcoded media, etc. This keeps cached results in a file
tree based on a content key, whose default function is
`cs.hashutils.file_checksum('sha256')`.
*Method `ConvCache.__init__(self, fspath: Optional[str] = None, content_key_func=None)`*:
Initialise a `ConvCache`.
Parameters:
* `fspath`: optional base path of the cache, default from
`ConvCache.DEFAULT_CACHE_BASEPATH`;
if this does not exist it will be created using `os.mkdir`
* `content_key_func`: optional function to compute a key
from the contents of a file, default `cs.hashindex.file_checksum`
(the sha256 hash of the contents)
*Method `ConvCache.content_key(self, srcpath)`*:
Return a content key for the filesystem path `srcpath`.
*Method `ConvCache.content_subpath(self, srcpath)`*:
Return the content key based subpath component.
This default assumes the content key is a hash code and
breaks it hex representation into a 3 level hierarchy
such as `'d6/d9/c510785c468c9aa4b7bda343fb79'`.
*Method `ConvCache.convof(self, srcpath, conv_subpath, conv_func, ext=None)`*:
Return the filesystem path of the cached conversion of `srcpath` via `conv_func`.
Parameters:
* `srcpath`: the source filesystem path
* `conv_subpath`: a name for the conversion which encompasses
the salient aspaects such as `'png/64/64'` for a 64x64 pixel
thumbnail in PNG format
* `conv_func`: a callable of the form `conv_func(srcpath,dstpath)`
to convert the contents of `srcpath` and write the result
to the filesystem path `dstpath`
* `ext`: an optional filename extension, default from the
first component of `conv_subpath`
## Function `convof(srcpath, conv_subpath, conv_func, ext=None)`
`ConvCache.convof` using the default cache.
## Class `LRU_Cache`
A simple least recently used cache.
Unlike `functools.lru_cache`
this provides `on_add` and `on_remove` callbacks.
*Method `LRU_Cache.__init__(self, max_size, *, on_add=None, on_remove=None)`*:
Initialise the LRU_Cache with maximum size `max`,
additon callback `on_add` and removal callback `on_remove`.
*Method `LRU_Cache.__delitem__(self, key)`*:
Delete the specified `key`, calling the on_remove callback.
*Method `LRU_Cache.__setitem__(self, key, value)`*:
Store the item in the cache. Prune if necessary.
*Method `LRU_Cache.flush(self)`*:
Clear the cache.
*Method `LRU_Cache.get(self, key, default=None)`*:
Mapping method: get value for `key` or `default`.
*Method `LRU_Cache.items(self)`*:
Items from the cache.
*Method `LRU_Cache.keys(self)`*:
Keys from the cache.
## Function `lru_cache(max_size=None, cache=None, on_add=None, on_remove=None)`
Enhanced workalike of @functools.lru_cache.
# Release Log
*Release 20240422.1*:
ConvCache docstring update.
*Release 20240422*:
New ConvCache and convof: a cache for conversions of file contents such as thumbnails or transcoded media.
*Release 20240412*:
* New CachingMapping, a caching front end for another mapping.
* LRU_Cache: add keys() and items().
*Release 20181228*:
Initial PyPI release.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs.cache",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/b4/87/502c0ac0a9f60484a65e74492d8920195d63234cecd66832c0d531ec62dc/cs.cache-20240422.1.tar.gz",
"platform": null,
"description": "A few caching data structures and other lossy things with capped sizes.\n\n*Latest release 20240422.1*:\nConvCache docstring update.\n\n## Class `CachingMapping(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)`\n\nA caching front end for another mapping.\nThis is intended as a generic superclass for a proxy to a\nslower mapping such as a database or remote key value store.\n\nNote that this subclasses `MultiOpenMixin` to start/stop the worker `Thread`.\nUsers must enclose use of a `CachingMapping` in a `with` statement.\nIf subclasses also subclass `MultiOpenMixin` their `startup_shutdown`\nmethod needs to also call our `startup_shutdown` method.\n\nExample:\n\n class Store:\n \"\"\" A key value store with a slower backend.\n \"\"\"\n def __init__(self, mapping:Mapping):\n self.mapping = CachingMapping(mapping)\n\n .....\n S = Store(slow_mapping)\n with S:\n ... work with S ...\n\n*Method `CachingMapping.__init__(self, mapping: Mapping, *, max_size=1024, queue_length=1024, delitem_bg: Optional[Callable[[Any], cs.result.Result]] = None, setitem_bg: Optional[Callable[[Any, Any], cs.result.Result]] = None, missing_fallthrough: bool = False)`*:\nInitialise the cache.\n\nParameters:\n* `mapping`: the backing store, a mapping\n* `max_size`: optional maximum size for the cache, default 1024\n* `queue_length`: option size for the queue to the worker, default 1024\n* `delitem_bg`: optional callable to queue a delete of a\n key in the backing store; if unset then deleted are\n serialised in the worker thread\n* `setitem_bg`: optional callable to queue setting the value\n for a key in the backing store; if unset then deleted are\n serialised in the worker thread\n* `missing_fallthrough`: is true (default `False`) always\n fall back to the backing mapping if a key is not in the cache\n\n*Method `CachingMapping.flush(self)`*:\nWait for outstanding requests in the queue to complete.\nReturn the UNIX time of completion.\n\n*Method `CachingMapping.items(self)`*:\nGenerator yielding `(k,v)` pairs.\n\n*Method `CachingMapping.keys(self)`*:\nGenerator yielding the keys.\n\n## Class `ConvCache(cs.fs.HasFSPath)`\n\nA cache for conversions of file contents such as thumbnails\nor transcoded media, etc. This keeps cached results in a file\ntree based on a content key, whose default function is\n`cs.hashutils.file_checksum('sha256')`.\n\n*Method `ConvCache.__init__(self, fspath: Optional[str] = None, content_key_func=None)`*:\nInitialise a `ConvCache`.\n\nParameters:\n* `fspath`: optional base path of the cache, default from\n `ConvCache.DEFAULT_CACHE_BASEPATH`;\n if this does not exist it will be created using `os.mkdir`\n* `content_key_func`: optional function to compute a key\n from the contents of a file, default `cs.hashindex.file_checksum`\n (the sha256 hash of the contents)\n\n*Method `ConvCache.content_key(self, srcpath)`*:\nReturn a content key for the filesystem path `srcpath`.\n\n*Method `ConvCache.content_subpath(self, srcpath)`*:\nReturn the content key based subpath component.\n\nThis default assumes the content key is a hash code and\nbreaks it hex representation into a 3 level hierarchy\nsuch as `'d6/d9/c510785c468c9aa4b7bda343fb79'`.\n\n*Method `ConvCache.convof(self, srcpath, conv_subpath, conv_func, ext=None)`*:\nReturn the filesystem path of the cached conversion of `srcpath` via `conv_func`.\n\nParameters:\n* `srcpath`: the source filesystem path\n* `conv_subpath`: a name for the conversion which encompasses\n the salient aspaects such as `'png/64/64'` for a 64x64 pixel\n thumbnail in PNG format\n* `conv_func`: a callable of the form `conv_func(srcpath,dstpath)`\n to convert the contents of `srcpath` and write the result\n to the filesystem path `dstpath`\n* `ext`: an optional filename extension, default from the\n first component of `conv_subpath`\n\n## Function `convof(srcpath, conv_subpath, conv_func, ext=None)`\n\n`ConvCache.convof` using the default cache.\n\n## Class `LRU_Cache`\n\nA simple least recently used cache.\n\nUnlike `functools.lru_cache`\nthis provides `on_add` and `on_remove` callbacks.\n\n*Method `LRU_Cache.__init__(self, max_size, *, on_add=None, on_remove=None)`*:\nInitialise the LRU_Cache with maximum size `max`,\nadditon callback `on_add` and removal callback `on_remove`.\n\n*Method `LRU_Cache.__delitem__(self, key)`*:\nDelete the specified `key`, calling the on_remove callback.\n\n*Method `LRU_Cache.__setitem__(self, key, value)`*:\nStore the item in the cache. Prune if necessary.\n\n*Method `LRU_Cache.flush(self)`*:\nClear the cache.\n\n*Method `LRU_Cache.get(self, key, default=None)`*:\nMapping method: get value for `key` or `default`.\n\n*Method `LRU_Cache.items(self)`*:\nItems from the cache.\n\n*Method `LRU_Cache.keys(self)`*:\nKeys from the cache.\n\n## Function `lru_cache(max_size=None, cache=None, on_add=None, on_remove=None)`\n\nEnhanced workalike of @functools.lru_cache.\n\n# Release Log\n\n\n\n*Release 20240422.1*:\nConvCache docstring update.\n\n*Release 20240422*:\nNew ConvCache and convof: a cache for conversions of file contents such as thumbnails or transcoded media.\n\n*Release 20240412*:\n* New CachingMapping, a caching front end for another mapping.\n* LRU_Cache: add keys() and items().\n\n*Release 20181228*:\nInitial PyPI release.\n\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "A few caching data structures and other lossy things with capped sizes.",
"version": "20240422.1",
"project_urls": {
"URL": "https://bitbucket.org/cameron_simpson/css/commits/all"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d808941840e71a588231240607219e9f15dfc7902e149a5c206ebecab0e99279",
"md5": "40513f597a3a95e95f253ccb76aa5935",
"sha256": "e9c40a16f5f0ff5fb58d74c6ad78249344e689940b114ac9c1a6968e8dae03bb"
},
"downloads": -1,
"filename": "cs.cache-20240422.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "40513f597a3a95e95f253ccb76aa5935",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 8480,
"upload_time": "2024-04-22T05:33:11",
"upload_time_iso_8601": "2024-04-22T05:33:11.533325Z",
"url": "https://files.pythonhosted.org/packages/d8/08/941840e71a588231240607219e9f15dfc7902e149a5c206ebecab0e99279/cs.cache-20240422.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b487502c0ac0a9f60484a65e74492d8920195d63234cecd66832c0d531ec62dc",
"md5": "989bb68f6a9d49a9bcb07f4a58455ebb",
"sha256": "b5c1a22939b98d268f5da6d46498f2e7c0c87d4b48b3047b6753d8c43050212a"
},
"downloads": -1,
"filename": "cs.cache-20240422.1.tar.gz",
"has_sig": false,
"md5_digest": "989bb68f6a9d49a9bcb07f4a58455ebb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8005,
"upload_time": "2024-04-22T05:33:13",
"upload_time_iso_8601": "2024-04-22T05:33:13.716793Z",
"url": "https://files.pythonhosted.org/packages/b4/87/502c0ac0a9f60484a65e74492d8920195d63234cecd66832c0d531ec62dc/cs.cache-20240422.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-22 05:33:13",
"github": false,
"gitlab": false,
"bitbucket": true,
"codeberg": false,
"bitbucket_user": "cameron_simpson",
"bitbucket_project": "css",
"lcname": "cs.cache"
}
JSON