A few caching data structures and other lossy things with capped sizes.
*Latest release 20241122*:
* convof, ConvCache.convof: new options force=False parameter to bypass existing cache entries.
* ConvCache.convof: check the conv_subpath using cs.fs.validate_rpath.
## <a name="CachingMapping"></a>Class `CachingMapping(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)`
A caching front end for another mapping.
This is intended as a generic superclass for a proxy to a
slower mapping such as a database or remote key value store.
Note that this subclasses `MultiOpenMixin` to start/stop the worker `Thread`.
Users must enclose use of a `CachingMapping` in a `with` statement.
If subclasses also subclass `MultiOpenMixin` their `startup_shutdown`
method needs to also call our `startup_shutdown` method.
Example:
class Store:
""" A key value store with a slower backend.
"""
def __init__(self, mapping:Mapping):
self.mapping = CachingMapping(mapping)
.....
S = Store(slow_mapping)
with S:
... work with S ...
*`CachingMapping.__init__(self, mapping: Mapping, *, max_size=1024, queue_length=1024, delitem_bg: Optional[Callable[[Any], cs.result.Result]] = None, setitem_bg: Optional[Callable[[Any, Any], cs.result.Result]] = None, missing_fallthrough: bool = False)`*:
Initialise the cache.
Parameters:
* `mapping`: the backing store, a mapping
* `max_size`: optional maximum size for the cache, default 1024
* `queue_length`: option size for the queue to the worker, default 1024
* `delitem_bg`: optional callable to queue a delete of a
key in the backing store; if unset then deleted are
serialised in the worker thread
* `setitem_bg`: optional callable to queue setting the value
for a key in the backing store; if unset then deleted are
serialised in the worker thread
* `missing_fallthrough`: is true (default `False`) always
fall back to the backing mapping if a key is not in the cache
*`CachingMapping.flush(self)`*:
Wait for outstanding requests in the queue to complete.
Return the UNIX time of completion.
*`CachingMapping.items(self)`*:
Generator yielding `(k,v)` pairs.
*`CachingMapping.keys(self)`*:
Generator yielding the keys.
## <a name="ConvCache"></a>Class `ConvCache(cs.fs.HasFSPath)`
A cache for conversions of file contents such as thumbnails
or transcoded media, etc. This keeps cached results in a file
tree based on a content key, whose default function is
`cs.hashutils.file_checksum('blake3')`.
*`ConvCache.__init__(self, fspath: Optional[str] = None, content_key_func=None)`*:
Initialise a `ConvCache`.
Parameters:
* `fspath`: optional base path of the cache, default from
`ConvCache.DEFAULT_CACHE_BASEPATH`;
if this does not exist it will be created using `os.mkdir`
* `content_key_func`: optional function to compute a key
from the contents of a file, default `cs.hashindex.file_checksum`
(the blake3 hash of the contents)
*`ConvCache.content_key(self, srcpath)`*:
Return a content key for the filesystem path `srcpath`.
*`ConvCache.content_subpath(self, srcpath) -> str`*:
Return the content key based subpath component.
This default assumes the content key is a hash code and
breaks it hex representation into a 3 level hierarchy
such as `'d6/d9/c510785c468c9aa4b7bda343fb79'`.
*`ConvCache.convof(self, srcpath, conv_subpath, conv_func, *, ext=None, force=False) -> str`*:
Return the filesystem path of the cached conversion of
`srcpath` via `conv_func`.
Parameters:
* `srcpath`: the source filesystem path
* `conv_subpath`: a name for the conversion which encompasses
the salient aspaects such as `'png/64/64'` for a 64x64 pixel
thumbnail in PNG format
* `conv_func`: a callable of the form `conv_func(srcpath,dstpath)`
to convert the contents of `srcpath` and write the result
to the filesystem path `dstpath`
* `ext`: an optional filename extension, default from the
first component of `conv_subpath`
* `force`: option flag to require conversion even if the
cache has an entry
## <a name="convof"></a>`convof(srcpath, conv_subpath, conv_func, *, ext=None, force=False)`
`ConvCache.convof` using the default cache.
## <a name="LRU_Cache"></a>Class `LRU_Cache`
A simple least recently used cache.
Unlike `functools.lru_cache`
this provides `on_add` and `on_remove` callbacks.
*`LRU_Cache.__init__(self, max_size, *, on_add=None, on_remove=None)`*:
Initialise the LRU_Cache with maximum size `max`,
additon callback `on_add` and removal callback `on_remove`.
*`LRU_Cache.__delitem__(self, key)`*:
Delete the specified `key`, calling the on_remove callback.
*`LRU_Cache.__setitem__(self, key, value)`*:
Store the item in the cache. Prune if necessary.
*`LRU_Cache.flush(self)`*:
Clear the cache.
*`LRU_Cache.get(self, key, default=None)`*:
Mapping method: get value for `key` or `default`.
*`LRU_Cache.items(self)`*:
Items from the cache.
*`LRU_Cache.keys(self)`*:
Keys from the cache.
## <a name="lru_cache"></a>`lru_cache(max_size=None, cache=None, on_add=None, on_remove=None)`
Enhanced workalike of @functools.lru_cache.
# Release Log
*Release 20241122*:
* convof, ConvCache.convof: new options force=False parameter to bypass existing cache entries.
* ConvCache.convof: check the conv_subpath using cs.fs.validate_rpath.
*Release 20240422.1*:
ConvCache docstring update.
*Release 20240422*:
New ConvCache and convof: a cache for conversions of file contents such as thumbnails or transcoded media.
*Release 20240412*:
* New CachingMapping, a caching front end for another mapping.
* LRU_Cache: add keys() and items().
*Release 20181228*:
Initial PyPI release.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs-cache",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/4f/87/6af31ec7439cd2ff0c5f4959e89854868bd6b134857b0640776dbebed896/cs_cache-20241122.tar.gz",
"platform": null,
"description": "A few caching data structures and other lossy things with capped sizes.\n\n*Latest release 20241122*:\n* convof, ConvCache.convof: new options force=False parameter to bypass existing cache entries.\n* ConvCache.convof: check the conv_subpath using cs.fs.validate_rpath.\n\n## <a name=\"CachingMapping\"></a>Class `CachingMapping(cs.resources.MultiOpenMixin, collections.abc.MutableMapping)`\n\nA caching front end for another mapping.\nThis is intended as a generic superclass for a proxy to a\nslower mapping such as a database or remote key value store.\n\nNote that this subclasses `MultiOpenMixin` to start/stop the worker `Thread`.\nUsers must enclose use of a `CachingMapping` in a `with` statement.\nIf subclasses also subclass `MultiOpenMixin` their `startup_shutdown`\nmethod needs to also call our `startup_shutdown` method.\n\nExample:\n\n class Store:\n \"\"\" A key value store with a slower backend.\n \"\"\"\n def __init__(self, mapping:Mapping):\n self.mapping = CachingMapping(mapping)\n\n .....\n S = Store(slow_mapping)\n with S:\n ... work with S ...\n\n*`CachingMapping.__init__(self, mapping: Mapping, *, max_size=1024, queue_length=1024, delitem_bg: Optional[Callable[[Any], cs.result.Result]] = None, setitem_bg: Optional[Callable[[Any, Any], cs.result.Result]] = None, missing_fallthrough: bool = False)`*:\nInitialise the cache.\n\nParameters:\n* `mapping`: the backing store, a mapping\n* `max_size`: optional maximum size for the cache, default 1024\n* `queue_length`: option size for the queue to the worker, default 1024\n* `delitem_bg`: optional callable to queue a delete of a\n key in the backing store; if unset then deleted are\n serialised in the worker thread\n* `setitem_bg`: optional callable to queue setting the value\n for a key in the backing store; if unset then deleted are\n serialised in the worker thread\n* `missing_fallthrough`: is true (default `False`) always\n fall back to the backing mapping if a key is not in the cache\n\n*`CachingMapping.flush(self)`*:\nWait for outstanding requests in the queue to complete.\nReturn the UNIX time of completion.\n\n*`CachingMapping.items(self)`*:\nGenerator yielding `(k,v)` pairs.\n\n*`CachingMapping.keys(self)`*:\nGenerator yielding the keys.\n\n## <a name=\"ConvCache\"></a>Class `ConvCache(cs.fs.HasFSPath)`\n\nA cache for conversions of file contents such as thumbnails\nor transcoded media, etc. This keeps cached results in a file\ntree based on a content key, whose default function is\n`cs.hashutils.file_checksum('blake3')`.\n\n*`ConvCache.__init__(self, fspath: Optional[str] = None, content_key_func=None)`*:\nInitialise a `ConvCache`.\n\nParameters:\n* `fspath`: optional base path of the cache, default from\n `ConvCache.DEFAULT_CACHE_BASEPATH`;\n if this does not exist it will be created using `os.mkdir`\n* `content_key_func`: optional function to compute a key\n from the contents of a file, default `cs.hashindex.file_checksum`\n (the blake3 hash of the contents)\n\n*`ConvCache.content_key(self, srcpath)`*:\nReturn a content key for the filesystem path `srcpath`.\n\n*`ConvCache.content_subpath(self, srcpath) -> str`*:\nReturn the content key based subpath component.\n\nThis default assumes the content key is a hash code and\nbreaks it hex representation into a 3 level hierarchy\nsuch as `'d6/d9/c510785c468c9aa4b7bda343fb79'`.\n\n*`ConvCache.convof(self, srcpath, conv_subpath, conv_func, *, ext=None, force=False) -> str`*:\nReturn the filesystem path of the cached conversion of\n`srcpath` via `conv_func`.\n\nParameters:\n* `srcpath`: the source filesystem path\n* `conv_subpath`: a name for the conversion which encompasses\n the salient aspaects such as `'png/64/64'` for a 64x64 pixel\n thumbnail in PNG format\n* `conv_func`: a callable of the form `conv_func(srcpath,dstpath)`\n to convert the contents of `srcpath` and write the result\n to the filesystem path `dstpath`\n* `ext`: an optional filename extension, default from the\n first component of `conv_subpath`\n* `force`: option flag to require conversion even if the\n cache has an entry\n\n## <a name=\"convof\"></a>`convof(srcpath, conv_subpath, conv_func, *, ext=None, force=False)`\n\n`ConvCache.convof` using the default cache.\n\n## <a name=\"LRU_Cache\"></a>Class `LRU_Cache`\n\nA simple least recently used cache.\n\nUnlike `functools.lru_cache`\nthis provides `on_add` and `on_remove` callbacks.\n\n*`LRU_Cache.__init__(self, max_size, *, on_add=None, on_remove=None)`*:\nInitialise the LRU_Cache with maximum size `max`,\nadditon callback `on_add` and removal callback `on_remove`.\n\n*`LRU_Cache.__delitem__(self, key)`*:\nDelete the specified `key`, calling the on_remove callback.\n\n*`LRU_Cache.__setitem__(self, key, value)`*:\nStore the item in the cache. Prune if necessary.\n\n*`LRU_Cache.flush(self)`*:\nClear the cache.\n\n*`LRU_Cache.get(self, key, default=None)`*:\nMapping method: get value for `key` or `default`.\n\n*`LRU_Cache.items(self)`*:\nItems from the cache.\n\n*`LRU_Cache.keys(self)`*:\nKeys from the cache.\n\n## <a name=\"lru_cache\"></a>`lru_cache(max_size=None, cache=None, on_add=None, on_remove=None)`\n\nEnhanced workalike of @functools.lru_cache.\n\n# Release Log\n\n\n\n*Release 20241122*:\n* convof, ConvCache.convof: new options force=False parameter to bypass existing cache entries.\n* ConvCache.convof: check the conv_subpath using cs.fs.validate_rpath.\n\n*Release 20240422.1*:\nConvCache docstring update.\n\n*Release 20240422*:\nNew ConvCache and convof: a cache for conversions of file contents such as thumbnails or transcoded media.\n\n*Release 20240412*:\n* New CachingMapping, a caching front end for another mapping.\n* LRU_Cache: add keys() and items().\n\n*Release 20181228*:\nInitial PyPI release.\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "A few caching data structures and other lossy things with capped sizes.",
"version": "20241122",
"project_urls": {
"MonoRepo Commits": "https://bitbucket.org/cameron_simpson/css/commits/branch/main",
"Monorepo Git Mirror": "https://github.com/cameron-simpson/css",
"Monorepo Hg/Mercurial Mirror": "https://hg.sr.ht/~cameron-simpson/css",
"Source": "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/cache.py"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8483614a1a96ff19aec62d9a485915644233205a90c0490ad17534b356c293ab",
"md5": "5385ab697dbc6a4b76db325a4342681b",
"sha256": "e7027385b4f8868800f6afe9136df2a3f801dd2a8d1c8aade3b2a2de9dc80e64"
},
"downloads": -1,
"filename": "cs_cache-20241122-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5385ab697dbc6a4b76db325a4342681b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 8844,
"upload_time": "2024-11-22T09:16:50",
"upload_time_iso_8601": "2024-11-22T09:16:50.490818Z",
"url": "https://files.pythonhosted.org/packages/84/83/614a1a96ff19aec62d9a485915644233205a90c0490ad17534b356c293ab/cs_cache-20241122-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4f876af31ec7439cd2ff0c5f4959e89854868bd6b134857b0640776dbebed896",
"md5": "fef3beddc04741b4b49622b53e5194f9",
"sha256": "57fdd6d2e2e249797612cc7765f86142b37a24dcbfa150a21937cd352c7ac8d6"
},
"downloads": -1,
"filename": "cs_cache-20241122.tar.gz",
"has_sig": false,
"md5_digest": "fef3beddc04741b4b49622b53e5194f9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9215,
"upload_time": "2024-11-22T09:16:52",
"upload_time_iso_8601": "2024-11-22T09:16:52.391523Z",
"url": "https://files.pythonhosted.org/packages/4f/87/6af31ec7439cd2ff0c5f4959e89854868bd6b134857b0640776dbebed896/cs_cache-20241122.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 09:16:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cameron-simpson",
"github_project": "css",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cs-cache"
}
JSON
