|Build Status|
CacheMan
========
A Python interface for managing dependent caches.
‘Ba-Bop-Ba-Dop-Bop’
Description
-----------
This module acts as a dependency manager for caches and is ideal for
instances where a program has many repeated computations that could be
safely persisted. This usually entails a DB layer to house key value
pairs. However, such a layer is sometimes overkill and managing a DB
along with a project can be more effort than it’s worth. That’s where
CacheMan comes in and provides an interface through which you can define
savers, loaders, builders, and dependencies with disk-based defaults.
By default all caches will auto save when 10k changes occur over 60
seconds, 10 changes occur over 300 seconds (but after 60 seconds), or 1
change occurs within 900 seconds (after 300 seconds). This behavior can
be changed by instantiating an AutoSyncCache from the autosync
submodule.
Dependencies
------------
psutil – for asynchronous cache saving
Features
--------
- Drop in replacement for local memory dictionaries
- Default persistent pickle caches
- Non-persistent caching
- Cache load/save/delete hooks w/ defaults
- Cache validation hooks
- Cache builder hooks
- Dependent invalidation
- Auto-Syncing caches
How to use
----------
Below are some simple examples for how to use the repository.
Setting up a simple persistent cache
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
from cacheman import cacher
manager = cacher.get_cache_manager() # Optional manager name argument can be used here
cache = manager.register_cache('my_simple_cache') # You now have a cache!
print cache.get('my_key') # `None` first run, 'my_value' if this code was executed earlier
cache['my_key'] = 'my_value'
cache.save() # Changes are now persisted to disk
manager.save_cache_contents('my_simple_cache') # Alternative way to save a cache
Non-persistent caches
~~~~~~~~~~~~~~~~~~~~~
::
from cacheman import cacher
manager = cacher.get_cache_manager()
cache = manager.register_custom_cache('my_simple_cache', persistent=False) # You cache won't save to disk
cache.save() # This is a no-op
Registering hooks
~~~~~~~~~~~~~~~~~
::
from cacheman import cacher
from cacheman import cachewrap
def my_saver(cache_name, contents):
print("Save requested on {} cache content: {}".format(cache_name, contents))
def my_loader(cache_name):
return { 'load': 'faked' }
manager = cacher.get_cache_manager()
cache = cachewrap.PersistentCache('my_cache', saver=my_saver, loader=my_loader)
# Can also use manager to set savers/loaders
#manager.retrieve_cache('my_cache')
#manager.register_saver('my_cache', my_saver)
#manager.register_loader('my_cache', my_loader)
cache.save() # Will print 'Save ... : { 'load': 'faked' }'
cache['new'] = 'real' # Add something to the cache
cache.save() # Will print 'Save ... : { 'load': 'faked', 'new': 'real' }'
Dependent caches
~~~~~~~~~~~~~~~~
::
from cacheman import cacher
manager = cacher.get_cache_manager()
edge_cache = manager.retrieve_cache('edge_cache')
root_cache = manager.register_cache('root_cache')
manager.register_dependent_cache('root_cache', 'edge_cache')
def set_processed_value():
# Computes and caches 'processed' from root's 'raw' value
processed = edge_cache.get('processed')
if processed is None:
processed = (root_cache.get('raw') or 0) * 5
edge_cache['processed'] = processed
return processed
# A common problem with caching computed or dependent values:
print set_processed_value() # 0 without raw value
root_cache['raw'] = 1
print set_processed_value() # still 0 because it's cache in edge
# Now we use cache invalidation to tell downstream caches they're no longer valid
root_cache.invalidate() # Invalidates dependent caches
print edge_cache # Prints {} even though we only invalidated the root_cache
root_cache['raw'] = 1
print set_processed_value() # Now 5 because the edge was cleared before the request
print edge_cache # Can see {'processed': 5} propogated
Setting cache directory
~~~~~~~~~~~~~~~~~~~~~~~
::
from cacheman import cacher
# Default cache directory is '/tmp/general_cacher' or 'user\appadata\local\temp\general_cache'
# All pickle caches now save to namespaced directories within the base_cache_directory directory
manager = cacher.get_cache_manager(base_cache_directory='secret/cache/location')
cache = manager.register_cache('my_cache')
cache['new'] = 'real' # Add something to the cache
cache.save('my_cache') # Will save contents to 'secret/cache/location/general_cache/my_cache.pkl'
Navigating the Repo
-------------------
.. _cacheman-1:
cacheman
~~~~~~~~
Package wrapper for the repo.
tests
~~~~~
All unit tests for the repo.
Language Preferences
--------------------
- Google Style Guide
- Object Oriented (with a few exceptions)
TODO
----
- Better argument checks
- Changelog
Author
------
Author(s): Matthew Seal
.. |Build Status| image:: https://travis-ci.org/MSeal/py_cache_manager.svg?branch=master
:target: https://travis-ci.org/MSeal/py_cache_manager
Raw data
{
"_id": null,
"home_page": "https://github.com/MSeal/py_cache_manager",
"name": "CacheMan",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "tables,data,analysis,extraction",
"author": "Matthew Seal",
"author_email": "mseal007@gmail.com",
"download_url": "https://github.com/MSeal/py_cache_manager/tarball/v2.2.0",
"platform": null,
"description": "|Build Status|\n\nCacheMan\n========\n\nA Python interface for managing dependent caches.\n\n\u2018Ba-Bop-Ba-Dop-Bop\u2019\n\nDescription\n-----------\n\nThis module acts as a dependency manager for caches and is ideal for\ninstances where a program has many repeated computations that could be\nsafely persisted. This usually entails a DB layer to house key value\npairs. However, such a layer is sometimes overkill and managing a DB\nalong with a project can be more effort than it\u2019s worth. That\u2019s where\nCacheMan comes in and provides an interface through which you can define\nsavers, loaders, builders, and dependencies with disk-based defaults.\n\nBy default all caches will auto save when 10k changes occur over 60\nseconds, 10 changes occur over 300 seconds (but after 60 seconds), or 1\nchange occurs within 900 seconds (after 300 seconds). This behavior can\nbe changed by instantiating an AutoSyncCache from the autosync\nsubmodule.\n\nDependencies\n------------\n\npsutil \u2013 for asynchronous cache saving\n\nFeatures\n--------\n\n- Drop in replacement for local memory dictionaries\n- Default persistent pickle caches\n- Non-persistent caching\n- Cache load/save/delete hooks w/ defaults\n- Cache validation hooks\n- Cache builder hooks\n- Dependent invalidation\n- Auto-Syncing caches\n\nHow to use\n----------\n\nBelow are some simple examples for how to use the repository.\n\nSetting up a simple persistent cache\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n::\n\n from cacheman import cacher\n\n manager = cacher.get_cache_manager() # Optional manager name argument can be used here\n cache = manager.register_cache('my_simple_cache') # You now have a cache!\n print cache.get('my_key') # `None` first run, 'my_value' if this code was executed earlier\n cache['my_key'] = 'my_value'\n cache.save() # Changes are now persisted to disk\n manager.save_cache_contents('my_simple_cache') # Alternative way to save a cache\n\nNon-persistent caches\n~~~~~~~~~~~~~~~~~~~~~\n\n::\n\n from cacheman import cacher\n\n manager = cacher.get_cache_manager()\n cache = manager.register_custom_cache('my_simple_cache', persistent=False) # You cache won't save to disk\n cache.save() # This is a no-op\n\nRegistering hooks\n~~~~~~~~~~~~~~~~~\n\n::\n\n from cacheman import cacher\n from cacheman import cachewrap\n\n def my_saver(cache_name, contents):\n print(\"Save requested on {} cache content: {}\".format(cache_name, contents))\n\n def my_loader(cache_name):\n return { 'load': 'faked' }\n\n manager = cacher.get_cache_manager()\n\n cache = cachewrap.PersistentCache('my_cache', saver=my_saver, loader=my_loader)\n # Can also use manager to set savers/loaders\n #manager.retrieve_cache('my_cache')\n #manager.register_saver('my_cache', my_saver)\n #manager.register_loader('my_cache', my_loader)\n\n cache.save() # Will print 'Save ... : { 'load': 'faked' }'\n cache['new'] = 'real' # Add something to the cache\n cache.save() # Will print 'Save ... : { 'load': 'faked', 'new': 'real' }'\n\nDependent caches\n~~~~~~~~~~~~~~~~\n\n::\n\n from cacheman import cacher\n\n manager = cacher.get_cache_manager()\n edge_cache = manager.retrieve_cache('edge_cache')\n root_cache = manager.register_cache('root_cache')\n manager.register_dependent_cache('root_cache', 'edge_cache')\n\n def set_processed_value():\n # Computes and caches 'processed' from root's 'raw' value\n processed = edge_cache.get('processed')\n if processed is None:\n processed = (root_cache.get('raw') or 0) * 5\n edge_cache['processed'] = processed\n return processed\n\n # A common problem with caching computed or dependent values:\n print set_processed_value() # 0 without raw value\n root_cache['raw'] = 1\n print set_processed_value() # still 0 because it's cache in edge\n\n # Now we use cache invalidation to tell downstream caches they're no longer valid\n root_cache.invalidate() # Invalidates dependent caches\n print edge_cache # Prints {} even though we only invalidated the root_cache\n root_cache['raw'] = 1\n print set_processed_value() # Now 5 because the edge was cleared before the request\n print edge_cache # Can see {'processed': 5} propogated\n\nSetting cache directory\n~~~~~~~~~~~~~~~~~~~~~~~\n\n::\n\n from cacheman import cacher\n\n # Default cache directory is '/tmp/general_cacher' or 'user\\appadata\\local\\temp\\general_cache'\n # All pickle caches now save to namespaced directories within the base_cache_directory directory\n manager = cacher.get_cache_manager(base_cache_directory='secret/cache/location')\n\n cache = manager.register_cache('my_cache')\n cache['new'] = 'real' # Add something to the cache\n cache.save('my_cache') # Will save contents to 'secret/cache/location/general_cache/my_cache.pkl'\n\nNavigating the Repo\n-------------------\n\n.. _cacheman-1:\n\ncacheman\n~~~~~~~~\n\nPackage wrapper for the repo.\n\ntests\n~~~~~\n\nAll unit tests for the repo.\n\nLanguage Preferences\n--------------------\n\n- Google Style Guide\n- Object Oriented (with a few exceptions)\n\nTODO\n----\n\n- Better argument checks\n- Changelog\n\nAuthor\n------\n\nAuthor(s): Matthew Seal\n\n.. |Build Status| image:: https://travis-ci.org/MSeal/py_cache_manager.svg?branch=master\n :target: https://travis-ci.org/MSeal/py_cache_manager\n",
"bugtrack_url": null,
"license": "New BSD",
"summary": "A dependent cache manager",
"version": "2.2.0",
"split_keywords": [
"tables",
"data",
"analysis",
"extraction"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "0f467ea4610d3ee048dedba42224ffb4",
"sha256": "d41529ac09dfe34eec388def50fdba64bec83ce1c2325ed9eddc4ffc6e664add"
},
"downloads": -1,
"filename": "CacheMan-2.2.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "0f467ea4610d3ee048dedba42224ffb4",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 13152,
"upload_time": "2022-12-14T09:29:51",
"upload_time_iso_8601": "2022-12-14T09:29:51.496710Z",
"url": "https://files.pythonhosted.org/packages/bb/b1/3a64d6731e72e0d82964214d26ec312f76c016ef74cbb43ea289d609b7b0/CacheMan-2.2.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-14 09:29:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "MSeal",
"github_project": "py_cache_manager",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "cacheman"
}
JSON
