-------------------------------------------------------
klepto: persistent caching to memory, disk, or database
-------------------------------------------------------
About Klepto
============
``klepto`` extends Python's ``lru_cache`` to utilize different keymaps and
alternate caching algorithms, such as ``lfu_cache`` and ``mru_cache``.
While caching is meant for fast access to saved results, ``klepto`` also
has archiving capabilities, for longer-term storage. ``klepto`` uses a
simple dictionary-sytle interface for all caches and archives, and all
caches can be applied to any Python function as a decorator. Keymaps
are algorithms for converting a function's input signature to a unique
dictionary, where the function's results are the dictionary value.
Thus for ``y = f(x)``, ``y`` will be stored in ``cache[x]`` (e.g. ``{x:y}``).
``klepto`` provides both standard and *"safe"* caching, where *"safe"* caches
are slower but can recover from hashing errors. ``klepto`` is intended
to be used for distributed and parallel computing, where several of
the keymaps serialize the stored objects. Caches and archives are
intended to be read/write accessible from different threads and
processes. ``klepto`` enables a user to decorate a function, save the
results to a file or database archive, close the interpreter,
start a new session, and reload the function and it's cache.
``klepto`` is part of ``pathos``, a Python framework for heterogeneous computing.
``klepto`` is in active development, so any user feedback, bug reports, comments,
or suggestions are highly appreciated. A list of issues is located at https://github.com/uqfoundation/klepto/issues, with a legacy list maintained at https://uqfoundation.github.io/project/pathos/query.
Major Features
==============
``klepto`` has standard and *"safe"* variants of the following:
- ``lfu_cache`` - the least-frequently-used caching algorithm
- ``lru_cache`` - the least-recently-used caching algorithm
- ``mru_cache`` - the most-recently-used caching algorithm
- ``rr_cache`` - the random-replacement caching algorithm
- ``no_cache`` - a dummy caching interface to archiving
- ``inf_cache`` - an infinitely-growing cache
``klepto`` has the following archive types:
- ``file_archive`` - a dictionary-style interface to a file
- ``dir_archive`` - a dictionary-style interface to a folder of files
- ``sqltable_archive`` - a dictionary-style interface to a sql database table
- ``sql_archive`` - a dictionary-style interface to a sql database
- ``hdfdir_archive`` - a dictionary-style interface to a folder of hdf5 files
- ``hdf_archive`` - a dictionary-style interface to a hdf5 file
- ``dict_archive`` - a dictionary with an archive interface
- ``null_archive`` - a dictionary-style interface to a dummy archive
``klepto`` provides the following keymaps:
- ``keymap`` - keys are raw Python objects
- ``hashmap`` - keys are a hash for the Python object
- ``stringmap`` - keys are the Python object cast as a string
- ``picklemap`` - keys are the serialized Python object
``klepto`` also includes a few useful decorators providing:
- simple, shallow, or deep rounding of function arguments
- cryptographic key generation, with masking of selected arguments
Current Release
===============
The latest released version of ``klepto`` is available from:
https://pypi.org/project/klepto
``klepto`` is distributed under a 3-clause BSD license.
Development Version
===================
You can get the latest development version with all the shiny new features at:
https://github.com/uqfoundation
If you have a new contribution, please submit a pull request.
Installation
============
``klepto`` can be installed with ``pip``::
$ pip install klepto
To include optional archive backends, such as HDF5 and SQL, in the install::
$ pip install klepto[archives]
To include optional serializers, such as ``jsonpickle``, in the install::
$ pip install klepto[crypto]
Requirements
============
``klepto`` requires:
- ``python`` (or ``pypy``), **>=3.8**
- ``setuptools``, **>=42**
- ``dill``, **>=0.3.8**
- ``pox``, **>=0.3.4**
Optional requirements:
- ``h5py``, **>=2.8.0**
- ``pandas``, **>=0.17.0**
- ``sqlalchemy``, **>=1.4.0**
- ``jsonpickle``, **>=0.9.6**
- ``cloudpickle``, **>=0.5.2**
More Information
================
Probably the best way to get started is to look at the documentation at
http://klepto.rtfd.io. Also see ``klepto.tests`` for a set of scripts that
test the caching and archiving functionalities in ``klepto``.
You can run the test suite with ``python -m klepto.tests``. The
source code is also generally well documented, so further questions may
be resolved by inspecting the code itself. Please feel free to submit
a ticket on github, or ask a question on stackoverflow (**@Mike McKerns**).
If you would like to share how you use ``klepto`` in your work, please send
an email (to **mmckerns at uqfoundation dot org**).
Citation
========
If you use ``klepto`` to do research that leads to publication, we ask that you
acknowledge use of ``klepto`` by citing the following in your publication::
Michael McKerns and Michael Aivazis,
"pathos: a framework for heterogeneous computing", 2010- ;
https://uqfoundation.github.io/project/pathos
Please see https://uqfoundation.github.io/project/pathos or
http://arxiv.org/pdf/1202.1056 for further information.
Raw data
{
"_id": null,
"home_page": "https://github.com/uqfoundation/klepto",
"name": "klepto",
"maintainer": "Mike McKerns",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "mmckerns@uqfoundation.org",
"keywords": "",
"author": "Mike McKerns",
"author_email": "mmckerns@uqfoundation.org",
"download_url": "https://files.pythonhosted.org/packages/97/81/2faf5ccf7c66f7ea2a4dd45da0ede3fdaac6ef861e15c79a054c8966b6e8/klepto-0.2.5.tar.gz",
"platform": "Linux",
"description": "-------------------------------------------------------\nklepto: persistent caching to memory, disk, or database\n-------------------------------------------------------\n\nAbout Klepto\n============\n\n``klepto`` extends Python's ``lru_cache`` to utilize different keymaps and\nalternate caching algorithms, such as ``lfu_cache`` and ``mru_cache``.\nWhile caching is meant for fast access to saved results, ``klepto`` also\nhas archiving capabilities, for longer-term storage. ``klepto`` uses a\nsimple dictionary-sytle interface for all caches and archives, and all\ncaches can be applied to any Python function as a decorator. Keymaps\nare algorithms for converting a function's input signature to a unique\ndictionary, where the function's results are the dictionary value.\nThus for ``y = f(x)``, ``y`` will be stored in ``cache[x]`` (e.g. ``{x:y}``).\n\n``klepto`` provides both standard and *\"safe\"* caching, where *\"safe\"* caches\nare slower but can recover from hashing errors. ``klepto`` is intended\nto be used for distributed and parallel computing, where several of\nthe keymaps serialize the stored objects. Caches and archives are\nintended to be read/write accessible from different threads and\nprocesses. ``klepto`` enables a user to decorate a function, save the\nresults to a file or database archive, close the interpreter,\nstart a new session, and reload the function and it's cache.\n\n``klepto`` is part of ``pathos``, a Python framework for heterogeneous computing.\n``klepto`` is in active development, so any user feedback, bug reports, comments,\nor suggestions are highly appreciated. A list of issues is located at https://github.com/uqfoundation/klepto/issues, with a legacy list maintained at https://uqfoundation.github.io/project/pathos/query.\n\n\nMajor Features\n==============\n\n``klepto`` has standard and *\"safe\"* variants of the following:\n\n - ``lfu_cache`` - the least-frequently-used caching algorithm\n - ``lru_cache`` - the least-recently-used caching algorithm\n - ``mru_cache`` - the most-recently-used caching algorithm\n - ``rr_cache`` - the random-replacement caching algorithm\n - ``no_cache`` - a dummy caching interface to archiving\n - ``inf_cache`` - an infinitely-growing cache\n\n``klepto`` has the following archive types:\n\n - ``file_archive`` - a dictionary-style interface to a file\n - ``dir_archive`` - a dictionary-style interface to a folder of files\n - ``sqltable_archive`` - a dictionary-style interface to a sql database table\n - ``sql_archive`` - a dictionary-style interface to a sql database\n - ``hdfdir_archive`` - a dictionary-style interface to a folder of hdf5 files\n - ``hdf_archive`` - a dictionary-style interface to a hdf5 file\n - ``dict_archive`` - a dictionary with an archive interface\n - ``null_archive`` - a dictionary-style interface to a dummy archive \n\n``klepto`` provides the following keymaps:\n\n - ``keymap`` - keys are raw Python objects\n - ``hashmap`` - keys are a hash for the Python object\n - ``stringmap`` - keys are the Python object cast as a string\n - ``picklemap`` - keys are the serialized Python object\n\n``klepto`` also includes a few useful decorators providing:\n\n - simple, shallow, or deep rounding of function arguments\n - cryptographic key generation, with masking of selected arguments\n\n\nCurrent Release\n===============\n\nThe latest released version of ``klepto`` is available from:\n\n https://pypi.org/project/klepto\n\n``klepto`` is distributed under a 3-clause BSD license.\n\n\nDevelopment Version\n===================\n\nYou can get the latest development version with all the shiny new features at:\n\n https://github.com/uqfoundation\n\nIf you have a new contribution, please submit a pull request.\n\n\nInstallation\n============\n\n``klepto`` can be installed with ``pip``::\n\n $ pip install klepto\n\nTo include optional archive backends, such as HDF5 and SQL, in the install::\n\n $ pip install klepto[archives]\n\nTo include optional serializers, such as ``jsonpickle``, in the install::\n\n $ pip install klepto[crypto]\n\n\nRequirements\n============\n\n``klepto`` requires:\n\n - ``python`` (or ``pypy``), **>=3.8**\n - ``setuptools``, **>=42**\n - ``dill``, **>=0.3.8**\n - ``pox``, **>=0.3.4**\n\nOptional requirements:\n\n - ``h5py``, **>=2.8.0**\n - ``pandas``, **>=0.17.0**\n - ``sqlalchemy``, **>=1.4.0**\n - ``jsonpickle``, **>=0.9.6**\n - ``cloudpickle``, **>=0.5.2**\n\n\nMore Information\n================\n\nProbably the best way to get started is to look at the documentation at\nhttp://klepto.rtfd.io. Also see ``klepto.tests`` for a set of scripts that\ntest the caching and archiving functionalities in ``klepto``.\nYou can run the test suite with ``python -m klepto.tests``. The\nsource code is also generally well documented, so further questions may\nbe resolved by inspecting the code itself. Please feel free to submit\na ticket on github, or ask a question on stackoverflow (**@Mike McKerns**).\nIf you would like to share how you use ``klepto`` in your work, please send\nan email (to **mmckerns at uqfoundation dot org**).\n\n\nCitation\n========\n\nIf you use ``klepto`` to do research that leads to publication, we ask that you\nacknowledge use of ``klepto`` by citing the following in your publication::\n\n Michael McKerns and Michael Aivazis,\n \"pathos: a framework for heterogeneous computing\", 2010- ;\n https://uqfoundation.github.io/project/pathos\n\nPlease see https://uqfoundation.github.io/project/pathos or\nhttp://arxiv.org/pdf/1202.1056 for further information.\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "persistent caching to memory, disk, or database",
"version": "0.2.5",
"project_urls": {
"Bug Tracker": "https://github.com/uqfoundation/klepto/issues",
"Documentation": "http://klepto.rtfd.io",
"Download": "https://pypi.org/project/klepto/#files",
"Homepage": "https://github.com/uqfoundation/klepto",
"Source Code": "https://github.com/uqfoundation/klepto"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ee8574652f2d9fdcbcac7312f6338911b235f200377b2009c99ca2e776cef94e",
"md5": "5dbd31ae22e26731a8356c1fc4baca57",
"sha256": "f469154328dc17d83447d8606da3ba0dd8a3a467a077cb1fbfe98ea879d5d139"
},
"downloads": -1,
"filename": "klepto-0.2.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5dbd31ae22e26731a8356c1fc4baca57",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 82047,
"upload_time": "2024-01-28T00:09:16",
"upload_time_iso_8601": "2024-01-28T00:09:16.520769Z",
"url": "https://files.pythonhosted.org/packages/ee/85/74652f2d9fdcbcac7312f6338911b235f200377b2009c99ca2e776cef94e/klepto-0.2.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "97812faf5ccf7c66f7ea2a4dd45da0ede3fdaac6ef861e15c79a054c8966b6e8",
"md5": "6285577756a298c1c844d9530bec1227",
"sha256": "31db560f1c5d6af49e699c42d622428fd150c69308a1b134643e97b66d0334e9"
},
"downloads": -1,
"filename": "klepto-0.2.5.tar.gz",
"has_sig": false,
"md5_digest": "6285577756a298c1c844d9530bec1227",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 158278,
"upload_time": "2024-01-28T00:09:18",
"upload_time_iso_8601": "2024-01-28T00:09:18.397064Z",
"url": "https://files.pythonhosted.org/packages/97/81/2faf5ccf7c66f7ea2a4dd45da0ede3fdaac6ef861e15c79a054c8966b6e8/klepto-0.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-28 00:09:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "uqfoundation",
"github_project": "klepto",
"travis_ci": true,
"coveralls": true,
"github_actions": false,
"tox": true,
"lcname": "klepto"
}
JSON
