| |travisci| |version| |versions| |impls| |wheel| |coverage| |br-coverage|
.. |travisci| image:: https://travis-ci.org/jonathaneunice/mementos.svg?branch=master
:alt: Travis CI build status
:target: https://travis-ci.org/jonathaneunice/mementos
.. |version| image:: http://img.shields.io/pypi/v/mementos.svg?style=flat
:alt: PyPI Package latest release
:target: https://pypi.org/project/mementos
.. |versions| image:: https://img.shields.io/pypi/pyversions/mementos.svg
:alt: Supported versions
:target: https://pypi.org/project/mementos
.. |impls| image:: https://img.shields.io/pypi/implementation/mementos.svg
:alt: Supported implementations
:target: https://pypi.org/project/mementos
.. |wheel| image:: https://img.shields.io/pypi/wheel/mementos.svg
:alt: Wheel packaging support
:target: https://pypi.org/project/mementos
.. |coverage| image:: https://img.shields.io/badge/test_coverage-100%25-6600CC.svg
:alt: Test line coverage
:target: https://pypi.org/project/mementos
.. |br-coverage| image:: https://img.shields.io/badge/branch_coverage-100%25-6600CC.svg
:alt: Test branch coverage
:target: https://pypi.org/project/mementos
A quick way to make Python classes automatically memoize (a.k.a. cache) their
instances based on the arguments with which they are instantiated (i.e. args to
their ``__init__``).
It's a simple way to avoid repetitively creating expensive-to-create objects,
and to make sure objects that have a natural 'identity' are created only once.
If you want to be fancy, ``mementos`` implements the `Multiton
<https://en.wikipedia.org/wiki/Multiton_pattern>`_ software pattern.
Usage
=====
Say you have a class ``Thing`` that requires expensive computation to create, or
that should be created only once. Easy peasy::
from mementos import mementos
class Thing(mementos):
def __init__(self, name):
self.name = name
...
Then ``Thing`` objects will be memoized::
t1 = Thing("one")
t2 = Thing("one")
assert t1 is t2 # same instantiation args => same object
Under the Hood
==============
When you define a class ``class Thing(mementos)``, it looks like you're
subclassing the ``mementos`` class. Not really. ``mementos`` is a metaclass,
not a superclass. The full expression is equivalent to ``class
Thing(with_metaclass(MementoMetaclass, object))``, where ``with_metaclass`` and
``MementoMetaclass`` are also provided by the ``mementos`` module.
Metaclasses are not normal superclasses; instead they define how a class is
constructed. In effect, they define the mysterious ``__new__`` method that most
classes don't bother defining. In this case, ``mementos`` says in effect, "hey,
look in the cache for this object before you create another one."
If you like, you can use the longer invocation with the full ``with_metaclass``
spec, but it's not necessary unless you define your own memoizing functions.
More on that below.
Python 2 vs. Python 3
=====================
Python 2 and 3 have different forms for specifying metaclasses.
In Python 2::
from mementos import MementoMetaclass
class Thing(object):
__metaclass__ = MementoMetaclass # now I'm memoized!
...
Whereas Python 3 uses::
class Thing3(object, metaclass=MementoMetaclass):
...
``mementos`` supports either of these. But Python 2 and Python 3 don't
recognize each other's syntax for metaclass specification, so straightforward
code for one won't even compile for the other. The ``with_metaclass()``
function shown above is the way to go for cross-version compatibility. It's
very similar to that found in the ``six`` cross-version compatibility module.
Careful with Call Signatures
============================
``MementoMetaclass`` caches on call signature, which can vary greatly in Python,
even for logically identical calls. This is especially true if kwargs are used.
E.g. ``def func(a, b=2): pass`` can be called ``func(1)``, ``func(1, 2)``,
``func(a=1)``, ``func(1, b=2)``, or ``func(a=2, b=2)``. All of these resolve to
the same logical call--and this is just for two parameters! If there is more
than one keyword, they can be arbitrarily ordered, creating *many* logically
identical permutations.
So if you instantiate an object once, then again with a logically identical call
but using a different calling structure/signature, the object won't be created
and cached just once--it will be created and cached multiple times.::
o1 = Thing("lovely")
o2 = Thing(name="lovely")
assert o1 is not o2 # because the call signature is different
This may degrade performance, and can also create errors, if you're counting on
``mementos`` to create just one object. So don't do that. Use a consistent
calling style, and it won't be a problem.
In most cases, this isn't an issue, because objects tend to be instantiated with
a limited number of parameters, and you can take care that you instantiate them
with parallel call signatures. Since this works 99% of the time and has a simple
implementation, it's worth the price of this inelegance.
Partial Signatures
==================
If you want only part of the initialization-time call signature (i.e. arguments
to ``__init__``) to define an object's identity/cache key, there are two
approaches. One is to use ``MementoMetaclass`` and design ``__init__`` without
superfluous attributes, then create one or more secondary methods to add/set
useful-but-not-essential data. E.g.::
class OtherThing(with_metaclass(MementoMetaclass, object)):
def __init__(self, name):
self.name = name
self.color = None # unset for now
self.weight = None
def set(self, color=None, weight=None):
self.color = color or self.color
self.weight = weight or self.weight
return self
ot1 = OtherThing("one").set(color='blue')
ot2 = OtherThing("one").set(weight='light')
assert ot1 is ot2
assert ot1.color == ot2.color == 'blue'
assert ot1.weight == ot2.weight == 'light'
Or you can just define your own memoizing metaclass, using the factory function
described below.
Visiting the Factory
====================
The first iteration of ``mementos`` defined a single metaclass. It's since been
reimplemented as a parameterized meta-metaclass. Cool, huh? That basically means
that it defines a function, ``memento_factory()`` that, given a metaclass name
and a function defining how cache keys are constructed, returns a corresponding
metaclass. ``MementoMetaclass`` is the only metaclass that the module
pre-defines, but it's easy to define your own memoizing metaclass.::
from mementos import memento_factory, with_metaclass
IdTracker = memento_factory('IdTracker',
lambda cls, args, kwargs: (cls, id(args[0])) )
class MyTracker(with_metaclass(IdTracker, object)):
...
# object identity is the object id of first argument to __init__
# (and there must be one, else the args[0] reference => IndexError)
The first argument to ``memento_factory()`` is the name of the metaclass being
defined. The second is a callable (e.g. lambda expression or function object)
that takes three arguments: a class object, an argument ``list``, and a keyword
arg ``dict``. Note that there is no ``*`` or ``**`` magic--args passed to the
key function have already been resolved into basic data structures.
The callable must return a globally-unique, hashable key for an object. This key
will be stored in the ``_memento_cache``, which is a simple ``dict``.
When various arguments are used as the cache key/object identity, you may use a
``tuple`` that includes the class and arguments you want to key off of. This can
also help debugging, should you need to examine the ``_memento_cache`` cache
directly. But in cases like the ``IdTracker`` above, it's not mandatory that you
keep extra information around. The raw ``id(args[0])`` integer value would
suffice, as would a constructed string or other immutable, hashable value.
In cases where arguments are very flexible, or involve flexible data types,
a high-powered hashing function such as that provided by
`SuperHash <http://pypi.python.org/pypi/SuperHash>`_ might come in handy.
E.g.::
from superhash import superhash
SuperHashMeta = memento_factory('SuperHashMeta',
lambda cls, args, kwargs: (cls, superhash(args)) )
For the 1% edge-cases where multiple call variations must be
conclusively resolved to a unique canonical signature, that can be done on a
custom basis (based on the specific args). Or in Python 2.7 and 3.x, the
``inspect`` module's ``getcallargs()`` function can be used to create a generic
"call fingerprint" that can be used as a key. (See the tests for example code.)
Notes
=====
* See ``CHANGES.rst`` for the extended Change Log.
* ``mementos`` is not to be confused with `memento
<http://pypi.python.org/pypi/memento>`_, which does something completely
different.
* ``mementos`` was originally derived from `an ActiveState recipe
<http://code.activestate.com/recipes/286132-memento-design-pattern-in-python/>`_
by Valentino Volonghi. While the current implementation quite different and
the scope much broader, the availability of that recipe was what enabled
this module and the growing list of modules that depend on it. This is what
open source evolution is all about. Thank you, Valentino!
* It is safe to memoize multiple classes at the same time. They will all be
stored in the same cache, but their class is a part of the cache key, so the
values are distinct.
* This implementation is *not* thread-safe, in and of itself. If you're in a
multi-threaded environment, consider wrapping object instantiation in a
lock.
* Automated multi-version testing managed with `pytest
<http://pypi.python.org/pypi/pytest>`_, `pytest-cov
<http://pypi.python.org/pypi/pytest-cov>`_,
`coverage <https://pypi.python.org/pypi/coverage/4.0b1>`_
and `tox
<http://pypi.python.org/pypi/tox>`_. Continuous integration testing
with `Travis-CI <https://travis-ci.org/jonathaneunice/mementos>`_.
Packaging linting with `pyroma <https://pypi.python.org/pypi/pyroma>`_.
* The author, `Jonathan Eunice <mailto:jonathan.eunice@gmail.com>`_
or `@jeunice on Twitter <http://twitter.com/jeunice>`_ welcomes
your comments and suggestions.
Installation
============
To install or upgrade to the latest version::
pip install -U mementos
You may need to prefix these with ``sudo`` to authorize
installation. In environments without super-user privileges, you may want to
use ``pip``'s ``--user`` option, to install only for a single user, rather
than system-wide. Depending on your system configuration, you may also
need to use separate ``pip2`` and ``pip3`` programs to install for Python
2 and 3 respectively. As a fall-back for cases where the releationship between
``pip`` and the Python interpreter you want to run is unclear, you can
invoke ``pip`` as a module under a specific Python executable::
python3.6 -m pip install -U mementos
Testing
=======
To run the module tests, use one of these commands::
tox # normal run - speed optimized
tox -e py27 # run for a specific version only (e.g. py27, py34)
tox -c toxcov.ini # run full coverage tests
Raw data
{
"_id": null,
"home_page": "https://bitbucket.org/jeunice/mementos",
"name": "mementos",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Jonathan Eunice",
"author_email": "jonathan.eunice@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/48/0c/1287a95aa809b618ca2961509bb77d0e3e8ef984a99b52f72f1e76514e21/mementos-1.3.1.zip",
"platform": "",
"description": "\n| |travisci| |version| |versions| |impls| |wheel| |coverage| |br-coverage|\n\n.. |travisci| image:: https://travis-ci.org/jonathaneunice/mementos.svg?branch=master\n :alt: Travis CI build status\n :target: https://travis-ci.org/jonathaneunice/mementos\n\n.. |version| image:: http://img.shields.io/pypi/v/mementos.svg?style=flat\n :alt: PyPI Package latest release\n :target: https://pypi.org/project/mementos\n\n.. |versions| image:: https://img.shields.io/pypi/pyversions/mementos.svg\n :alt: Supported versions\n :target: https://pypi.org/project/mementos\n\n.. |impls| image:: https://img.shields.io/pypi/implementation/mementos.svg\n :alt: Supported implementations\n :target: https://pypi.org/project/mementos\n\n.. |wheel| image:: https://img.shields.io/pypi/wheel/mementos.svg\n :alt: Wheel packaging support\n :target: https://pypi.org/project/mementos\n\n.. |coverage| image:: https://img.shields.io/badge/test_coverage-100%25-6600CC.svg\n :alt: Test line coverage\n :target: https://pypi.org/project/mementos\n\n.. |br-coverage| image:: https://img.shields.io/badge/branch_coverage-100%25-6600CC.svg\n :alt: Test branch coverage\n :target: https://pypi.org/project/mementos\n\nA quick way to make Python classes automatically memoize (a.k.a. cache) their\ninstances based on the arguments with which they are instantiated (i.e. args to\ntheir ``__init__``).\n\nIt's a simple way to avoid repetitively creating expensive-to-create objects,\nand to make sure objects that have a natural 'identity' are created only once.\nIf you want to be fancy, ``mementos`` implements the `Multiton\n<https://en.wikipedia.org/wiki/Multiton_pattern>`_ software pattern.\n\nUsage\n=====\n\nSay you have a class ``Thing`` that requires expensive computation to create, or\nthat should be created only once. Easy peasy::\n\n from mementos import mementos\n\n class Thing(mementos):\n\n def __init__(self, name):\n self.name = name\n\n ...\n\nThen ``Thing`` objects will be memoized::\n\n t1 = Thing(\"one\")\n t2 = Thing(\"one\")\n assert t1 is t2 # same instantiation args => same object\n\n\nUnder the Hood\n==============\n\nWhen you define a class ``class Thing(mementos)``, it looks like you're\nsubclassing the ``mementos`` class. Not really. ``mementos`` is a metaclass,\nnot a superclass. The full expression is equivalent to ``class\nThing(with_metaclass(MementoMetaclass, object))``, where ``with_metaclass`` and\n``MementoMetaclass`` are also provided by the ``mementos`` module. \n\nMetaclasses are not normal superclasses; instead they define how a class is\nconstructed. In effect, they define the mysterious ``__new__`` method that most\nclasses don't bother defining. In this case, ``mementos`` says in effect, \"hey,\nlook in the cache for this object before you create another one.\"\n\nIf you like, you can use the longer invocation with the full ``with_metaclass``\nspec, but it's not necessary unless you define your own memoizing functions.\nMore on that below.\n\nPython 2 vs. Python 3\n=====================\n\nPython 2 and 3 have different forms for specifying metaclasses.\nIn Python 2::\n\n from mementos import MementoMetaclass\n\n class Thing(object):\n\n __metaclass__ = MementoMetaclass # now I'm memoized!\n\n ...\n\nWhereas Python 3 uses::\n\n class Thing3(object, metaclass=MementoMetaclass):\n\n ...\n\n``mementos`` supports either of these. But Python 2 and Python 3 don't\nrecognize each other's syntax for metaclass specification, so straightforward\ncode for one won't even compile for the other. The ``with_metaclass()``\nfunction shown above is the way to go for cross-version compatibility. It's\nvery similar to that found in the ``six`` cross-version compatibility module.\n\nCareful with Call Signatures\n============================\n\n``MementoMetaclass`` caches on call signature, which can vary greatly in Python,\neven for logically identical calls. This is especially true if kwargs are used.\nE.g. ``def func(a, b=2): pass`` can be called ``func(1)``, ``func(1, 2)``,\n``func(a=1)``, ``func(1, b=2)``, or ``func(a=2, b=2)``. All of these resolve to\nthe same logical call--and this is just for two parameters! If there is more\nthan one keyword, they can be arbitrarily ordered, creating *many* logically\nidentical permutations.\n\nSo if you instantiate an object once, then again with a logically identical call\nbut using a different calling structure/signature, the object won't be created\nand cached just once--it will be created and cached multiple times.::\n\n o1 = Thing(\"lovely\")\n o2 = Thing(name=\"lovely\")\n assert o1 is not o2 # because the call signature is different\n\nThis may degrade performance, and can also create errors, if you're counting on\n``mementos`` to create just one object. So don't do that. Use a consistent\ncalling style, and it won't be a problem.\n\nIn most cases, this isn't an issue, because objects tend to be instantiated with\na limited number of parameters, and you can take care that you instantiate them\nwith parallel call signatures. Since this works 99% of the time and has a simple\nimplementation, it's worth the price of this inelegance.\n\nPartial Signatures\n==================\n\nIf you want only part of the initialization-time call signature (i.e. arguments\nto ``__init__``) to define an object's identity/cache key, there are two\napproaches. One is to use ``MementoMetaclass`` and design ``__init__`` without\nsuperfluous attributes, then create one or more secondary methods to add/set\nuseful-but-not-essential data. E.g.::\n\n class OtherThing(with_metaclass(MementoMetaclass, object)):\n\n def __init__(self, name):\n self.name = name\n self.color = None # unset for now\n self.weight = None\n\n def set(self, color=None, weight=None):\n self.color = color or self.color\n self.weight = weight or self.weight\n return self\n\n ot1 = OtherThing(\"one\").set(color='blue')\n ot2 = OtherThing(\"one\").set(weight='light')\n assert ot1 is ot2\n assert ot1.color == ot2.color == 'blue'\n assert ot1.weight == ot2.weight == 'light'\n\nOr you can just define your own memoizing metaclass, using the factory function\ndescribed below.\n\nVisiting the Factory\n====================\n\nThe first iteration of ``mementos`` defined a single metaclass. It's since been\nreimplemented as a parameterized meta-metaclass. Cool, huh? That basically means\nthat it defines a function, ``memento_factory()`` that, given a metaclass name\nand a function defining how cache keys are constructed, returns a corresponding\nmetaclass. ``MementoMetaclass`` is the only metaclass that the module\npre-defines, but it's easy to define your own memoizing metaclass.::\n\n from mementos import memento_factory, with_metaclass\n\n IdTracker = memento_factory('IdTracker',\n lambda cls, args, kwargs: (cls, id(args[0])) )\n\n class MyTracker(with_metaclass(IdTracker, object)):\n ...\n\n # object identity is the object id of first argument to __init__\n # (and there must be one, else the args[0] reference => IndexError)\n\nThe first argument to ``memento_factory()`` is the name of the metaclass being\ndefined. The second is a callable (e.g. lambda expression or function object)\nthat takes three arguments: a class object, an argument ``list``, and a keyword\narg ``dict``. Note that there is no ``*`` or ``**`` magic--args passed to the\nkey function have already been resolved into basic data structures.\n\nThe callable must return a globally-unique, hashable key for an object. This key\nwill be stored in the ``_memento_cache``, which is a simple ``dict``.\n\nWhen various arguments are used as the cache key/object identity, you may use a\n``tuple`` that includes the class and arguments you want to key off of. This can\nalso help debugging, should you need to examine the ``_memento_cache`` cache\ndirectly. But in cases like the ``IdTracker`` above, it's not mandatory that you\nkeep extra information around. The raw ``id(args[0])`` integer value would\nsuffice, as would a constructed string or other immutable, hashable value.\n\nIn cases where arguments are very flexible, or involve flexible data types,\na high-powered hashing function such as that provided by\n`SuperHash <http://pypi.python.org/pypi/SuperHash>`_ might come in handy.\nE.g.::\n\n from superhash import superhash\n\n SuperHashMeta = memento_factory('SuperHashMeta',\n lambda cls, args, kwargs: (cls, superhash(args)) )\n\nFor the 1% edge-cases where multiple call variations must be\nconclusively resolved to a unique canonical signature, that can be done on a\ncustom basis (based on the specific args). Or in Python 2.7 and 3.x, the\n``inspect`` module's ``getcallargs()`` function can be used to create a generic\n\"call fingerprint\" that can be used as a key. (See the tests for example code.)\n\nNotes\n=====\n\n* See ``CHANGES.rst`` for the extended Change Log.\n\n* ``mementos`` is not to be confused with `memento\n <http://pypi.python.org/pypi/memento>`_, which does something completely\n different.\n\n* ``mementos`` was originally derived from `an ActiveState recipe\n <http://code.activestate.com/recipes/286132-memento-design-pattern-in-python/>`_\n by Valentino Volonghi. While the current implementation quite different and\n the scope much broader, the availability of that recipe was what enabled\n this module and the growing list of modules that depend on it. This is what\n open source evolution is all about. Thank you, Valentino!\n\n* It is safe to memoize multiple classes at the same time. They will all be\n stored in the same cache, but their class is a part of the cache key, so the\n values are distinct.\n\n* This implementation is *not* thread-safe, in and of itself. If you're in a\n multi-threaded environment, consider wrapping object instantiation in a\n lock.\n\n* Automated multi-version testing managed with `pytest\n <http://pypi.python.org/pypi/pytest>`_, `pytest-cov\n <http://pypi.python.org/pypi/pytest-cov>`_,\n `coverage <https://pypi.python.org/pypi/coverage/4.0b1>`_\n and `tox\n <http://pypi.python.org/pypi/tox>`_. Continuous integration testing\n with `Travis-CI <https://travis-ci.org/jonathaneunice/mementos>`_.\n Packaging linting with `pyroma <https://pypi.python.org/pypi/pyroma>`_.\n\n* The author, `Jonathan Eunice <mailto:jonathan.eunice@gmail.com>`_\n or `@jeunice on Twitter <http://twitter.com/jeunice>`_ welcomes\n your comments and suggestions.\n\nInstallation\n============\n\nTo install or upgrade to the latest version::\n\n pip install -U mementos\n\nYou may need to prefix these with ``sudo`` to authorize\ninstallation. In environments without super-user privileges, you may want to\nuse ``pip``'s ``--user`` option, to install only for a single user, rather\nthan system-wide. Depending on your system configuration, you may also\nneed to use separate ``pip2`` and ``pip3`` programs to install for Python \n2 and 3 respectively. As a fall-back for cases where the releationship between\n``pip`` and the Python interpreter you want to run is unclear, you can\ninvoke ``pip`` as a module under a specific Python executable::\n\n python3.6 -m pip install -U mementos\n\nTesting\n=======\n\nTo run the module tests, use one of these commands::\n\n tox # normal run - speed optimized\n tox -e py27 # run for a specific version only (e.g. py27, py34)\n tox -c toxcov.ini # run full coverage tests\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Memoizing metaclass. Drop-dead simple way to create cached objects",
"version": "1.3.1",
"project_urls": {
"Homepage": "https://bitbucket.org/jeunice/mementos"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "329f44d24c854245214c1ace565aa8269b64d4ab189f4eb9783a2ac00c98072c",
"md5": "2a570330f8421ed4b41cab6cda76f1af",
"sha256": "fee20b2440a06657bb942b8d935f0c3d468b5ad58b07b33b609fd92fe864bc7f"
},
"downloads": -1,
"filename": "mementos-1.3.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "2a570330f8421ed4b41cab6cda76f1af",
"packagetype": "bdist_wheel",
"python_version": "3.6",
"requires_python": null,
"size": 12074,
"upload_time": "2018-06-03T21:49:26",
"upload_time_iso_8601": "2018-06-03T21:49:26.632438Z",
"url": "https://files.pythonhosted.org/packages/32/9f/44d24c854245214c1ace565aa8269b64d4ab189f4eb9783a2ac00c98072c/mementos-1.3.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "480c1287a95aa809b618ca2961509bb77d0e3e8ef984a99b52f72f1e76514e21",
"md5": "46e7bb3db4ffbbbefabd6023a8052e49",
"sha256": "e3574b3d16a1b3cbfd03e049701bedcfcff131ea5f93904563672221109f53e7"
},
"downloads": -1,
"filename": "mementos-1.3.1.zip",
"has_sig": false,
"md5_digest": "46e7bb3db4ffbbbefabd6023a8052e49",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 22693,
"upload_time": "2018-06-03T21:49:24",
"upload_time_iso_8601": "2018-06-03T21:49:24.022233Z",
"url": "https://files.pythonhosted.org/packages/48/0c/1287a95aa809b618ca2961509bb77d0e3e8ef984a99b52f72f1e76514e21/mementos-1.3.1.zip",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2018-06-03 21:49:24",
"github": false,
"gitlab": false,
"bitbucket": true,
"codeberg": false,
"bitbucket_user": "jeunice",
"bitbucket_project": "mementos",
"lcname": "mementos"
}
JSON