Django Cachalot
===============
Caches your Django ORM queries and automatically invalidates them.
Documentation: http://django-cachalot.readthedocs.io
----
.. image:: http://img.shields.io/pypi/v/django-cachalot.svg?style=flat-square&maxAge=3600
:target: https://pypi.python.org/pypi/django-cachalot
.. image:: https://img.shields.io/pypi/pyversions/django-cachalot
:target: https://django-cachalot.readthedocs.io/en/latest/
.. image:: https://github.com/noripyt/django-cachalot/actions/workflows/ci.yml/badge.svg
:target: https://github.com/noripyt/django-cachalot/actions/workflows/ci.yml
.. image:: http://img.shields.io/coveralls/noripyt/django-cachalot/master.svg?style=flat-square&maxAge=3600
:target: https://coveralls.io/r/noripyt/django-cachalot?branch=master
.. image:: http://img.shields.io/scrutinizer/g/noripyt/django-cachalot/master.svg?style=flat-square&maxAge=3600
:target: https://scrutinizer-ci.com/g/noripyt/django-cachalot/
.. image:: https://img.shields.io/discord/773656139207802881
:target: https://discord.gg/WFGFBk8rSU
----
Table of Contents:
- Quickstart
- Usage
- Hacking
- Benchmark
- Third-Party Cache Comparison
- Discussion
Quickstart
----------
Cachalot officially supports Python 3.7-3.11 and Django 3.2, 4.1, 4.2, 5.0, 5.1 with the databases PostgreSQL, SQLite, and MySQL.
Note: an upper limit on Django version is set for your safety. Please do not ignore it.
Usage
-----
#. ``pip install django-cachalot``
#. Add ``'cachalot',`` to your ``INSTALLED_APPS``
#. If you use multiple servers with a common cache server,
`double check their clock synchronisation <https://django-cachalot.readthedocs.io/en/latest/limits.html#multiple-servers>`_
#. If you modify data outside Django
– typically after restoring a SQL database –,
use the `manage.py command <https://django-cachalot.readthedocs.io/en/latest/quickstart.html#command>`_
#. Be aware of `the few other limits <https://django-cachalot.readthedocs.io/en/latest/limits.html#limits>`_
#. If you use
`django-debug-toolbar <https://github.com/jazzband/django-debug-toolbar>`_,
you can add ``'cachalot.panels.CachalotPanel',``
to your ``DEBUG_TOOLBAR_PANELS``
#. Enjoy!
Hacking
-------
To start developing, install the requirements
and run the tests via tox.
Make sure you have the following services:
* Memcached
* Redis
* PostgreSQL
* MySQL
For setup:
#. Install: ``pip install -r requirements/hacking.txt``
#. For PostgreSQL: ``CREATE ROLE cachalot LOGIN SUPERUSER;``
#. Run: ``tox --current-env`` to run the test suite on your current Python version.
#. You can also run specific databases and Django versions: ``tox -e py38-django3.1-postgresql-redis``
Benchmark
---------
Currently, benchmarks are supported on Linux and Mac/Darwin.
You will need a database called "cachalot" on MySQL and PostgreSQL.
Additionally, on PostgreSQL, you will need to create a role
called "cachalot." You can also run the benchmark, and it'll raise
errors with specific instructions for how to fix it.
#. Install: ``pip install -r requirements/benchmark.txt``
#. Run: ``python benchmark.py``
The output will be in benchmark/TODAY'S_DATE/
TODO Create Docker-compose file to allow for easier running of data.
Third-Party Cache Comparison
----------------------------
There are three main third party caches: cachalot, cache-machine, and cache-ops. Which do you use? We suggest a mix:
TL;DR Use cachalot for cold or modified <50 times per minutes (Most people should stick with only cachalot since you
most likely won't need to scale to the point of needing cache-machine added to the bowl). If you're an enterprise that
already has huge statistics, then mixing cold caches for cachalot and your hot caches with cache-machine is the best
mix. However, when performing joins with ``select_related`` and ``prefetch_related``, you can
get a nearly 100x speed up for your initial deployment.
Recall, cachalot caches THE ENTIRE TABLE. That's where its inefficiency stems from: if you keep updating the records,
then the cachalot constantly invalidates the table and re-caches. Luckily caching is very efficient, it's just the cache
invalidation part that kills all our systems. Look at Note 1 below to see how Reddit deals with it.
Cachalot is more-or-less intended for cold caches or "just-right" conditions. If you find a partition library for
Django (also authored but work-in-progress by `Andrew Chen Wang`_), then the caching will work better since sharding
the cold/accessed-the-least records aren't invalidated as much.
Cachalot is good when there are <50 modifications per minute on a hot cached table. This is mostly due to cache invalidation. It's the same with any cache,
which is why we suggest you use cache-machine for hot caches. Cache-machine caches individual objects, taking up more in the memory store but
invalidates those individual objects instead of the entire table like cachalot.
Yes, the bane of our entire existence lies in cache invalidation and naming variables. Why does cachalot suck when
stuck with a huge table that's modified rapidly? Since you've mixed your cold (90% of) with your hot (10% of) records,
you're caching and invalidating an entire table. It's like trying to boil 1 ton of noodles inside ONE pot instead of
100 pots boiling 1 ton of noodles. Which is more efficient? The splitting up of them.
Note 1: My personal experience with caches stems from Reddit's: https://web.archive.org/web/20210803213621/https://redditblog.com/2017/01/17/caching-at-reddit/
Note 2: Technical comparison: https://django-cachalot.readthedocs.io/en/latest/introduction.html#comparison-with-similar-tools
Discussion
----------
Help? Technical chat? `It's here on Discord <https://discord.gg/WFGFBk8rSU>`_.
Legacy chats:
- https://gitter.im/django-cachalot/Lobby
- https://join.slack.com/t/cachalotdjango/shared_invite/zt-dd0tj27b-cIH6VlaSOjAWnTG~II5~qw
.. _Andrew Chen Wang: https://github.com/Andrew-Chen-Wang
.. image:: https://raw.github.com/noripyt/django-cachalot/master/django-cachalot.jpg
Raw data
{
"_id": null,
"home_page": "https://github.com/noripyt/django-cachalot",
"name": "django-cachalot",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Bertrand Bordage, Andrew Chen Wang",
"author_email": "acwangpython@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/47/37/24b5344f682808d0f5488ef154e8ddb0d9c65a68e155ded7787d080667a3/django_cachalot-2.7.0.tar.gz",
"platform": null,
"description": "Django Cachalot\n===============\n\nCaches your Django ORM queries and automatically invalidates them.\n\nDocumentation: http://django-cachalot.readthedocs.io\n\n----\n\n.. image:: http://img.shields.io/pypi/v/django-cachalot.svg?style=flat-square&maxAge=3600\n :target: https://pypi.python.org/pypi/django-cachalot\n\n.. image:: https://img.shields.io/pypi/pyversions/django-cachalot\n :target: https://django-cachalot.readthedocs.io/en/latest/\n\n.. image:: https://github.com/noripyt/django-cachalot/actions/workflows/ci.yml/badge.svg\n :target: https://github.com/noripyt/django-cachalot/actions/workflows/ci.yml\n\n.. image:: http://img.shields.io/coveralls/noripyt/django-cachalot/master.svg?style=flat-square&maxAge=3600\n :target: https://coveralls.io/r/noripyt/django-cachalot?branch=master\n\n.. image:: http://img.shields.io/scrutinizer/g/noripyt/django-cachalot/master.svg?style=flat-square&maxAge=3600\n :target: https://scrutinizer-ci.com/g/noripyt/django-cachalot/\n\n.. image:: https://img.shields.io/discord/773656139207802881\n :target: https://discord.gg/WFGFBk8rSU\n\n----\n\nTable of Contents:\n\n- Quickstart\n- Usage\n- Hacking\n- Benchmark\n- Third-Party Cache Comparison\n- Discussion\n\nQuickstart\n----------\n\nCachalot officially supports Python 3.7-3.11 and Django 3.2, 4.1, 4.2, 5.0, 5.1 with the databases PostgreSQL, SQLite, and MySQL.\n\nNote: an upper limit on Django version is set for your safety. Please do not ignore it.\n\nUsage\n-----\n\n#. ``pip install django-cachalot``\n#. Add ``'cachalot',`` to your ``INSTALLED_APPS``\n#. If you use multiple servers with a common cache server,\n `double check their clock synchronisation <https://django-cachalot.readthedocs.io/en/latest/limits.html#multiple-servers>`_\n#. If you modify data outside Django\n \u2013\u00a0typically after restoring a SQL database\u00a0\u2013,\n use the `manage.py command <https://django-cachalot.readthedocs.io/en/latest/quickstart.html#command>`_\n#. Be aware of `the few other limits <https://django-cachalot.readthedocs.io/en/latest/limits.html#limits>`_\n#. If you use\n `django-debug-toolbar <https://github.com/jazzband/django-debug-toolbar>`_,\n you can add ``'cachalot.panels.CachalotPanel',``\n to your ``DEBUG_TOOLBAR_PANELS``\n#. Enjoy!\n\nHacking\n-------\n\nTo start developing, install the requirements\nand run the tests via tox.\n\nMake sure you have the following services:\n\n* Memcached\n* Redis\n* PostgreSQL\n* MySQL\n\nFor setup:\n\n#. Install: ``pip install -r requirements/hacking.txt``\n#. For PostgreSQL: ``CREATE ROLE cachalot LOGIN SUPERUSER;``\n#. Run: ``tox --current-env`` to run the test suite on your current Python version.\n#. You can also run specific databases and Django versions: ``tox -e py38-django3.1-postgresql-redis``\n\nBenchmark\n---------\n\nCurrently, benchmarks are supported on Linux and Mac/Darwin.\nYou will need a database called \"cachalot\" on MySQL and PostgreSQL.\nAdditionally, on PostgreSQL, you will need to create a role\ncalled \"cachalot.\" You can also run the benchmark, and it'll raise\nerrors with specific instructions for how to fix it.\n\n#. Install: ``pip install -r requirements/benchmark.txt``\n#. Run: ``python benchmark.py``\n\nThe output will be in benchmark/TODAY'S_DATE/\n\nTODO Create Docker-compose file to allow for easier running of data.\n\nThird-Party Cache Comparison\n----------------------------\n\nThere are three main third party caches: cachalot, cache-machine, and cache-ops. Which do you use? We suggest a mix:\n\nTL;DR Use cachalot for cold or modified <50 times per minutes (Most people should stick with only cachalot since you\nmost likely won't need to scale to the point of needing cache-machine added to the bowl). If you're an enterprise that\nalready has huge statistics, then mixing cold caches for cachalot and your hot caches with cache-machine is the best\nmix. However, when performing joins with ``select_related`` and ``prefetch_related``, you can\nget a nearly 100x speed up for your initial deployment.\n\nRecall, cachalot caches THE ENTIRE TABLE. That's where its inefficiency stems from: if you keep updating the records,\nthen the cachalot constantly invalidates the table and re-caches. Luckily caching is very efficient, it's just the cache\ninvalidation part that kills all our systems. Look at Note 1 below to see how Reddit deals with it.\n\nCachalot is more-or-less intended for cold caches or \"just-right\" conditions. If you find a partition library for\nDjango (also authored but work-in-progress by `Andrew Chen Wang`_), then the caching will work better since sharding\nthe cold/accessed-the-least records aren't invalidated as much.\n\nCachalot is good when there are <50 modifications per minute on a hot cached table. This is mostly due to cache invalidation. It's the same with any cache,\nwhich is why we suggest you use cache-machine for hot caches. Cache-machine caches individual objects, taking up more in the memory store but\ninvalidates those individual objects instead of the entire table like cachalot.\n\nYes, the bane of our entire existence lies in cache invalidation and naming variables. Why does cachalot suck when\nstuck with a huge table that's modified rapidly? Since you've mixed your cold (90% of) with your hot (10% of) records,\nyou're caching and invalidating an entire table. It's like trying to boil 1 ton of noodles inside ONE pot instead of\n100 pots boiling 1 ton of noodles. Which is more efficient? The splitting up of them.\n\nNote 1: My personal experience with caches stems from Reddit's: https://web.archive.org/web/20210803213621/https://redditblog.com/2017/01/17/caching-at-reddit/\n\nNote 2: Technical comparison: https://django-cachalot.readthedocs.io/en/latest/introduction.html#comparison-with-similar-tools\n\nDiscussion\n----------\n\nHelp? Technical chat? `It's here on Discord <https://discord.gg/WFGFBk8rSU>`_.\n\nLegacy chats:\n\n- https://gitter.im/django-cachalot/Lobby\n- https://join.slack.com/t/cachalotdjango/shared_invite/zt-dd0tj27b-cIH6VlaSOjAWnTG~II5~qw\n\n.. _Andrew Chen Wang: https://github.com/Andrew-Chen-Wang\n\n.. image:: https://raw.github.com/noripyt/django-cachalot/master/django-cachalot.jpg\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Caches your Django ORM queries and automatically invalidates them.",
"version": "2.7.0",
"project_urls": {
"Homepage": "https://github.com/noripyt/django-cachalot"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "91136290ac3e7ba6643585aa38e3bd448a99927f61a9004229847d129e9f8137",
"md5": "13b9fdeb48d67cc1b214ac7b373b7e3c",
"sha256": "a96fa16a25dc07110770d0bde0b01b444c065c7693d4c162c75da9e447f1750c"
},
"downloads": -1,
"filename": "django_cachalot-2.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "13b9fdeb48d67cc1b214ac7b373b7e3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 55475,
"upload_time": "2024-11-04T18:31:15",
"upload_time_iso_8601": "2024-11-04T18:31:15.915346Z",
"url": "https://files.pythonhosted.org/packages/91/13/6290ac3e7ba6643585aa38e3bd448a99927f61a9004229847d129e9f8137/django_cachalot-2.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "473724b5344f682808d0f5488ef154e8ddb0d9c65a68e155ded7787d080667a3",
"md5": "b0e45ecf16c6ad27ee66fabb62a56d5b",
"sha256": "834b95d5cffdc8953018a33a4e16ef63c8cfe2226a4dc36573e8bf3d62130175"
},
"downloads": -1,
"filename": "django_cachalot-2.7.0.tar.gz",
"has_sig": false,
"md5_digest": "b0e45ecf16c6ad27ee66fabb62a56d5b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 74110,
"upload_time": "2024-11-04T18:31:16",
"upload_time_iso_8601": "2024-11-04T18:31:16.976845Z",
"url": "https://files.pythonhosted.org/packages/47/37/24b5344f682808d0f5488ef154e8ddb0d9c65a68e155ded7787d080667a3/django_cachalot-2.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-04 18:31:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "noripyt",
"github_project": "django-cachalot",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "django-cachalot"
}
JSON
