Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Description: ========
nplusone
========
.. image:: https://img.shields.io/pypi/v/nplusone.svg
:target: http://badge.fury.io/py/nplusone
:alt: Latest version
.. image:: https://img.shields.io/travis/jmcarp/nplusone/master.svg
:target: https://travis-ci.org/jmcarp/nplusone
:alt: Travis-CI
.. image:: https://img.shields.io/codecov/c/github/jmcarp/nplusone/master.svg
:target: https://codecov.io/github/jmcarp/nplusone
:alt: Code coverage
nplusone is a library for detecting the n+1 queries problem in Python ORMs, including SQLAlchemy, Peewee, and the Django ORM.
The Problem
===========
Many object-relational mapping (ORM) libraries default to lazy loading for relationships. This pattern can be efficient when related rows are rarely accessed, but quickly becomes inefficient as relationships are accessed more frequently. In these cases, loading related rows eagerly using a ``JOIN`` can be vastly more performant. Unfortunately, understanding when to use lazy versus eager loading can be challenging: you might not notice the problem until your app has slowed to a crawl.
``nplusone`` is an ORM profiling tool to help diagnose and improve poor performance caused by inappropriate lazy loading. ``nplusone`` monitors applications using Django or SQLAlchemy and sends notifications when potentially expensive lazy loads are emitted. It can identify the offending relationship attribute and specific lines of code behind the problem, and recommend fixes for better performance.
``nplusone`` also detects inappropriate eager loading for Flask-SQLAlchemy and the Django ORM, emitting a warning when related data are eagerly loaded but never accessed within the current request.
Installation
============
::
pip install -U nplusone
nplusone supports Python >= 2.7 or >= 3.3.
Usage
=====
Note: ``nplusone`` should only be used for development and should not be deployed to production environments.
Django
******
Note: ``nplusone`` supports Django >= 1.8.
Add ``nplusone`` to ``INSTALLED_APPS``: ::
INSTALLED_APPS = (
...
'nplusone.ext.django',
)
Add ``NPlusOneMiddleware``: ::
MIDDLEWARE = (
'nplusone.ext.django.NPlusOneMiddleware',
...
)
Optionally configure logging settings: ::
NPLUSONE_LOGGER = logging.getLogger('nplusone')
NPLUSONE_LOG_LEVEL = logging.WARN
Configure logging handlers: ::
LOGGING = {
'version': 1,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'loggers': {
'nplusone': {
'handlers': ['console'],
'level': 'WARN',
},
},
}
When your app loads data lazily, ``nplusone`` will emit a log message: ::
Potential n+1 query detected on `<model>.<field>`
Consider using `select_related <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#select-related>`_ or `prefetch_related <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#prefetch-related>`_ in this case.
When your app eagerly loads related data without accessing it, ``nplusone`` will log a warning: ::
Potential unnecessary eager load detected on `<model>.<field>`
Flask-SQLAlchemy
****************
Wrap application with ``NPlusOne``: ::
from flask import Flask
from nplusone.ext.flask_sqlalchemy import NPlusOne
app = Flask(__name__)
NPlusOne(app)
Optionally configure logging settings: ::
app = Flask(__name__)
app.config['NPLUSONE_LOGGER'] = logging.getLogger('app.nplusone')
app.config['NPLUSONE_LOG_LEVEL'] = logging.ERROR
NPlusOne(app)
When your app loads data lazily, ``nplusone`` will emit a log message: ::
Potential n+1 query detected on `<model>.<field>`
Consider using ``subqueryload`` or ``joinedload`` in this case; see SQLAlchemy's guide to `relationship loading <http://docs.sqlalchemy.org/en/latest/orm/loading_relationships.html>`_ for complete documentation.
When your app eagerly loads related data without accessing it, ``nplusone`` will log a warning: ::
Potential unnecessary eager load detected on `<model>.<field>`
WSGI
****
For other frameworks that follow the WSGI specification, wrap your application with `NPlusOneMiddleware`. You must also import the relevant ``nplusone`` extension for your ORM: ::
import bottle
from nplusone.ext.wsgi import NPlusOneMiddleware
import nplusone.ext.sqlalchemy
app = NPlusOneMiddleware(bottle.app())
Generic
*******
The integrations above are coupled to the request-response cycle. To use ``nplusone`` outside the context of an HTTP request, use the ``Profiler`` context manager: You must also import the relevant ``nplusone`` extension for your ORM: ::
from nplusone.core import profiler
import nplusone.ext.sqlalchemy
with profiler.Profiler():
...
Customizing notifications
*************************
By default, ``nplusone`` logs all potentially unnecessary queries using a logger named "nplusone". When the `NPLUSONE_RAISE` configuration option is set, ``nplusone`` will also raise an ``NPlusOneError``. This can be used to force all automated tests involving unnecessary queries to fail. ::
# Django config
NPLUSONE_RAISE = True
# Flask config
app.config['NPLUSONE_RAISE'] = True
The exception type can also be specified, if desired, using the ``NPLUSONE_ERROR`` option.
Ignoring notifications
**********************
To ignore notifications from ``nplusone`` globally, configure the whitelist using the `NPLUSONE_WHITELIST` option: ::
# Django config
NPLUSONE_WHITELIST = [
{'label': 'n_plus_one', 'model': 'myapp.MyModel'}
]
# Flask-SQLAlchemy config
app.config['NPLUSONE_WHITELIST'] = [
{'label': 'unused_eager_load', 'model': 'MyModel', 'field': 'my_field'}
]
You can whitelist models by exact name or by `fnmatch <https://docs.python.org/3/library/fnmatch.html>`_ patterns: ::
# Django config
NPLUSONE_WHITELIST = [
{'model': 'myapp.*'}
]
To suppress notifications locally, use the ``ignore`` context manager: ::
from nplusone.core import signals
with signals.ignore(signals.lazy_load):
# lazy-load rows
# ...
License
=======
MIT licensed. See the bundled `LICENSE <https://github.com/jmcarp/nplusone/blob/master/LICENSE>`_ file for more details.
Keywords: nplusone
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Raw data
{
"_id": null,
"home_page": "https://github.com/jmcarp/nplusone",
"name": "nplusone",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Joshua Carp",
"author_email": "jm.carp@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/26/da/663f551cdda166eaf75a564f64d022c6eb03c710ba83c3fb0f4ac664ebde/nplusone-1.0.0.tar.gz",
"platform": "",
"description": "Permission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n\nDescription: ========\n nplusone\n ========\n \n .. image:: https://img.shields.io/pypi/v/nplusone.svg\n :target: http://badge.fury.io/py/nplusone\n :alt: Latest version\n \n .. image:: https://img.shields.io/travis/jmcarp/nplusone/master.svg\n :target: https://travis-ci.org/jmcarp/nplusone\n :alt: Travis-CI\n \n .. image:: https://img.shields.io/codecov/c/github/jmcarp/nplusone/master.svg\n :target: https://codecov.io/github/jmcarp/nplusone\n :alt: Code coverage\n \n nplusone is a library for detecting the n+1 queries problem in Python ORMs, including SQLAlchemy, Peewee, and the Django ORM.\n \n The Problem\n ===========\n \n Many object-relational mapping (ORM) libraries default to lazy loading for relationships. This pattern can be efficient when related rows are rarely accessed, but quickly becomes inefficient as relationships are accessed more frequently. In these cases, loading related rows eagerly using a ``JOIN`` can be vastly more performant. Unfortunately, understanding when to use lazy versus eager loading can be challenging: you might not notice the problem until your app has slowed to a crawl.\n \n ``nplusone`` is an ORM profiling tool to help diagnose and improve poor performance caused by inappropriate lazy loading. ``nplusone`` monitors applications using Django or SQLAlchemy and sends notifications when potentially expensive lazy loads are emitted. It can identify the offending relationship attribute and specific lines of code behind the problem, and recommend fixes for better performance.\n \n ``nplusone`` also detects inappropriate eager loading for Flask-SQLAlchemy and the Django ORM, emitting a warning when related data are eagerly loaded but never accessed within the current request.\n \n Installation\n ============\n \n ::\n \n pip install -U nplusone\n \n nplusone supports Python >= 2.7 or >= 3.3.\n \n Usage\n =====\n \n Note: ``nplusone`` should only be used for development and should not be deployed to production environments.\n \n Django\n ******\n \n Note: ``nplusone`` supports Django >= 1.8.\n \n Add ``nplusone`` to ``INSTALLED_APPS``: ::\n \n INSTALLED_APPS = (\n ...\n 'nplusone.ext.django',\n )\n \n Add ``NPlusOneMiddleware``: ::\n \n MIDDLEWARE = (\n 'nplusone.ext.django.NPlusOneMiddleware',\n ...\n )\n \n Optionally configure logging settings: ::\n \n NPLUSONE_LOGGER = logging.getLogger('nplusone')\n NPLUSONE_LOG_LEVEL = logging.WARN\n \n Configure logging handlers: ::\n \n LOGGING = {\n 'version': 1,\n 'handlers': {\n 'console': {\n 'class': 'logging.StreamHandler',\n },\n },\n 'loggers': {\n 'nplusone': {\n 'handlers': ['console'],\n 'level': 'WARN',\n },\n },\n }\n \n When your app loads data lazily, ``nplusone`` will emit a log message: ::\n \n Potential n+1 query detected on `<model>.<field>`\n \n Consider using `select_related <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#select-related>`_ or `prefetch_related <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#prefetch-related>`_ in this case.\n \n When your app eagerly loads related data without accessing it, ``nplusone`` will log a warning: ::\n \n Potential unnecessary eager load detected on `<model>.<field>`\n \n Flask-SQLAlchemy\n ****************\n \n Wrap application with ``NPlusOne``: ::\n \n from flask import Flask\n from nplusone.ext.flask_sqlalchemy import NPlusOne\n \n app = Flask(__name__)\n NPlusOne(app)\n \n Optionally configure logging settings: ::\n \n app = Flask(__name__)\n app.config['NPLUSONE_LOGGER'] = logging.getLogger('app.nplusone')\n app.config['NPLUSONE_LOG_LEVEL'] = logging.ERROR\n NPlusOne(app)\n \n When your app loads data lazily, ``nplusone`` will emit a log message: ::\n \n Potential n+1 query detected on `<model>.<field>`\n \n Consider using ``subqueryload`` or ``joinedload`` in this case; see SQLAlchemy's guide to `relationship loading <http://docs.sqlalchemy.org/en/latest/orm/loading_relationships.html>`_ for complete documentation.\n \n When your app eagerly loads related data without accessing it, ``nplusone`` will log a warning: ::\n \n Potential unnecessary eager load detected on `<model>.<field>`\n \n WSGI\n ****\n \n For other frameworks that follow the WSGI specification, wrap your application with `NPlusOneMiddleware`. You must also import the relevant ``nplusone`` extension for your ORM: ::\n \n import bottle\n from nplusone.ext.wsgi import NPlusOneMiddleware\n import nplusone.ext.sqlalchemy\n \n app = NPlusOneMiddleware(bottle.app())\n \n Generic\n *******\n \n The integrations above are coupled to the request-response cycle. To use ``nplusone`` outside the context of an HTTP request, use the ``Profiler`` context manager: You must also import the relevant ``nplusone`` extension for your ORM: ::\n \n from nplusone.core import profiler\n import nplusone.ext.sqlalchemy\n \n with profiler.Profiler():\n ...\n \n Customizing notifications\n *************************\n \n By default, ``nplusone`` logs all potentially unnecessary queries using a logger named \"nplusone\". When the `NPLUSONE_RAISE` configuration option is set, ``nplusone`` will also raise an ``NPlusOneError``. This can be used to force all automated tests involving unnecessary queries to fail. ::\n \n # Django config\n NPLUSONE_RAISE = True\n \n # Flask config\n app.config['NPLUSONE_RAISE'] = True\n \n The exception type can also be specified, if desired, using the ``NPLUSONE_ERROR`` option.\n \n Ignoring notifications\n **********************\n \n To ignore notifications from ``nplusone`` globally, configure the whitelist using the `NPLUSONE_WHITELIST` option: ::\n \n # Django config\n NPLUSONE_WHITELIST = [\n {'label': 'n_plus_one', 'model': 'myapp.MyModel'}\n ]\n \n # Flask-SQLAlchemy config\n app.config['NPLUSONE_WHITELIST'] = [\n {'label': 'unused_eager_load', 'model': 'MyModel', 'field': 'my_field'}\n ]\n \n You can whitelist models by exact name or by `fnmatch <https://docs.python.org/3/library/fnmatch.html>`_ patterns: ::\n \n # Django config\n NPLUSONE_WHITELIST = [\n {'model': 'myapp.*'}\n ]\n \n To suppress notifications locally, use the ``ignore`` context manager: ::\n \n from nplusone.core import signals\n \n with signals.ignore(signals.lazy_load):\n # lazy-load rows\n # ...\n \n License\n =======\n \n MIT licensed. See the bundled `LICENSE <https://github.com/jmcarp/nplusone/blob/master/LICENSE>`_ file for more details.\n \nKeywords: nplusone\nPlatform: UNKNOWN\nClassifier: Development Status :: 2 - Pre-Alpha\nClassifier: Intended Audience :: Developers\nClassifier: License :: OSI Approved :: MIT License\nClassifier: Natural Language :: English\nClassifier: Programming Language :: Python :: 2\nClassifier: Programming Language :: Python :: 2.7\nClassifier: Programming Language :: Python :: 3\nClassifier: Programming Language :: Python :: 3.3\nClassifier: Programming Language :: Python :: 3.4\nClassifier: Programming Language :: Python :: 3.5\nClassifier: Programming Language :: Python :: 3.6\n",
"bugtrack_url": null,
"license": "Copyright 2016 Joshua Carp",
"summary": "Detecting the n+1 queries problem in Python",
"version": "1.0.0",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "28ad956b86d90d826c25880274f8ebf3",
"sha256": "96b1e6e29e6af3e71b67d0cc012a5ec8c97c6a2f5399f4ba41a2bbe0e253a9ac"
},
"downloads": -1,
"filename": "nplusone-1.0.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "28ad956b86d90d826c25880274f8ebf3",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 15920,
"upload_time": "2018-05-21T03:40:23",
"upload_time_iso_8601": "2018-05-21T03:40:23.690434Z",
"url": "https://files.pythonhosted.org/packages/13/6b/9721ba7c68036316bd8aeb596b397253590c87d7045c9d6fc82b7364eff4/nplusone-1.0.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "a4baf3a278801ddbe5fb5fa484e0546e",
"sha256": "1726c0a10c0aa7eabb04e24db2882ff97b6b7ee29d729a8d97dcbd12ef5a5651"
},
"downloads": -1,
"filename": "nplusone-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "a4baf3a278801ddbe5fb5fa484e0546e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13501,
"upload_time": "2018-05-21T03:40:25",
"upload_time_iso_8601": "2018-05-21T03:40:25.010476Z",
"url": "https://files.pythonhosted.org/packages/26/da/663f551cdda166eaf75a564f64d022c6eb03c710ba83c3fb0f4ac664ebde/nplusone-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2018-05-21 03:40:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "jmcarp",
"github_project": "nplusone",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "nplusone"
}
JSON
