[](https://github.com/netzkolchose/django-fast-update/actions/workflows/django.yml)
[](https://coveralls.io/github/netzkolchose/django-fast-update?branch=master)
## django-fast-update ##
Faster db updates using `UPDATE FROM VALUES` sql variants.
### Installation & Usage ###
Run `pip install django-fast-update` and place `fast_update` in INSTALLED_APPS.
With attaching `FastUpdateManager` as a manager to your model, `fast_update`
can be used instead of `bulk_update`, e.g.:
```python
from django.db import models
from fast_update.query import FastUpdateManager
class MyModel(models.Model):
objects = FastUpdateManager()
field_a = ...
field_b = ...
field_c = ...
# to update multiple instances at once:
MyModel.objects.fast_update(bunch_of_instances, ['field_a', 'field_b', 'field_c'])
```
Alternatively `fast.fast_update` can be used directly with a queryset as first argument
(Warning - this skips most sanity checks with up to 30% speed gain,
so make sure not to feed something totally off).
### Compatibility ###
`fast_update` is known to work with these database versions:
- SQLite 3.15+
- PostgreSQL
- MariaDB 10.2+
- MySQL 5.7+
For unsupported database backends or outdated versions `fast_update` will fall back to `bulk_update`.
(It is possible to register fast update implementations for other db vendors with `register_implementation`.
See `fast_update/fast.py` for more details.)
Note that with `fast_update` f-expressions cannot be used anymore.
This is a design decision to not penalize update performance by some swiss-army-knife functionality.
If you have f-expressions in your update data, consider re-grouping the update steps and update those
fields with `update` or `bulk_update` instead.
Other than with `bulk_update` duplicates in a changeset are not allowed and will raise.
This is mainly a safety guard to not let slip through duplicates, where the final update state
would be undetermined or directly depend on the database's compatibility.
### copy_update ###
This is a PostgreSQL only update implementation based on `COPY FROM`. This runs even faster
than `fast_update` for medium to big changesets (but tends to be slower than `fast_update` for <100 objects).
`copy_update` follows the same interface idea as `bulk_update` and `fast_update`, minus a `batch_size`
argument (data is always transferred in one big batch). It can be used likewise from the `FastUpdateManager`.
`copy_update` also has no support for f-expressions, also duplicates will raise.
**Note** `copy_update` will probably never leave the alpha/PoC-state, as psycopg3 brings great COPY support,
which does a more secure value conversion and has a very fast C-version.
**Note** Django 4.2 brings psycopg3 support, which is currently not yet supported by `copy_update`.
While psycopg2 will keep working as before, psycopg3 will raise on attempts to use `copy_update` until #16 got resolved.
### Status ###
Currently beta, still some TODOs left.
The whole package is tested with Django 3.2 & 4.2 on Python 3.8 & 3.11.
### Performance ###
There is a management command in the example app testing performance of updates on the `FieldUpdate`
model (`./manage.py perf`).
`fast_update` is at least 8 times faster than `bulk_update`, and keeps making ground for bigger changesets.
This indicates different runtime complexity. `fast_update` grows almost linear for very big numbers of rows
(tested during some perf series against `copy_update` up to 10M), while `bulk_update` grows much faster
(looks quadratic to me, which can be lowered to linear by applying a proper `batch_size`,
but it stays very steep compared to `fast_update`).
For very big changesets `copy_update` is the clear winner, and even shows a substantial increase in updated rows/s
(within my test range, as upper estimate this of course cannot grow slower than linear,
as the data pumping will saturate to linear).
Raw data
{
"_id": null,
"home_page": "https://github.com/netzkolchose/django-fast-update",
"name": "django-fast-update",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "django,bulk_update,fast,update,fast_update",
"author": "netzkolchose",
"author_email": "j.breitbart@netzkolchose.de",
"download_url": "https://files.pythonhosted.org/packages/6b/81/cc21244eeec89d8b16bfafa475f118bd1b8e40c9628742478a7821f18c1e/django-fast-update-0.2.3.tar.gz",
"platform": null,
"description": "[](https://github.com/netzkolchose/django-fast-update/actions/workflows/django.yml)\n[](https://coveralls.io/github/netzkolchose/django-fast-update?branch=master)\n\n\n## django-fast-update ##\n\nFaster db updates using `UPDATE FROM VALUES` sql variants.\n\n### Installation & Usage ###\n\nRun `pip install django-fast-update` and place `fast_update` in INSTALLED_APPS.\n\nWith attaching `FastUpdateManager` as a manager to your model, `fast_update`\ncan be used instead of `bulk_update`, e.g.:\n\n```python\nfrom django.db import models\nfrom fast_update.query import FastUpdateManager\n\nclass MyModel(models.Model):\n objects = FastUpdateManager()\n field_a = ...\n field_b = ...\n field_c = ...\n\n\n# to update multiple instances at once:\nMyModel.objects.fast_update(bunch_of_instances, ['field_a', 'field_b', 'field_c'])\n```\n\nAlternatively `fast.fast_update` can be used directly with a queryset as first argument\n(Warning - this skips most sanity checks with up to 30% speed gain,\nso make sure not to feed something totally off).\n\n\n### Compatibility ###\n\n`fast_update` is known to work with these database versions:\n\n- SQLite 3.15+\n- PostgreSQL\n- MariaDB 10.2+\n- MySQL 5.7+\n\nFor unsupported database backends or outdated versions `fast_update` will fall back to `bulk_update`.\n(It is possible to register fast update implementations for other db vendors with `register_implementation`.\nSee `fast_update/fast.py` for more details.)\n\nNote that with `fast_update` f-expressions cannot be used anymore.\nThis is a design decision to not penalize update performance by some swiss-army-knife functionality.\nIf you have f-expressions in your update data, consider re-grouping the update steps and update those\nfields with `update` or `bulk_update` instead.\n\nOther than with `bulk_update` duplicates in a changeset are not allowed and will raise.\nThis is mainly a safety guard to not let slip through duplicates, where the final update state\nwould be undetermined or directly depend on the database's compatibility.\n\n\n### copy_update ###\n\nThis is a PostgreSQL only update implementation based on `COPY FROM`. This runs even faster\nthan `fast_update` for medium to big changesets (but tends to be slower than `fast_update` for <100 objects).\n\n`copy_update` follows the same interface idea as `bulk_update` and `fast_update`, minus a `batch_size`\nargument (data is always transferred in one big batch). It can be used likewise from the `FastUpdateManager`.\n`copy_update` also has no support for f-expressions, also duplicates will raise.\n\n**Note** `copy_update` will probably never leave the alpha/PoC-state, as psycopg3 brings great COPY support,\nwhich does a more secure value conversion and has a very fast C-version.\n\n**Note** Django 4.2 brings psycopg3 support, which is currently not yet supported by `copy_update`.\nWhile psycopg2 will keep working as before, psycopg3 will raise on attempts to use `copy_update` until #16 got resolved.\n\n\n### Status ###\n\nCurrently beta, still some TODOs left.\n\nThe whole package is tested with Django 3.2 & 4.2 on Python 3.8 & 3.11.\n\n\n### Performance ###\n\nThere is a management command in the example app testing performance of updates on the `FieldUpdate`\nmodel (`./manage.py perf`).\n\n`fast_update` is at least 8 times faster than `bulk_update`, and keeps making ground for bigger changesets.\nThis indicates different runtime complexity. `fast_update` grows almost linear for very big numbers of rows\n(tested during some perf series against `copy_update` up to 10M), while `bulk_update` grows much faster\n(looks quadratic to me, which can be lowered to linear by applying a proper `batch_size`,\nbut it stays very steep compared to `fast_update`).\n\nFor very big changesets `copy_update` is the clear winner, and even shows a substantial increase in updated rows/s\n(within my test range, as upper estimate this of course cannot grow slower than linear,\nas the data pumping will saturate to linear).",
"bugtrack_url": null,
"license": "MIT",
"summary": "Faster db updates for Django using UPDATE FROM VALUES sql variants.",
"version": "0.2.3",
"project_urls": {
"Download": "https://github.com/netzkolchose/django-fast-update/archive/v0.2.3.tar.gz",
"Homepage": "https://github.com/netzkolchose/django-fast-update"
},
"split_keywords": [
"django",
"bulk_update",
"fast",
"update",
"fast_update"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6b81cc21244eeec89d8b16bfafa475f118bd1b8e40c9628742478a7821f18c1e",
"md5": "884933aa205a58267afd3bde201cb3d5",
"sha256": "76ad711a1dc193e20233458d584e51ad6d354663b8952ada768eec1d997abc3c"
},
"downloads": -1,
"filename": "django-fast-update-0.2.3.tar.gz",
"has_sig": false,
"md5_digest": "884933aa205a58267afd3bde201cb3d5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18894,
"upload_time": "2023-05-08T08:33:49",
"upload_time_iso_8601": "2023-05-08T08:33:49.478034Z",
"url": "https://files.pythonhosted.org/packages/6b/81/cc21244eeec89d8b16bfafa475f118bd1b8e40c9628742478a7821f18c1e/django-fast-update-0.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-08 08:33:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "netzkolchose",
"github_project": "django-fast-update",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "django-fast-update"
}
JSON
