django-linear-migrations


Namedjango-linear-migrations JSON
Version 1.6.0 PyPI version JSON
download
home_pagehttps://github.com/adamchainz/django-linear-migrations
SummaryEnsure your migrations are linear.
upload_time2021-04-08 11:39:52
maintainer
docs_urlNone
authorAdam Johnson
requires_python>=3.6
licenseMIT
keywords django
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            ========================
django-linear-migrations
========================

.. image:: https://img.shields.io/github/workflow/status/adamchainz/django-linear-migrations/CI/main?style=for-the-badge
   :target: https://github.com/adamchainz/django-linear-migrations/actions?workflow=CI

.. image:: https://img.shields.io/codecov/c/github/adamchainz/django-linear-migrations/main?style=for-the-badge
   :target: https://app.codecov.io/gh/adamchainz/django-linear-migrations

.. image:: https://img.shields.io/pypi/v/django-linear-migrations.svg?style=for-the-badge
   :target: https://pypi.org/project/django-linear-migrations/

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg?style=for-the-badge
   :target: https://github.com/psf/black

.. image:: https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white&style=for-the-badge
   :target: https://github.com/pre-commit/pre-commit
   :alt: pre-commit

Ensure your migration history is linear.

For a bit of background, see the `introductory blog post <https://adamj.eu/tech/2020/12/10/introducing-django-linear-migrations/>`__.

Requirements
============

Python 3.6 to 3.9 supported.

Django 2.2 to 3.2 supported.

----

**Are your tests slow?**
Check out my book `Speed Up Your Django Tests <https://gumroad.com/l/suydt>`__ which covers loads of best practices so you can write faster, more accurate tests.

----

Installation
============

**First,** install with pip:

.. code-block:: bash

    python -m pip install django-linear-migrations

**Second,** add the app to your ``INSTALLED_APPS`` setting:

.. code-block:: python

    INSTALLED_APPS = [
        ...
        "django_linear_migrations",
        ...
    ]

The app relies on overriding the built-in ``makemigrations`` command.
*If your project has a custom* ``makemigrations`` *command,* ensure the app containing your custom command is **above** ``django_linear_migrations``, and that your command subclasses its ``Command`` class:

.. code-block:: python

    # myapp/management/commands/makemigrations.py
    from django_linear_migrations.management.commands.makemigrations import (
        Command as BaseCommand,
    )


    class Command(BaseCommand):
        ...

**Third,** check the automatic detection of first-party apps.
Run this command:

.. code-block:: sh

    python manage.py create-max-migration-files --dry-run

This command is for creating ``max_migration.txt`` files (more on which later) - in dry run mode it lists the apps it would make such files for.
It tries to automatically detect which apps are first-party, i.e. belong to your project.
The automatic detection checks the path of app’s code to see if is within a virtualenv, but this detection can sometimes fail, for example on editable packages installed with ``-e``.
If you see any apps listed that *aren’t* part of your project, define the list of first-party apps’ labels in a ``FIRST_PARTY_APPS`` setting that you combine into ``INSTALLED_APPS``:

.. code-block:: python

    FIRST_PARTY_APPS = [

    ]

    INSTALLED_APPS = FIRST_PARTY_APPS + [
        "django_linear_migrations",
        ...
    ]

(Note: Django recommends you always list first-party apps first in your project so they can override things in third-party and contrib apps.)

**Fourth,** create the ``max_migration.txt`` files for your first-party apps by re-running the command without the dry run flag:

.. code-block:: sh

    python manage.py create-max-migration-files

In the future, when you add a new app to your project, you’ll need to create its ``max_migration.txt`` file.
If you defined ``FIRST_PARTY_APPS``, add it there, then rerun the creation command for the new app by specifying its label:

.. code-block:: sh

    python manage.py create-max-migration-files my_new_app

Usage
=====

django-linear-migrations helps you work on Django projects where several branches adding migrations may be in progress at any time.
It enforces that your apps have a *linear* migration history, avoiding merge migrations and the problems they can cause from migrations running in different orders.
It does this by making ``makemigrations`` record the name of the latest migration in per-app ``max_migration.txt`` files.
These files will then cause a merge conflicts in your source control tool (Git, Mercurial, etc.) in the case of migrations being developed in parallel.
The first merged migration for an app will prevent the second from being merged, without addressing the conflict.
The included ``rebase-migration`` command can help automatically such conflicts.

System Checks
-------------

django-linear-migrations comes with several system checks that verify that your ``max_migration.txt`` files are in sync.
These are:

* ``dlm.E001``: ``<app_label>``'s max_migration.txt does not exist.
* ``dlm.E002``: ``<app_label>``'s max_migration.txt contains multiple lines.
* ``dlm.E003``: ``<app_label>``'s max_migration.txt points to non-existent migration '``<bad_migration_name>``'.
* ``dlm.E004``: ``<app_label>``'s max_migration.txt contains '``<max_migration_name>``', but the latest migration is '``<real_max_migration_name>``'.

``rebase-migration`` command
----------------------------

This management command can help you fix migration conflicts.
Following a conflicted “rebase” operation in your source control tool, run it with the name of the app to auto-fix the migrations for:

.. code-block:: console

    $ python manage.py rebase-migration <app_label>

Note rebasing the migration might not always be the *correct* thing to do.
If the migrations in main and feature branches have both affected the same models, rebasing the migration on the end may not make sense.
However, such parallel changes would *normally* cause conflicts in your models files or other parts of the source code as well.

Let's walk through an example using Git, although it should extend to other source control tools.

Imagine you were working on your project's ``books`` app in a feature branch called ``titles`` and created a migration called ``0002_longer_titles``.
Meanwhile a commit has been merged to your ``main`` branch with a *different* 2nd migration for ``books`` called ``0002_author_nicknames``.
Thanks to django-linear-migrations, the ``max_migration.txt`` file will show as conflicted between your feature and main branches.

You start the fix by reversing your new migration from your local database.
This is necessary since it will be renamed after rebasing and seen as unapplied.
You do this by switching to the feature branch ``titles`` migrating back to the last common migration:

.. code-block:: console

    $ git switch titles
    $ python manage.py migrate books 0001

You then fetch the latest code:

.. code-block:: console

    $ git switch main
    $ git pull
    ...

You then rebase your ``titles`` branch on top of it, for which Git will detect the conflict on ``max_migration.txt``:

.. code-block:: console

    $ git switch titles
    $ git rebase main
    Auto-merging books/models.py
    CONFLICT (content): Merge conflict in books/migrations/max_migration.txt
    error: could not apply 123456789... Increase Book title length
    Resolve all conflicts manually, mark them as resolved with
    "git add/rm <conflicted_files>", then run "git rebase --continue".
    You can instead skip this commit: run "git rebase --skip".
    To abort and get back to the state before "git rebase", run "git rebase --abort".
    Could not apply 123456789... Increase Book title length

If you look at the contents of the ``books`` app's ``max_migration.txt`` at this point, it will look something like this:

.. code-block:: console

    $ cat books/migrations/max_migration.txt
    <<<<<<< HEAD
    0002_author_nicknames
    =======
    0002_longer_titles
    >>>>>>> 123456789 (Increase Book title length)

It's at this point you can use ``rebase-migration`` to automatically fix the ``books`` migration history:

.. code-block:: console

    $ python manage.py rebase-migration books
    Renamed 0002_longer_titles.py to 0003_longer_titles.py, updated its dependencies, and updated max_migration.txt.

This places the conflicted migration on the end of the migration history.
It renames the file appropriately, modifies its ``dependencies = [...]`` declaration, and updates the migration named in ``max_migration.txt`` appropriately.

After this, you should be able to continue the rebase:

.. code-block:: console

    $ git add books/migrations
    $ git rebase --continue

And then migrate your local database to allow you to continue development:

.. code-block:: console

    $ python manage.py migrate books
    Operations to perform:
      Target specific migration: 0003_longer_titles, from books
    Running migrations:
      Applying books.0002_author_nicknames... OK
      Applying books.0003_longer_titles... OK

Inspiration
===========

I’ve seen similar techniques to the one implemented by django-linear-migrations at several places, and they acted as the inspiration for putting this package together.
My previous client `Pollen <https://pollen.co/>`__ and current client `ev.energy <https://ev.energy/>`__ both have implementations.
This `Doordash blogpost <https://medium.com/@DoorDash/tips-for-building-high-quality-django-apps-at-scale-a5a25917b2b5>`__ covers a similar system that uses a single file for tracking latest migrations.
And there's also a package called `django-migrations-git-conflicts <https://pypi.org/project/django-migrations-git-conflicts/>`__ which works fairly similarly.



            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/adamchainz/django-linear-migrations",
    "name": "django-linear-migrations",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.6",
    "maintainer_email": "",
    "keywords": "Django",
    "author": "Adam Johnson",
    "author_email": "me@adamj.eu",
    "download_url": "https://files.pythonhosted.org/packages/f2/1b/476819a1cac2fc627f746e09ae8ecd10d65157592a46c00cb994f96ef009/django-linear-migrations-1.6.0.tar.gz",
    "platform": "",
    "description": "========================\ndjango-linear-migrations\n========================\n\n.. image:: https://img.shields.io/github/workflow/status/adamchainz/django-linear-migrations/CI/main?style=for-the-badge\n   :target: https://github.com/adamchainz/django-linear-migrations/actions?workflow=CI\n\n.. image:: https://img.shields.io/codecov/c/github/adamchainz/django-linear-migrations/main?style=for-the-badge\n   :target: https://app.codecov.io/gh/adamchainz/django-linear-migrations\n\n.. image:: https://img.shields.io/pypi/v/django-linear-migrations.svg?style=for-the-badge\n   :target: https://pypi.org/project/django-linear-migrations/\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg?style=for-the-badge\n   :target: https://github.com/psf/black\n\n.. image:: https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white&style=for-the-badge\n   :target: https://github.com/pre-commit/pre-commit\n   :alt: pre-commit\n\nEnsure your migration history is linear.\n\nFor a bit of background, see the `introductory blog post <https://adamj.eu/tech/2020/12/10/introducing-django-linear-migrations/>`__.\n\nRequirements\n============\n\nPython 3.6 to 3.9 supported.\n\nDjango 2.2 to 3.2 supported.\n\n----\n\n**Are your tests slow?**\nCheck out my book `Speed Up Your Django Tests <https://gumroad.com/l/suydt>`__ which covers loads of best practices so you can write faster, more accurate tests.\n\n----\n\nInstallation\n============\n\n**First,** install with pip:\n\n.. code-block:: bash\n\n    python -m pip install django-linear-migrations\n\n**Second,** add the app to your ``INSTALLED_APPS`` setting:\n\n.. code-block:: python\n\n    INSTALLED_APPS = [\n        ...\n        \"django_linear_migrations\",\n        ...\n    ]\n\nThe app relies on overriding the built-in ``makemigrations`` command.\n*If your project has a custom* ``makemigrations`` *command,* ensure the app containing your custom command is **above** ``django_linear_migrations``, and that your command subclasses its ``Command`` class:\n\n.. code-block:: python\n\n    # myapp/management/commands/makemigrations.py\n    from django_linear_migrations.management.commands.makemigrations import (\n        Command as BaseCommand,\n    )\n\n\n    class Command(BaseCommand):\n        ...\n\n**Third,** check the automatic detection of first-party apps.\nRun this command:\n\n.. code-block:: sh\n\n    python manage.py create-max-migration-files --dry-run\n\nThis command is for creating ``max_migration.txt`` files (more on which later) - in dry run mode it lists the apps it would make such files for.\nIt tries to automatically detect which apps are first-party, i.e. belong to your project.\nThe automatic detection checks the path of app\u2019s code to see if is within a virtualenv, but this detection can sometimes fail, for example on editable packages installed with ``-e``.\nIf you see any apps listed that *aren\u2019t* part of your project, define the list of first-party apps\u2019 labels in a ``FIRST_PARTY_APPS`` setting that you combine into ``INSTALLED_APPS``:\n\n.. code-block:: python\n\n    FIRST_PARTY_APPS = [\n\n    ]\n\n    INSTALLED_APPS = FIRST_PARTY_APPS + [\n        \"django_linear_migrations\",\n        ...\n    ]\n\n(Note: Django recommends you always list first-party apps first in your project so they can override things in third-party and contrib apps.)\n\n**Fourth,** create the ``max_migration.txt`` files for your first-party apps by re-running the command without the dry run flag:\n\n.. code-block:: sh\n\n    python manage.py create-max-migration-files\n\nIn the future, when you add a new app to your project, you\u2019ll need to create its ``max_migration.txt`` file.\nIf you defined ``FIRST_PARTY_APPS``, add it there, then rerun the creation command for the new app by specifying its label:\n\n.. code-block:: sh\n\n    python manage.py create-max-migration-files my_new_app\n\nUsage\n=====\n\ndjango-linear-migrations helps you work on Django projects where several branches adding migrations may be in progress at any time.\nIt enforces that your apps have a *linear* migration history, avoiding merge migrations and the problems they can cause from migrations running in different orders.\nIt does this by making ``makemigrations`` record the name of the latest migration in per-app ``max_migration.txt`` files.\nThese files will then cause a merge conflicts in your source control tool (Git, Mercurial, etc.) in the case of migrations being developed in parallel.\nThe first merged migration for an app will prevent the second from being merged, without addressing the conflict.\nThe included ``rebase-migration`` command can help automatically such conflicts.\n\nSystem Checks\n-------------\n\ndjango-linear-migrations comes with several system checks that verify that your ``max_migration.txt`` files are in sync.\nThese are:\n\n* ``dlm.E001``: ``<app_label>``'s max_migration.txt does not exist.\n* ``dlm.E002``: ``<app_label>``'s max_migration.txt contains multiple lines.\n* ``dlm.E003``: ``<app_label>``'s max_migration.txt points to non-existent migration '``<bad_migration_name>``'.\n* ``dlm.E004``: ``<app_label>``'s max_migration.txt contains '``<max_migration_name>``', but the latest migration is '``<real_max_migration_name>``'.\n\n``rebase-migration`` command\n----------------------------\n\nThis management command can help you fix migration conflicts.\nFollowing a conflicted \u201crebase\u201d operation in your source control tool, run it with the name of the app to auto-fix the migrations for:\n\n.. code-block:: console\n\n    $ python manage.py rebase-migration <app_label>\n\nNote rebasing the migration might not always be the *correct* thing to do.\nIf the migrations in main and feature branches have both affected the same models, rebasing the migration on the end may not make sense.\nHowever, such parallel changes would *normally* cause conflicts in your models files or other parts of the source code as well.\n\nLet's walk through an example using Git, although it should extend to other source control tools.\n\nImagine you were working on your project's ``books`` app in a feature branch called ``titles`` and created a migration called ``0002_longer_titles``.\nMeanwhile a commit has been merged to your ``main`` branch with a *different* 2nd migration for ``books`` called ``0002_author_nicknames``.\nThanks to django-linear-migrations, the ``max_migration.txt`` file will show as conflicted between your feature and main branches.\n\nYou start the fix by reversing your new migration from your local database.\nThis is necessary since it will be renamed after rebasing and seen as unapplied.\nYou do this by switching to the feature branch ``titles`` migrating back to the last common migration:\n\n.. code-block:: console\n\n    $ git switch titles\n    $ python manage.py migrate books 0001\n\nYou then fetch the latest code:\n\n.. code-block:: console\n\n    $ git switch main\n    $ git pull\n    ...\n\nYou then rebase your ``titles`` branch on top of it, for which Git will detect the conflict on ``max_migration.txt``:\n\n.. code-block:: console\n\n    $ git switch titles\n    $ git rebase main\n    Auto-merging books/models.py\n    CONFLICT (content): Merge conflict in books/migrations/max_migration.txt\n    error: could not apply 123456789... Increase Book title length\n    Resolve all conflicts manually, mark them as resolved with\n    \"git add/rm <conflicted_files>\", then run \"git rebase --continue\".\n    You can instead skip this commit: run \"git rebase --skip\".\n    To abort and get back to the state before \"git rebase\", run \"git rebase --abort\".\n    Could not apply 123456789... Increase Book title length\n\nIf you look at the contents of the ``books`` app's ``max_migration.txt`` at this point, it will look something like this:\n\n.. code-block:: console\n\n    $ cat books/migrations/max_migration.txt\n    <<<<<<< HEAD\n    0002_author_nicknames\n    =======\n    0002_longer_titles\n    >>>>>>> 123456789 (Increase Book title length)\n\nIt's at this point you can use ``rebase-migration`` to automatically fix the ``books`` migration history:\n\n.. code-block:: console\n\n    $ python manage.py rebase-migration books\n    Renamed 0002_longer_titles.py to 0003_longer_titles.py, updated its dependencies, and updated max_migration.txt.\n\nThis places the conflicted migration on the end of the migration history.\nIt renames the file appropriately, modifies its ``dependencies = [...]`` declaration, and updates the migration named in ``max_migration.txt`` appropriately.\n\nAfter this, you should be able to continue the rebase:\n\n.. code-block:: console\n\n    $ git add books/migrations\n    $ git rebase --continue\n\nAnd then migrate your local database to allow you to continue development:\n\n.. code-block:: console\n\n    $ python manage.py migrate books\n    Operations to perform:\n      Target specific migration: 0003_longer_titles, from books\n    Running migrations:\n      Applying books.0002_author_nicknames... OK\n      Applying books.0003_longer_titles... OK\n\nInspiration\n===========\n\nI\u2019ve seen similar techniques to the one implemented by django-linear-migrations at several places, and they acted as the inspiration for putting this package together.\nMy previous client `Pollen <https://pollen.co/>`__ and current client `ev.energy <https://ev.energy/>`__ both have implementations.\nThis `Doordash blogpost <https://medium.com/@DoorDash/tips-for-building-high-quality-django-apps-at-scale-a5a25917b2b5>`__ covers a similar system that uses a single file for tracking latest migrations.\nAnd there's also a package called `django-migrations-git-conflicts <https://pypi.org/project/django-migrations-git-conflicts/>`__ which works fairly similarly.\n\n\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Ensure your migrations are linear.",
    "version": "1.6.0",
    "split_keywords": [
        "django"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "76299cba6d76404641aa30dc1e1f4154",
                "sha256": "948ae0bbd198edd3598edd4796a0f5260e644406b075f2395c6ef8612078d203"
            },
            "downloads": -1,
            "filename": "django_linear_migrations-1.6.0-py3-none-any.whl",
            "has_sig": true,
            "md5_digest": "76299cba6d76404641aa30dc1e1f4154",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 16862,
            "upload_time": "2021-04-08T11:39:51",
            "upload_time_iso_8601": "2021-04-08T11:39:51.381451Z",
            "url": "https://files.pythonhosted.org/packages/4c/83/1b8bc43a45d5f1124f33c57df6174a99c4909d0c072674feb2a049505437/django_linear_migrations-1.6.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "00ec30ce1f82accfaeb3e8c126f024c8",
                "sha256": "bea0121d7815003580a835c45008a1eb31ec989ee539972fd62c08698269cc90"
            },
            "downloads": -1,
            "filename": "django-linear-migrations-1.6.0.tar.gz",
            "has_sig": true,
            "md5_digest": "00ec30ce1f82accfaeb3e8c126f024c8",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 19945,
            "upload_time": "2021-04-08T11:39:52",
            "upload_time_iso_8601": "2021-04-08T11:39:52.973844Z",
            "url": "https://files.pythonhosted.org/packages/f2/1b/476819a1cac2fc627f746e09ae8ecd10d65157592a46c00cb994f96ef009/django-linear-migrations-1.6.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2021-04-08 11:39:52",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": null,
    "github_project": "adamchainz",
    "error": "Could not fetch GitHub repository",
    "lcname": "django-linear-migrations"
}
        
Elapsed time: 0.44126s