.. image:: https://travis-ci.org/cocreators-ee/migrate-anything.svg?branch=master
:target: https://travis-ci.org/cocreators-ee/migrate-anything
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
.. image:: https://codecov.io/gh/cocreators-ee/migrate-anything/branch/master/graph/badge.svg
:target: https://codecov.io/gh/cocreators-ee/migrate-anything
.. image:: https://sonarcloud.io/api/project_badges/measure?project=cocreators_migrate-anything&metric=alert_status
:target: https://sonarcloud.io/dashboard?id=cocreators_migrate-anything
.. image:: https://img.shields.io/github/issues/cocreators-ee/migrate-anything
:target: https://github.com/cocreators-ee/migrate-anything/issues
:alt: GitHub issues
.. image:: https://img.shields.io/pypi/dm/migrate-anything
:target: https://pypi.org/project/migrate-anything/
:alt: PyPI - Downloads
.. image:: https://img.shields.io/pypi/v/migrate-anything
:target: https://pypi.org/project/migrate-anything/
:alt: PyPI
.. image:: https://img.shields.io/pypi/pyversions/migrate-anything
:target: https://pypi.org/project/migrate-anything/
:alt: PyPI - Python Version
.. image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg
:target: https://opensource.org/licenses/BSD-3-Clause
Migrate anything - database (etc.) migration utility, especially for Python projects.
What is this?
=============
It's kinda annoying how often you run into the question of how to handle migrations in your project, and there hasn't seem to emerged any good, DB -agnostic, framework-agnostic, and storage-agnostic tool to manage them.
This project is an attempt to change that.
Basically what it does when you run :code:`migrate-anything migrations` is:
1. Find all the files :code:`migrations/*.py` and sort them
2. Any that are not yet registered in the DB will be loaded, their :code:`up()` is executed, and the file's contents stored in the DB
3. Any files that are missing from the fs but are in the DB will have their code loaded from the DB and their :code:`down()` is executed - in reverse order
License
-------
Licensing is important. This project uses BSD 3-clause license, and adds no other dependencies to your project (it does use a few things during build & testing) - that's about as simple, safe, and free to use as it gets.
For more information check the `LICENSE <https://github.com/cocreators-ee/migrate-anything/blob/master/LICENSE>`_ -file.
Usage examples
==============
Basic usage
-----------
Firstly you'll need this package in your project. Pick one of these:
.. code-block:: python
pip install -U migrate-anything
poetry add migrate-anything
pipenv install migrate-anything
Simply put, create a Python package, don't be too clever and call it e.g. ``migrations``. Then put files in that package:
.. code-block:: python
# migrations/__init__.py
from migrate_anything import configure, CSVStorage
configure(storage=CSVStorage("migration_status.csv"))
.. code-block:: python
# migrations/01-initialize-db.py
# Please note that this is built for a completely hypothetical DB layer
from my_db import get_db
DB = get_db()
def up():
DB.create_table("example")
def down():
DB.delete_table("example")
This would configure your migrations' status to be stored in a local file called ``migration_status.csv`` and set up your first migration script. If you have a ``my_db`` module that works like this you could just run this with a single command:
.. code-block:: shell
migrate-anything migrations
poetry run migrate-anything migrations
pipenv run migrate-anything migrations
Now in the real world you might want something more durable and a realistic example, so here's e.g. what you'd do when using MongoDB:
.. code-block:: python
# __init__.py
from migrate_anything import configure, MongoDBStorage
import pymongo
db = pymongo.MongoClient().my_db
configure(storage=MongoDBStorage(db.migrations))
.. code-block:: python
# 01-initialize-db.py
from pymongo import MongoClient
client = MongoClient()
db = client.my_db
def up():
db.posts.insert_one({
"id": "post-1",
"title": "We're live!",
"content": "This is our first post, yay."
})
db.posts.create_index("id")
def down():
db.posts.drop()
This would configure storage to a ``my_db.migrations`` MongoDB collection.
Command line flags
-----------------------
.. code-block:: shell
# Revert the last migration using migration code file.
migrate-anything migrations --revert-latest
Custom storage engines
-----------------------
Writing your own custom storage engine is easy.
.. code-block:: python
# __init__.py
from migrate_anything import configure
class CustomStorage(object):
def __init__(self, file):
self.file = file
def save_migration(self, name, code):
with open(self.file, "a", encoding="utf-8") as file:
file.write("{},{}\n".format(name, code))
def list_migrations(self):
try:
with open(self.file, encoding="utf-8") as file:
return [
line.split(",")
for line in file.readlines()
if line.strip() # Skip empty lines
]
except FileNotFoundError:
return []
def remove_migration(self, name):
migrations = [
migration for migration in self.list_migrations() if migration[0] != name
]
with open(self.file, "w", encoding="utf-8") as file:
for row in migrations:
file.write("{},{}\n".format(*row))
configure(storage=CustomStorage("test.txt"))
You can also check out the `examples <https://github.com/cocreators-ee/migrate-anything/tree/master/examples>`_.
Contributing
============
This project is run on GitHub using the issue tracking and pull requests here. If you want to contribute, feel free to `submit issues <https://github.com/cocreators-ee/migrate-anything/issues>`_ (incl. feature requests) or PRs here.
You will need `pre-commit <https://pre-commit.com/#install>`_ set up to make contributions.
To set up development tools for this, run:
.. code-block:: shell
pre-commit install
virtualenv .venv
.venv/bin/activate
# OR
.venv\Scripts\activate.bat
pip install -r dev-requirements.txt
pip install -e .
And then to run the tests
.. code-block:: shell
pytest
When you have improvements to make, commit (and include any cleanup pre-commit might do), push your changes to your own fork, make a PR.
Future ideas
=================
Future ideas include support for other DB engines (feel free to contribute),
and Kubernetes ConfigMap. Annoyingly storage to Kubernetes from inside a pod
and in code is not quite as simple as just running ``kubectl``.
Oh and your Kubernetes pods will likely require the necessary RBAC rules to manage their ConfigMap. It's unfortunately kinda complex, but I'm sure you can figure it out e.g. with this `guide <https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/>`_.
Financial support
=================
This project has been made possible thanks to `Cocreators <https://cocreators.ee>`_ and `Lietu <https://lietu.net>`_. You can help us continue our open source work by supporting us on `Buy me a coffee <https://www.buymeacoffee.com/cocreators>`_.
.. image:: https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png
:target: https://www.buymeacoffee.com/cocreators
Raw data
{
"_id": null,
"home_page": "https://github.com/cocreators-ee/migrate-anything",
"name": "migrate-anything",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.6",
"maintainer_email": null,
"keywords": "migrate database db release",
"author": "Cocreators O\u00dc",
"author_email": "janne@cocreators.ee",
"download_url": "https://files.pythonhosted.org/packages/9f/9d/aa0a8db8ab8c82be652fe1bcf95f4b94f0c9376e7d1dc04dced482c5ff56/migrate_anything-0.2.0.tar.gz",
"platform": null,
"description": ".. image:: https://travis-ci.org/cocreators-ee/migrate-anything.svg?branch=master\n :target: https://travis-ci.org/cocreators-ee/migrate-anything\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/psf/black\n\n.. image:: https://codecov.io/gh/cocreators-ee/migrate-anything/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/cocreators-ee/migrate-anything\n\n.. image:: https://sonarcloud.io/api/project_badges/measure?project=cocreators_migrate-anything&metric=alert_status\n :target: https://sonarcloud.io/dashboard?id=cocreators_migrate-anything\n\n.. image:: https://img.shields.io/github/issues/cocreators-ee/migrate-anything\n :target: https://github.com/cocreators-ee/migrate-anything/issues\n :alt: GitHub issues\n\n.. image:: https://img.shields.io/pypi/dm/migrate-anything\n :target: https://pypi.org/project/migrate-anything/\n :alt: PyPI - Downloads\n\n.. image:: https://img.shields.io/pypi/v/migrate-anything\n :target: https://pypi.org/project/migrate-anything/\n :alt: PyPI\n\n.. image:: https://img.shields.io/pypi/pyversions/migrate-anything\n :target: https://pypi.org/project/migrate-anything/\n :alt: PyPI - Python Version\n\n.. image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg\n :target: https://opensource.org/licenses/BSD-3-Clause\n\nMigrate anything - database (etc.) migration utility, especially for Python projects.\n\n\nWhat is this?\n=============\n\nIt's kinda annoying how often you run into the question of how to handle migrations in your project, and there hasn't seem to emerged any good, DB -agnostic, framework-agnostic, and storage-agnostic tool to manage them.\n\nThis project is an attempt to change that.\n\nBasically what it does when you run :code:`migrate-anything migrations` is:\n\n1. Find all the files :code:`migrations/*.py` and sort them\n2. Any that are not yet registered in the DB will be loaded, their :code:`up()` is executed, and the file's contents stored in the DB\n3. Any files that are missing from the fs but are in the DB will have their code loaded from the DB and their :code:`down()` is executed - in reverse order\n\n\nLicense\n-------\n\nLicensing is important. This project uses BSD 3-clause license, and adds no other dependencies to your project (it does use a few things during build & testing) - that's about as simple, safe, and free to use as it gets.\n\nFor more information check the `LICENSE <https://github.com/cocreators-ee/migrate-anything/blob/master/LICENSE>`_ -file.\n\n\nUsage examples\n==============\n\nBasic usage\n-----------\n\nFirstly you'll need this package in your project. Pick one of these:\n\n.. code-block:: python\n\n pip install -U migrate-anything\n poetry add migrate-anything\n pipenv install migrate-anything\n\nSimply put, create a Python package, don't be too clever and call it e.g. ``migrations``. Then put files in that package:\n\n.. code-block:: python\n\n # migrations/__init__.py\n from migrate_anything import configure, CSVStorage\n\n configure(storage=CSVStorage(\"migration_status.csv\"))\n\n.. code-block:: python\n\n # migrations/01-initialize-db.py\n # Please note that this is built for a completely hypothetical DB layer\n from my_db import get_db\n\n DB = get_db()\n\n def up():\n DB.create_table(\"example\")\n\n def down():\n DB.delete_table(\"example\")\n\nThis would configure your migrations' status to be stored in a local file called ``migration_status.csv`` and set up your first migration script. If you have a ``my_db`` module that works like this you could just run this with a single command:\n\n.. code-block:: shell\n\n migrate-anything migrations\n poetry run migrate-anything migrations\n pipenv run migrate-anything migrations\n\nNow in the real world you might want something more durable and a realistic example, so here's e.g. what you'd do when using MongoDB:\n\n.. code-block:: python\n\n # __init__.py\n from migrate_anything import configure, MongoDBStorage\n import pymongo\n\n db = pymongo.MongoClient().my_db\n\n configure(storage=MongoDBStorage(db.migrations))\n\n.. code-block:: python\n\n # 01-initialize-db.py\n from pymongo import MongoClient\n\n client = MongoClient()\n db = client.my_db\n\n def up():\n db.posts.insert_one({\n \"id\": \"post-1\",\n \"title\": \"We're live!\",\n \"content\": \"This is our first post, yay.\"\n })\n db.posts.create_index(\"id\")\n\n def down():\n db.posts.drop()\n\nThis would configure storage to a ``my_db.migrations`` MongoDB collection.\n\n\nCommand line flags\n-----------------------\n\n.. code-block:: shell\n\n # Revert the last migration using migration code file.\n migrate-anything migrations --revert-latest\n\n\nCustom storage engines\n-----------------------\n\nWriting your own custom storage engine is easy.\n\n.. code-block:: python\n\n # __init__.py\n from migrate_anything import configure\n\n\n class CustomStorage(object):\n def __init__(self, file):\n self.file = file\n\n def save_migration(self, name, code):\n with open(self.file, \"a\", encoding=\"utf-8\") as file:\n file.write(\"{},{}\\n\".format(name, code))\n\n def list_migrations(self):\n try:\n with open(self.file, encoding=\"utf-8\") as file:\n return [\n line.split(\",\")\n for line in file.readlines()\n if line.strip() # Skip empty lines\n ]\n except FileNotFoundError:\n return []\n\n def remove_migration(self, name):\n migrations = [\n migration for migration in self.list_migrations() if migration[0] != name\n ]\n\n with open(self.file, \"w\", encoding=\"utf-8\") as file:\n for row in migrations:\n file.write(\"{},{}\\n\".format(*row))\n\n\n configure(storage=CustomStorage(\"test.txt\"))\n\nYou can also check out the `examples <https://github.com/cocreators-ee/migrate-anything/tree/master/examples>`_.\n\n\nContributing\n============\n\nThis project is run on GitHub using the issue tracking and pull requests here. If you want to contribute, feel free to `submit issues <https://github.com/cocreators-ee/migrate-anything/issues>`_ (incl. feature requests) or PRs here.\n\nYou will need `pre-commit <https://pre-commit.com/#install>`_ set up to make contributions.\n\nTo set up development tools for this, run:\n\n.. code-block:: shell\n\n pre-commit install\n virtualenv .venv\n\n .venv/bin/activate\n # OR\n .venv\\Scripts\\activate.bat\n\n pip install -r dev-requirements.txt\n pip install -e .\n\nAnd then to run the tests\n\n.. code-block:: shell\n\n pytest\n\nWhen you have improvements to make, commit (and include any cleanup pre-commit might do), push your changes to your own fork, make a PR.\n\n\nFuture ideas\n=================\n\nFuture ideas include support for other DB engines (feel free to contribute),\nand Kubernetes ConfigMap. Annoyingly storage to Kubernetes from inside a pod\nand in code is not quite as simple as just running ``kubectl``.\n\nOh and your Kubernetes pods will likely require the necessary RBAC rules to manage their ConfigMap. It's unfortunately kinda complex, but I'm sure you can figure it out e.g. with this `guide <https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/>`_.\n\n\nFinancial support\n=================\n\nThis project has been made possible thanks to `Cocreators <https://cocreators.ee>`_ and `Lietu <https://lietu.net>`_. You can help us continue our open source work by supporting us on `Buy me a coffee <https://www.buymeacoffee.com/cocreators>`_.\n\n.. image:: https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png\n :target: https://www.buymeacoffee.com/cocreators\n",
"bugtrack_url": null,
"license": null,
"summary": "Helps manage migrations for databases and anything else",
"version": "0.2.0",
"project_urls": {
"Bug Reports": "https://github.com/cocreators-ee/migrate-anything/issues",
"Homepage": "https://github.com/cocreators-ee/migrate-anything",
"Source": "https://github.com/cocreators-ee/migrate-anything/"
},
"split_keywords": [
"migrate",
"database",
"db",
"release"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "31915daaaff4f245731faf8fc3d027a67a44ebd24222beeec036522adc0049be",
"md5": "a47b18005daa73394ddf62b26db2bd9b",
"sha256": "6b28e49533116a3f4c210d227f60bdf205b19b15145e81116faf1eb489b76264"
},
"downloads": -1,
"filename": "migrate_anything-0.2.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "a47b18005daa73394ddf62b26db2bd9b",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": "<4,>=3.6",
"size": 10126,
"upload_time": "2025-09-05T17:16:07",
"upload_time_iso_8601": "2025-09-05T17:16:07.906941Z",
"url": "https://files.pythonhosted.org/packages/31/91/5daaaff4f245731faf8fc3d027a67a44ebd24222beeec036522adc0049be/migrate_anything-0.2.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9f9daa0a8db8ab8c82be652fe1bcf95f4b94f0c9376e7d1dc04dced482c5ff56",
"md5": "9b7a32b66a1a77a3d9793a23ac6dd625",
"sha256": "8072b3e3e406294368dcf68db5ffcfdbe238b0efe4a6aa17d9de2f48d91e9959"
},
"downloads": -1,
"filename": "migrate_anything-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "9b7a32b66a1a77a3d9793a23ac6dd625",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.6",
"size": 11641,
"upload_time": "2025-09-05T17:16:09",
"upload_time_iso_8601": "2025-09-05T17:16:09.039357Z",
"url": "https://files.pythonhosted.org/packages/9f/9d/aa0a8db8ab8c82be652fe1bcf95f4b94f0c9376e7d1dc04dced482c5ff56/migrate_anything-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-05 17:16:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cocreators-ee",
"github_project": "migrate-anything",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "migrate-anything"
}
JSON
