# Educationwarehouse's Migrate
[](https://pypi.org/project/edwh-migrate)
[](https://pypi.org/project/edwh-migrate)
-----
**Table of Contents**
- [Installation](#installation)
- [Documentation](#documentation)
- [License](#license)
## Installation
```console
pip install edwh-migrate
# or to include extra dependencies (psycopg2, redis):
pip install edwh-migrate[full]
```
## Documentation
### Config: Environment variables
These variables can be set in the current environment or via `.env`:
* `MIGRATE_URI` (required): regular `postgres://user:password@host:port/database` or `sqlite:///path/to/database` URI
* `DATABASE_TO_RESTORE`: path to a (compressed) SQL file to restore. `.xz`,`.gz` and `.sql` are supported.
* `MIGRATE_CAT_COMMAND`: for unsupported compression formats, this command decompresses the file and produces sql on the
stdout.
* `SCHEMA_VERSION`: Used in case of schema versioning. Set by another process.
* `REDIS_HOST`: If set, all keys of the redis database 0 will be removed.
* `MIGRATE_TABLE`: name of the table where installed migrations are stored. Defaults to `ewh_implemented_features`.
* `FLAG_LOCATION`: when using schema versioned lock files, this directory is used to store the flags. Defaults to `/flags`.
* `CREATE_FLAG_LOCATION` (bool): should the directory above be created if it does not exist yet? Defaults to 0 (false).
* `SCHEMA`: (for postgres) set the default namespace (`search_path`). Defaults to `public`.
* `USE_TYPEDAL`: pass a TypeDAL instance to migrations instead of a regular pyDAL.
### Config: pyproject.toml
You can also set your config variables via the `[tool.migrate]` key in `pyproject.toml`.
First, these variables are loaded and then updated with variables from the environment.
This way, you can set static variables (the ones you want in git, e.g. the `migrate_table` name or path to the backup to
restore) in the toml, and keep private/dynamic vars in the environment (e.g. the database uri or schema version).
Example:
```toml
[tool.migrate]
migrate_uri = "" # filled in by .env
database-to-restore = "migrate/data/db_backup.sql"
# ...
```
### Creating a `migrations.py`
```python
from edwh_migrate import migration
@migration
def feature_1(db):
print("feature_1")
return True
@migration(requires=[feature_1]) # optional `requires` ensures previous migration(s) are installed
def functionalname_date_sequencenr(db: pydal.DAL):
db.executesql("""
CREATE TABLE ...
""")
db.commit()
return True
```
### Usage
When your configuration is set up properly and you have a file containing your migrations, you can simply run:
```bash
migrate
# or, to use a different name than migrations.py:
migrate path/to/my/migrate_file.py
```
## Advanced Topics
### Using `ViewMigrationManager` via Subclasses
`ViewMigrationManager` is designed to manage the lifecycle of view migrations in a database using context management. It ensures that migrations are properly handled with dependencies between different migrations.
#### Usage
1. **Define Subclasses**: Create subclasses of `ViewMigrationManager` and implement the required methods `up` and `down`.
```python
from edwh_migrate import ViewMigrationManager
class MyExampleView_V1(ViewMigrationManager):
# Define dependencies (optional)
uses = ()
# Specify a migration that must have run before this class may be used
since = "previous_migration"
def up(self):
# Logic to apply the migration
self.db.executesql(
'''
CREATE MATERIALIZED VIEW my_example_view AS
SELECT id, name FROM my_table;
'''
)
def down(self):
# Logic to reverse the migration
self.db.executesql(
'''
DROP MATERIALIZED VIEW IF EXISTS my_example_view;
'''
)
class AnotherExampleView(ViewMigrationManager):
# This class depends on MyExampleView_V1
uses = (MyExampleView_V1,)
def up(self):
# Logic to apply the migration
self.db.executesql(
'''
CREATE MATERIALIZED VIEW another_example_view AS
SELECT id, name FROM my_example_view;
'''
)
def down(self):
# Logic to reverse the migration
self.db.executesql(
'''
DROP MATERIALIZED VIEW IF EXISTS another_example_view;
'''
)
```
2. **Define the `previous_migration`**: Create a migration function that serves as the prerequisite for `MyExampleView_V1`.
```python
from edwh_migrate import migration
@migration
def previous_migration(db):
db.executesql('''
CREATE TABLE my_table (
id SERIAL PRIMARY KEY,
name VARCHAR(255)
);
''')
db.commit()
return True
```
3. **Use the Subclass in a Migration Function**: Utilize the subclass in a migration function to manage the view migration context.
```python
from edwh_migrate import migration
@migration
def upgrade_some_source_table_that_my_example_view_depends_on(db):
with MyExampleView_V1(db):
db.executesql('''
ALTER TABLE my_table
ADD COLUMN new_column VARCHAR(255);
''')
db.commit()
return True
```
In the example above:
- `MyExampleView_V1` is a subclass of `ViewMigrationManager` that manages the lifecycle of a materialized view named `my_example_view`.
- `AnotherExampleView` is another subclass that depends on `MyExampleView_V1` and manages the lifecycle of another materialized view named `another_example_view`.
- The `up` method in `MyExampleView_V1` contains the logic to create the materialized view `my_example_view`.
- The `down` method in `MyExampleView_V1` contains the logic to drop the materialized view `my_example_view`.
- The `up` method in `AnotherExampleView` contains the logic to create the materialized view `another_example_view` that references `my_example_view`.
- The `down` method in `AnotherExampleView` contains the logic to drop the materialized view `another_example_view`.
- The `since` attribute specifies that a particular migration (`previous_migration`) must have run before `MyExampleView_V1` may be used.
- The `previous_migration` function creates the table `my_table` and serves as a prerequisite for `MyExampleView_V1`.
- The `migration` decorator is used to define a migration function (`upgrade_some_source_table_that_my_example_view_depends_on`) that executes within the context of `MyExampleView_V1`.
- The `with MyExampleView_V1(db)` block ensures that the `down` method is called before the block executes and the `up` method is called after the block completes.
In addition to 'since', the inverse 'until' can also be used.
## License
`edwh-migrate` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.
Raw data
{
"_id": null,
"home_page": null,
"name": "edwh-migrate",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "database-migration, migrate, postgresql, pydal, schema-change",
"author": null,
"author_email": "Remco <remco@educationwarehouse.nl>, Robin <robin@educationwarehouse.nl>",
"download_url": "https://files.pythonhosted.org/packages/55/2c/cd1dc6d04bc9f84dfad2e2e06698faa94cf4fa952d81acfadd323ed2e695/edwh_migrate-0.10.4.tar.gz",
"platform": null,
"description": "# Educationwarehouse's Migrate\n\n[](https://pypi.org/project/edwh-migrate)\n[](https://pypi.org/project/edwh-migrate)\n\n-----\n\n**Table of Contents**\n\n- [Installation](#installation)\n- [Documentation](#documentation)\n- [License](#license)\n\n## Installation\n\n```console\npip install edwh-migrate\n# or to include extra dependencies (psycopg2, redis):\npip install edwh-migrate[full]\n```\n\n## Documentation\n\n### Config: Environment variables\n\nThese variables can be set in the current environment or via `.env`:\n\n* `MIGRATE_URI` (required): regular `postgres://user:password@host:port/database` or `sqlite:///path/to/database` URI\n* `DATABASE_TO_RESTORE`: path to a (compressed) SQL file to restore. `.xz`,`.gz` and `.sql` are supported.\n* `MIGRATE_CAT_COMMAND`: for unsupported compression formats, this command decompresses the file and produces sql on the\n stdout.\n* `SCHEMA_VERSION`: Used in case of schema versioning. Set by another process.\n* `REDIS_HOST`: If set, all keys of the redis database 0 will be removed.\n* `MIGRATE_TABLE`: name of the table where installed migrations are stored. Defaults to `ewh_implemented_features`. \n* `FLAG_LOCATION`: when using schema versioned lock files, this directory is used to store the flags. Defaults to `/flags`.\n* `CREATE_FLAG_LOCATION` (bool): should the directory above be created if it does not exist yet? Defaults to 0 (false). \n* `SCHEMA`: (for postgres) set the default namespace (`search_path`). Defaults to `public`.\n* `USE_TYPEDAL`: pass a TypeDAL instance to migrations instead of a regular pyDAL.\n\n### Config: pyproject.toml\n\nYou can also set your config variables via the `[tool.migrate]` key in `pyproject.toml`.\nFirst, these variables are loaded and then updated with variables from the environment.\nThis way, you can set static variables (the ones you want in git, e.g. the `migrate_table` name or path to the backup to\nrestore) in the toml, and keep private/dynamic vars in the environment (e.g. the database uri or schema version).\n\nExample:\n\n```toml\n[tool.migrate]\nmigrate_uri = \"\" # filled in by .env\ndatabase-to-restore = \"migrate/data/db_backup.sql\"\n# ...\n```\n\n### Creating a `migrations.py`\n\n```python\nfrom edwh_migrate import migration\n\n@migration\ndef feature_1(db):\n print(\"feature_1\")\n return True\n\n\n@migration(requires=[feature_1]) # optional `requires` ensures previous migration(s) are installed\ndef functionalname_date_sequencenr(db: pydal.DAL):\n db.executesql(\"\"\"\n CREATE TABLE ...\n \"\"\")\n db.commit()\n return True\n\n```\n\n### Usage\n\nWhen your configuration is set up properly and you have a file containing your migrations, you can simply run:\n\n```bash\nmigrate\n# or, to use a different name than migrations.py:\nmigrate path/to/my/migrate_file.py\n```\n\n## Advanced Topics\n\n### Using `ViewMigrationManager` via Subclasses\n\n`ViewMigrationManager` is designed to manage the lifecycle of view migrations in a database using context management. It ensures that migrations are properly handled with dependencies between different migrations.\n\n#### Usage\n\n1. **Define Subclasses**: Create subclasses of `ViewMigrationManager` and implement the required methods `up` and `down`.\n\n ```python\n from edwh_migrate import ViewMigrationManager\n\n class MyExampleView_V1(ViewMigrationManager):\n # Define dependencies (optional)\n uses = ()\n # Specify a migration that must have run before this class may be used\n since = \"previous_migration\"\n\n def up(self):\n # Logic to apply the migration\n self.db.executesql(\n '''\n CREATE MATERIALIZED VIEW my_example_view AS\n SELECT id, name FROM my_table;\n '''\n )\n\n def down(self):\n # Logic to reverse the migration\n self.db.executesql(\n '''\n DROP MATERIALIZED VIEW IF EXISTS my_example_view;\n '''\n )\n\n class AnotherExampleView(ViewMigrationManager):\n # This class depends on MyExampleView_V1\n uses = (MyExampleView_V1,)\n\n def up(self):\n # Logic to apply the migration\n self.db.executesql(\n '''\n CREATE MATERIALIZED VIEW another_example_view AS\n SELECT id, name FROM my_example_view;\n '''\n )\n\n def down(self):\n # Logic to reverse the migration\n self.db.executesql(\n '''\n DROP MATERIALIZED VIEW IF EXISTS another_example_view;\n '''\n )\n ```\n\n2. **Define the `previous_migration`**: Create a migration function that serves as the prerequisite for `MyExampleView_V1`.\n\n ```python\n from edwh_migrate import migration\n\n @migration\n def previous_migration(db):\n db.executesql('''\n CREATE TABLE my_table (\n id SERIAL PRIMARY KEY,\n name VARCHAR(255)\n );\n ''')\n db.commit()\n return True\n ```\n\n3. **Use the Subclass in a Migration Function**: Utilize the subclass in a migration function to manage the view migration context.\n\n ```python\n from edwh_migrate import migration\n\n @migration\n def upgrade_some_source_table_that_my_example_view_depends_on(db):\n with MyExampleView_V1(db):\n db.executesql('''\n ALTER TABLE my_table\n ADD COLUMN new_column VARCHAR(255);\n ''')\n db.commit()\n return True\n ```\n\nIn the example above:\n- `MyExampleView_V1` is a subclass of `ViewMigrationManager` that manages the lifecycle of a materialized view named `my_example_view`.\n- `AnotherExampleView` is another subclass that depends on `MyExampleView_V1` and manages the lifecycle of another materialized view named `another_example_view`.\n- The `up` method in `MyExampleView_V1` contains the logic to create the materialized view `my_example_view`.\n- The `down` method in `MyExampleView_V1` contains the logic to drop the materialized view `my_example_view`.\n- The `up` method in `AnotherExampleView` contains the logic to create the materialized view `another_example_view` that references `my_example_view`.\n- The `down` method in `AnotherExampleView` contains the logic to drop the materialized view `another_example_view`.\n- The `since` attribute specifies that a particular migration (`previous_migration`) must have run before `MyExampleView_V1` may be used.\n- The `previous_migration` function creates the table `my_table` and serves as a prerequisite for `MyExampleView_V1`.\n- The `migration` decorator is used to define a migration function (`upgrade_some_source_table_that_my_example_view_depends_on`) that executes within the context of `MyExampleView_V1`.\n- The `with MyExampleView_V1(db)` block ensures that the `down` method is called before the block executes and the `up` method is called after the block completes.\n\nIn addition to 'since', the inverse 'until' can also be used.\n\n## License\n\n`edwh-migrate` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.\n",
"bugtrack_url": null,
"license": null,
"summary": "Helps migrating database schema changes using pydal. ",
"version": "0.10.4",
"project_urls": {
"Documentation": "https://github.com/educationwarehouse/migrate#readme",
"Issues": "https://github.com/educationwarehouse/migrate/issues",
"Source": "https://github.com/educationwarehouse/migrate"
},
"split_keywords": [
"database-migration",
" migrate",
" postgresql",
" pydal",
" schema-change"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0d1e8b9dc0b99007469950e96a4c201000a38fdec31b73da1eac9ff0c7f34e94",
"md5": "153dc0c7fa78b1520a6ee26d05ebc32d",
"sha256": "7256d61fe3c208a40e851f3f355d143f2ea79d7ce4e7849b55f21eb0d70b5edb"
},
"downloads": -1,
"filename": "edwh_migrate-0.10.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "153dc0c7fa78b1520a6ee26d05ebc32d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 18301,
"upload_time": "2025-01-10T10:13:39",
"upload_time_iso_8601": "2025-01-10T10:13:39.517047Z",
"url": "https://files.pythonhosted.org/packages/0d/1e/8b9dc0b99007469950e96a4c201000a38fdec31b73da1eac9ff0c7f34e94/edwh_migrate-0.10.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "552ccd1dc6d04bc9f84dfad2e2e06698faa94cf4fa952d81acfadd323ed2e695",
"md5": "68617ded35cdabd20f9555b5a6cb60c0",
"sha256": "48294799b5e85606850d27145686ccac97f4a5ddc981d0e2e8559209e27e23fd"
},
"downloads": -1,
"filename": "edwh_migrate-0.10.4.tar.gz",
"has_sig": false,
"md5_digest": "68617ded35cdabd20f9555b5a6cb60c0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 29661,
"upload_time": "2025-01-10T10:13:35",
"upload_time_iso_8601": "2025-01-10T10:13:35.869239Z",
"url": "https://files.pythonhosted.org/packages/55/2c/cd1dc6d04bc9f84dfad2e2e06698faa94cf4fa952d81acfadd323ed2e695/edwh_migrate-0.10.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-10 10:13:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "educationwarehouse",
"github_project": "migrate#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "edwh-migrate"
}
JSON
