| Name | django-litestream JSON |
| Version |
0.3.0
JSON |
| download |
| home_page | None |
| Summary | Django integration with litestream, a a standalone streaming replication tool for SQLite. |
| upload_time | 2024-08-19 17:41:56 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License
Copyright (c) 2024-present Tobi DEGNON <tobidegnon@proton.me>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
backup
django
litestream
replication
sqlite
sqlite3
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# django-litestream
[](https://pypi.org/project/django-litestream)
[](https://pypi.org/project/django-litestream)
-----
> [!IMPORTANT]
> This package currently contains minimal features and is a work-in-progress
This package installs and integrates [litestream](https://litestream.io), the SQLite replication tool, as a Django command.
## Table of Contents
- [django-litestream](#django-litestream)
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Usage](#usage)
- [Configuration](#configuration)
- [Commands](#commands)
- [litestream init](#litestream-init)
- [litestream databases](#litestream-databases)
- [litestream generations](#litestream-generations)
- [litestream replicate](#litestream-replicate)
- [litestream restore](#litestream-restore)
- [litestream verify](#litestream-verify)
- [litestream snapshots](#litestream-snapshots)
- [litestream wal](#litestream-wal)
- [litestream version](#litestream-version)
- [License](#license)
## Installation
```console
pip install django-litestream
```
Add `django_litestream` to your Django `INSTALLED_APPS`.
## Usage
The package integrates all the commands and options from the `litestream` command-line tool with only minor changes.
> [!Note]
> Django 5.1 was released a few days ago (as of the time of writing). If you are
> looking for a good production configuration for SQLite, check out [this blog post](https://blog.pecar.me/sqlite-django-config#in-django-51-or-newer).
### Configuration
These are the available configurations for `django-litestream`:
```python
# settings.py
LITESTREAM = {
"config_file": "/etc/litestream.yml",
"path_prefix": None,
"bin_path": "litestream",
"dbs": [],
"extend_dbs": [],
"logging": {},
"addr": "",
}
```
The **config_file** is where the Litestream configuration file should be generated by the [init command](#litestream-init).
The **config_file** will be automatically passed to every command you run, so you can freely change the default location
without having to pass the `-config` argument manually each time.
For example, you could place it in your project directory:
```python
# settings.py
LITESTREAM = {
"config_file": BASE_DIR / "litestream.yml",
...
}
```
The **path_prefix** is a string that will be prepended to the path of every database in the `dbs` configuration. This is useful if you are replicating databases from different projects to the same bucket, you could set the `path_prefix` to the project name so that the databases are stored in different folders in the bucket.
The **bin_path** is the path to the Litestream binary. If you want to use a custom installation, specify it here.
The **dbs**, **logging**, and **addr** configurations are the same as those in the Litestream configuration file.
You can read more about them [here](https://litestream.io/reference/config/#database-settings).
This allows you to keep your litestream configuration in your Django settings.
The **extend_dbs** is a list of dictionaries with the same format as the `dbs` configuration, and,
as its name suggests, it will extend the `dbs` configuration when the final configuration is generated.
### Commands
You can run `python manage.py litestream` to see all available commands.
> [!Note]
> Wherever it says `dj`, assume it is an alias for `python manage.py`.
#### litestream init
```console
dj litestream init
```
This command will write the Litestream configuration to the indicated **config_file** based on your settings.
If you did not specify any values in the **dbs** key, it will automatically parse your Django `DATABASES` configuration
and write one `s3` replica for each SQLite database it finds.
For example, if you have the following `DATABASES` configuration:
```python
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "db.sqlite3",
},
"other": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "other.sqlite3",
},
}
```
And your `BASE_DIR` is `/home/tobi/myproject`, the generated configuration after running `init` will look like this:
```yaml
dbs:
- path: /home/tobi/myproject/db.sqlite3
replicas:
- type: s3
bucket: $LITESTREAM_REPLICA_BUCKET
path: db.sqlite3
access-key-id: $LITESTREAM_ACCESS_KEY_ID
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
- path: /home/tobi/myproject/other.sqlite3
replicas:
- type: s3
bucket: $LITESTREAM_REPLICA_BUCKET
path: other.sqlite3
access-key-id: $LITESTREAM_ACCESS_KEY_ID
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
```
You can tweak these settings according to your preferences. Check the [databases settings reference](https://litestream.io/reference/config/#database-settings) for more information.
If you have any entries in the **dbs** configuration, the `init` command won’t automatically parse the `DATABASES` configuration.
To extend the configuration generated by the `init` command, you should use the **extend_dbs** configuration, for example:
```python
# settings.py
LITESTREAM = {
"config_file": BASE_DIR / "litestream.yml",
"extend_dbs": [
{
"path": BASE_DIR / "cache.sqlite3",
"replicas": [
{
"type": "s3",
"bucket": "$LITESTREAM_REPLICA_BUCKET",
"path": "cache.sqlite3",
"access-key-id": "$LITESTREAM_ACCESS_KEY_ID",
"secret-access-key": "$LITESTREAM_SECRET_ACCESS_KEY",
}
]
}
]
}
```
You can omit the `access-key-id` and `secret-access-key` keys and litestream will automatically use any of the environment variables below if available:
- `AWS_ACCESS_KEY_ID` or `LITESTREAM_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY` or `LITESTREAM_SECRET_ACCESS_KEY`
#### litestream databases
This works exactly like the equivalent [litestream command](https://litestream.io/reference/databases/) and lists all the databases.
Examples:
```console
dj litestream databases
```
> [!IMPORTANT]
> For the rest of the commands, wherever you are asked to specify the database path `db_path`,
> you can use the Django database alias instead, for example, `default` instead of `/home/tobi/myproject/db.sqlite3`.
#### litestream generations
This works exactly like the equivalent [litestream command](https://litestream.io/reference/generations/).
Examples:
```console
dj litestream generations default
dj litestream generations -replica s3 default
```
#### litestream replicate
This works exactly like the equivalent [litestream command](https://litestream.io/reference/replicate/), except it does not support the ability to replicate a single file.
Running `litestream replicate db_path replica_url` won't work. You can only run:
```console
dj litestream replicate
dj litestream replicate -exec "gunicorn myproject.wsgi:application"
```
This is the command you will run in production using a process manager as its own process. You can run it separately (the first example) or
use it to run your main Django process (second example). It would basically act as a process manager itself and run both the replicate
and the Django process. The replication process will exit when your web server shuts down.
#### litestream restore
This works exactly like the equivalent [litestream command](https://litestream.io/reference/restore/).
Examples:
```console
dj litestream restore default
dj litestream restore -if-replica-exists default
```
#### litestream verify
This command verifies the integrity of your backed-up databases. This process is inspired by the [verify command](https://github.com/fractaledmind/litestream-ruby?tab=readme-ov-file#verification) of the `litestream-ruby` gem.
The verification process involves the following steps:
1. **Add Verification Data**: A new row is added to a `_litestream_verification` table in the specified database. This table is created if it does not already exist. The row contains a unique code and the current timestamp.
2. **Wait for Replication**: The command waits for 10 seconds to allow Litestream to replicate the new row to the configured storage providers.
3. **Restore Backup**: The latest backup is restored from the storage provider to a temporary location.
4. **Check Verification Data**: The restored database is checked to ensure that the verification row is present. This ensures that the backup is both restorable and up-to-date.
If the verification row is not found in the restored database, the command will return an error indicating that the backup data is out of sync. If the row is found, the command confirms that the backup data is in sync.
Examples:
```console
dj litestream verify default
```
This check ensures that the restored database file Exists, can be opened by SQLite, and has up-to-date data.
#### litestream snapshots
This works exactly like the equivalent [litestream command](https://litestream.io/reference/snapshots/).
Examples:
```console
dj litestream snapshots default
dj litestream snapshots -replica s3 default
```
#### litestream wal
This works exactly like the equivalent [litestream command](https://litestream.io/reference/wal/).
Examples:
```console
dj litestream wal default
dj litestream wal -replica s3 default
```
#### litestream version
Print the version of the Litestream binary.
```console
dj litestream version
```
## License
`django-litestream` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.
Raw data
{
"_id": null,
"home_page": null,
"name": "django-litestream",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "backup, django, litestream, replication, sqlite, sqlite3",
"author": null,
"author_email": "Tobi DEGNON <tobidegnon@proton.me>",
"download_url": "https://files.pythonhosted.org/packages/b6/50/b178b640ceb6a6aba5734a66c6bea8d1e3951ad05aa169fbfe22d4d23fdc/django_litestream-0.3.0.tar.gz",
"platform": null,
"description": "# django-litestream\n\n[](https://pypi.org/project/django-litestream)\n[](https://pypi.org/project/django-litestream)\n\n-----\n\n> [!IMPORTANT]\n> This package currently contains minimal features and is a work-in-progress\n\nThis package installs and integrates [litestream](https://litestream.io), the SQLite replication tool, as a Django command.\n\n## Table of Contents\n\n- [django-litestream](#django-litestream)\n - [Table of Contents](#table-of-contents)\n - [Installation](#installation)\n - [Usage](#usage)\n - [Configuration](#configuration)\n - [Commands](#commands)\n - [litestream init](#litestream-init)\n - [litestream databases](#litestream-databases)\n - [litestream generations](#litestream-generations)\n - [litestream replicate](#litestream-replicate)\n - [litestream restore](#litestream-restore)\n - [litestream verify](#litestream-verify)\n - [litestream snapshots](#litestream-snapshots)\n - [litestream wal](#litestream-wal)\n - [litestream version](#litestream-version)\n - [License](#license)\n\n## Installation\n\n```console\npip install django-litestream\n```\n\nAdd `django_litestream` to your Django `INSTALLED_APPS`.\n\n## Usage\n\nThe package integrates all the commands and options from the `litestream` command-line tool with only minor changes.\n\n> [!Note]\n> Django 5.1 was released a few days ago (as of the time of writing). If you are \n> looking for a good production configuration for SQLite, check out [this blog post](https://blog.pecar.me/sqlite-django-config#in-django-51-or-newer).\n\n### Configuration\n\nThese are the available configurations for `django-litestream`:\n\n```python\n# settings.py\nLITESTREAM = {\n \"config_file\": \"/etc/litestream.yml\",\n \"path_prefix\": None,\n \"bin_path\": \"litestream\",\n \"dbs\": [],\n \"extend_dbs\": [],\n \"logging\": {},\n \"addr\": \"\",\n}\n```\n\nThe **config_file** is where the Litestream configuration file should be generated by the [init command](#litestream-init).\nThe **config_file** will be automatically passed to every command you run, so you can freely change the default location \nwithout having to pass the `-config` argument manually each time. \nFor example, you could place it in your project directory:\n\n```python\n# settings.py\nLITESTREAM = {\n \"config_file\": BASE_DIR / \"litestream.yml\",\n ...\n}\n```\n\nThe **path_prefix** is a string that will be prepended to the path of every database in the `dbs` configuration. This is useful if you are replicating databases from different projects to the same bucket, you could set the `path_prefix` to the project name so that the databases are stored in different folders in the bucket.\n\nThe **bin_path** is the path to the Litestream binary. If you want to use a custom installation, specify it here.\n\nThe **dbs**, **logging**, and **addr** configurations are the same as those in the Litestream configuration file. \nYou can read more about them [here](https://litestream.io/reference/config/#database-settings). \nThis allows you to keep your litestream configuration in your Django settings.\n\nThe **extend_dbs** is a list of dictionaries with the same format as the `dbs` configuration, and, \nas its name suggests, it will extend the `dbs` configuration when the final configuration is generated.\n\n### Commands\n\nYou can run `python manage.py litestream` to see all available commands.\n\n> [!Note]\n> Wherever it says `dj`, assume it is an alias for `python manage.py`.\n\n#### litestream init\n\n```console\ndj litestream init\n```\n\nThis command will write the Litestream configuration to the indicated **config_file** based on your settings. \nIf you did not specify any values in the **dbs** key, it will automatically parse your Django `DATABASES` configuration \nand write one `s3` replica for each SQLite database it finds. \n\nFor example, if you have the following `DATABASES` configuration:\n\n```python\nDATABASES = {\n \"default\": {\n \"ENGINE\": \"django.db.backends.sqlite3\",\n \"NAME\": BASE_DIR / \"db.sqlite3\",\n },\n \"other\": {\n \"ENGINE\": \"django.db.backends.sqlite3\",\n \"NAME\": BASE_DIR / \"other.sqlite3\",\n },\n}\n```\n\nAnd your `BASE_DIR` is `/home/tobi/myproject`, the generated configuration after running `init` will look like this:\n\n```yaml\ndbs:\n- path: /home/tobi/myproject/db.sqlite3\n replicas:\n - type: s3\n bucket: $LITESTREAM_REPLICA_BUCKET\n path: db.sqlite3\n access-key-id: $LITESTREAM_ACCESS_KEY_ID\n secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY\n- path: /home/tobi/myproject/other.sqlite3\n replicas:\n - type: s3\n bucket: $LITESTREAM_REPLICA_BUCKET\n path: other.sqlite3\n access-key-id: $LITESTREAM_ACCESS_KEY_ID\n secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY\n```\n\nYou can tweak these settings according to your preferences. Check the [databases settings reference](https://litestream.io/reference/config/#database-settings) for more information.\n\nIf you have any entries in the **dbs** configuration, the `init` command won\u2019t automatically parse the `DATABASES` configuration.\nTo extend the configuration generated by the `init` command, you should use the **extend_dbs** configuration, for example:\n\n```python\n# settings.py\nLITESTREAM = {\n \"config_file\": BASE_DIR / \"litestream.yml\",\n \"extend_dbs\": [\n {\n \"path\": BASE_DIR / \"cache.sqlite3\",\n \"replicas\": [\n {\n \"type\": \"s3\",\n \"bucket\": \"$LITESTREAM_REPLICA_BUCKET\",\n \"path\": \"cache.sqlite3\",\n \"access-key-id\": \"$LITESTREAM_ACCESS_KEY_ID\",\n \"secret-access-key\": \"$LITESTREAM_SECRET_ACCESS_KEY\",\n }\n ]\n }\n ]\n}\n```\n\nYou can omit the `access-key-id` and `secret-access-key` keys and litestream will automatically use any of the environment variables below if available:\n\n- `AWS_ACCESS_KEY_ID` or `LITESTREAM_ACCESS_KEY_ID`\n- `AWS_SECRET_ACCESS_KEY` or `LITESTREAM_SECRET_ACCESS_KEY`\n\n\n#### litestream databases\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/databases/) and lists all the databases.\n\nExamples:\n\n```console\ndj litestream databases\n```\n\n> [!IMPORTANT]\n> For the rest of the commands, wherever you are asked to specify the database path `db_path`, \n> you can use the Django database alias instead, for example, `default` instead of `/home/tobi/myproject/db.sqlite3`.\n\n#### litestream generations\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/generations/).\n\nExamples:\n\n```console\ndj litestream generations default\ndj litestream generations -replica s3 default\n```\n\n#### litestream replicate\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/replicate/), except it does not support the ability to replicate a single file. \nRunning `litestream replicate db_path replica_url` won't work. You can only run:\n\n```console\ndj litestream replicate \ndj litestream replicate -exec \"gunicorn myproject.wsgi:application\"\n```\n\nThis is the command you will run in production using a process manager as its own process. You can run it separately (the first example) or \nuse it to run your main Django process (second example). It would basically act as a process manager itself and run both the replicate \nand the Django process. The replication process will exit when your web server shuts down.\n\n#### litestream restore\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/restore/).\n\nExamples:\n\n```console\ndj litestream restore default\ndj litestream restore -if-replica-exists default\n```\n\n#### litestream verify\n\nThis command verifies the integrity of your backed-up databases. This process is inspired by the [verify command](https://github.com/fractaledmind/litestream-ruby?tab=readme-ov-file#verification) of the `litestream-ruby` gem.\nThe verification process involves the following steps:\n\n1. **Add Verification Data**: A new row is added to a `_litestream_verification` table in the specified database. This table is created if it does not already exist. The row contains a unique code and the current timestamp.\n2. **Wait for Replication**: The command waits for 10 seconds to allow Litestream to replicate the new row to the configured storage providers.\n3. **Restore Backup**: The latest backup is restored from the storage provider to a temporary location.\n4. **Check Verification Data**: The restored database is checked to ensure that the verification row is present. This ensures that the backup is both restorable and up-to-date.\n\nIf the verification row is not found in the restored database, the command will return an error indicating that the backup data is out of sync. If the row is found, the command confirms that the backup data is in sync.\n\nExamples:\n\n```console\ndj litestream verify default\n```\n\nThis check ensures that the restored database file Exists, can be opened by SQLite, and has up-to-date data.\n\n#### litestream snapshots\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/snapshots/). \n\nExamples:\n\n```console\ndj litestream snapshots default\ndj litestream snapshots -replica s3 default\n```\n\n#### litestream wal\n\nThis works exactly like the equivalent [litestream command](https://litestream.io/reference/wal/).\n\nExamples:\n\n```console\ndj litestream wal default\ndj litestream wal -replica s3 default\n```\n\n#### litestream version\n\nPrint the version of the Litestream binary.\n\n```console\ndj litestream version\n```\n\n## License\n\n`django-litestream` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2024-present Tobi DEGNON <tobidegnon@proton.me>\n \n Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Django integration with litestream, a a standalone streaming replication tool for SQLite.",
"version": "0.3.0",
"project_urls": {
"Documentation": "https://github.com/Tobi-De/django-litestream#readme",
"Issues": "https://github.com/Tobi-De/django-litestream/issues",
"Source": "https://github.com/Tobi-De/django-litestream"
},
"split_keywords": [
"backup",
" django",
" litestream",
" replication",
" sqlite",
" sqlite3"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0995c53d28c9bd22c4aa1701517c5c765e86f73997b321cc97b7b19f78119b3d",
"md5": "0ed7aa094ea448b868ac1a369c9775cc",
"sha256": "651433723907ae8cbe9423f80f4e6d0207df9d0c3bebb71b4f13a014c6c10bd8"
},
"downloads": -1,
"filename": "django_litestream-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0ed7aa094ea448b868ac1a369c9775cc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11252,
"upload_time": "2024-08-19T17:41:57",
"upload_time_iso_8601": "2024-08-19T17:41:57.536651Z",
"url": "https://files.pythonhosted.org/packages/09/95/c53d28c9bd22c4aa1701517c5c765e86f73997b321cc97b7b19f78119b3d/django_litestream-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b650b178b640ceb6a6aba5734a66c6bea8d1e3951ad05aa169fbfe22d4d23fdc",
"md5": "1aaa362609131dffcd086606770e6c74",
"sha256": "c342340be427e98b6f45fa0ad47814ce4f449c270cb8c1922ad2ea08068720df"
},
"downloads": -1,
"filename": "django_litestream-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "1aaa362609131dffcd086606770e6c74",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 16788,
"upload_time": "2024-08-19T17:41:56",
"upload_time_iso_8601": "2024-08-19T17:41:56.302404Z",
"url": "https://files.pythonhosted.org/packages/b6/50/b178b640ceb6a6aba5734a66c6bea8d1e3951ad05aa169fbfe22d4d23fdc/django_litestream-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-19 17:41:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Tobi-De",
"github_project": "django-litestream#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "django-litestream"
}
