# CoBRA - Comprehensive Backing up and Restoration Archiver
Cobra is a tool for creating, managing and restoring backups.
It is designed to cover docker powered applications as well as it allows backing up of
regular file system folders.
## How to use
```
pip install cobra-archiver
```
### CLI
After that `cobra` command will be available from the command line.
To get the cli description please issue `cobra --help` or
e.g. `cobra backup --help` to get help on certain command.
This will backup all the docker volumes as well as `/want/this/dir/backed/up`
directory, but `skip-this-volume` `and-this-one`.
```bash
cobra backup build --push --dir /want/this/dir/backed/up \
--creds /path/to/google-service-acc-key.json --folder-id google-drive-folder-id \
--exclude skip-this-volume and-this-one
```
This restores latest backup from the given remote folder.
```bash
cobra backup pull --latest --restore \
--creds /path/to/google-service-acc-key.json --folder-id google-drive-folder-id
```
### Remote storage
For now Google Drive only supported. If you find this project useful you can contribute
to enhance it. Or at least you can post a feature request.
1. To have this work the [Google Service Account](https://cloud.google.com/iam/docs/service-accounts) is necessary.
The service account id (email) looks like `<the-name-you-choose>@hip-heading-376120.iam.gserviceaccount.com`.
2. Under the service account you've created add the key pair and download it in `.json` format.
3. Now create the folder within your Google Drive you wish to push the backups in.
4. Share this folder with the service account (email) from step 1.
### Hooks
They are listed below.
```python
HOOKS = ('before_build', 'after_build', 'before_push', 'after_push',
'before_pull', 'after_pull', 'before_restore', 'after_restore')
```
One can either issue `cobra hooks init` that populates hook files to the default directory.
Or put the hook files with the names e.g. `before_build.py` or `before_build.sh`.
For shell script `chmod +x before_build.sh` is necessary.
Cobra searches for `.py` file first if found imports it and execute `hook` function as.
```python
hook(hook_name=hook_name, hooks_dir=hooks_dir, backup_dir=backup_dir,
filename=backup_filename, docker=docker_client)
```
* `hook_name` is the one from the list above
* `hooks_dir` the directory where hooks reside
* `backup_dir` the local backup directory where backup is stored
* `filename` the backup file name
* `docker` [DockerClient](https://docker-py.readthedocs.io/en/stable/client.html#docker.client.DockerClient) object
If `.py` file is not found. The default hook is called that continue searching for `.sh` file.
If latter found it's called via `subprocess.check_call()`. With the same params except `docker`.
By default `cobra` copies and packs the content of a volume.
To backup database with tools like `mongodump` or `pg_dump` one may use `before_build` hook
and `--exclude volume-name` from the processing.
`before_build` hook may look like this in such a case.
```bash
#!/usr/bin/env bash
# Stop any containers that mangle database while dumping to have consistent dump
docker stop my-excellent-app
MONGO_DUMP_DIR=/tmp/mongodump
mkdir -p "${MONGO_DUMP_DIR}"
mongodump --archive="${MONGO_DUMP_DIR}/mongo-dump-by-hook.tar.gz" --db=test --gzip mongodb://mongo-container-name:27017
# Then start them again
docker start my-excellent-app
```
Errors that are propagated from hooks stop farther processing.
To see more details please inspect e2e test sources.
### Default locations
To find out paths used by cobra one can issue following.
On my system I have this output.
```bash
$ cobra dirs
/home/q/.local/share/cobra/backup
/home/q/.cache/cobra
/home/q/.local/share/cobra/hooks
```
### Python
Minimum python version is 3.7.
```python
from cobra.api import Api
from cobra.hooks import Hooks
from docker import DockerClient
api = Api(gateway=DockerClient(), hooks=Hooks())
api.backup_build()
```
Method parameters are described in cli help `cobra backup --help` e.g.
### Security notice
This code is subject to command injection vulnerabilty. There are no such a checks.
The caller should provide all checks on his own.
## Run tests
```bash
git clone https://github.com/yell0w4x/cobra.git
cd cobra
./run-tests --unit
```
The above runs unit tests. To execute end-to-end tests run is as follows.
Note that docker must reside in the system.
To install it on Ubuntu use `wget -qO- https://get.docker.com | sudo bash`.
On Manjaro (Arch) issue `sudo pacman -S docker`.
```bash
./run-tests --e2e --folder-id goolge-drive-folder-id --key path/to/google-service-account-key.json
```
or
```bash
GOOGLE_DRIVE_FOLDER_ID=goolge-drive-folder-id GOOGLE_SERVICE_ACC_KEY=path/to/key.json ./run-tests --e2e
```
The tests are based on pytest. All the extra arguments are passed to pytest.
E.g. to have verbose output use `-v` or `-vv`. To show stdout `-s`.
To run certain tests use `-k test_name` and etc. For details see the pytest docs.
```
./run-tests --help
Run cobra unit and e2e tests.
Usage:
./run-tests [OPTIONS] [EXTRA_ARGS]
All the EXTRA_ARGS are passed to pytest
Options:
--help Show help message
--unit Run unit tests
--e2e Run e2e tests
--skip-build Skip building dist files
--folder-id FOLDER_ID Google drive folder id to use as remote storage for e2e tests.
If not given read from GOOGLE_DRIVE_FOLDER_ID environment variable.
--key KEY_FN Path to google service account key file in json format
If not given read from GOOGLE_SERVICE_ACC_KEY environment variable.
```
Raw data
{
"_id": null,
"home_page": "https://github.com/yell0w4x/cobra",
"name": "cobra-archiver",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "Docker volumes backup",
"author": "yell0w4x",
"author_email": "yell0w4x@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ee/13/d4201cc2660e11bfbff687827061d1831087afe43d4360e7ef79c1b5fddc/cobra-archiver-0.0.5.tar.gz",
"platform": null,
"description": "# CoBRA - Comprehensive Backing up and Restoration Archiver\n\nCobra is a tool for creating, managing and restoring backups. \nIt is designed to cover docker powered applications as well as it allows backing up of \nregular file system folders.\n\n\n\n## How to use\n\n```\npip install cobra-archiver\n```\n\n### CLI\n\nAfter that `cobra` command will be available from the command line.\n\nTo get the cli description please issue `cobra --help` or \ne.g. `cobra backup --help` to get help on certain command.\n\nThis will backup all the docker volumes as well as `/want/this/dir/backed/up` \ndirectory, but `skip-this-volume` `and-this-one`.\n\n```bash\ncobra backup build --push --dir /want/this/dir/backed/up \\\n --creds /path/to/google-service-acc-key.json --folder-id google-drive-folder-id \\\n --exclude skip-this-volume and-this-one\n```\n\nThis restores latest backup from the given remote folder.\n\n```bash\ncobra backup pull --latest --restore \\\n --creds /path/to/google-service-acc-key.json --folder-id google-drive-folder-id\n```\n\n### Remote storage\n\nFor now Google Drive only supported. If you find this project useful you can contribute \nto enhance it. Or at least you can post a feature request.\n\n1. To have this work the [Google Service Account](https://cloud.google.com/iam/docs/service-accounts) is necessary.\n The service account id (email) looks like `<the-name-you-choose>@hip-heading-376120.iam.gserviceaccount.com`. \n2. Under the service account you've created add the key pair and download it in `.json` format. \n3. Now create the folder within your Google Drive you wish to push the backups in.\n4. Share this folder with the service account (email) from step 1.\n\n### Hooks\n\nThey are listed below.\n\n```python\nHOOKS = ('before_build', 'after_build', 'before_push', 'after_push', \n 'before_pull', 'after_pull', 'before_restore', 'after_restore')\n```\n\nOne can either issue `cobra hooks init` that populates hook files to the default directory. \nOr put the hook files with the names e.g. `before_build.py` or `before_build.sh`. \nFor shell script `chmod +x before_build.sh` is necessary.\n\nCobra searches for `.py` file first if found imports it and execute `hook` function as.\n\n```python\nhook(hook_name=hook_name, hooks_dir=hooks_dir, backup_dir=backup_dir, \n filename=backup_filename, docker=docker_client)\n```\n\n* `hook_name` is the one from the list above\n* `hooks_dir` the directory where hooks reside\n* `backup_dir` the local backup directory where backup is stored \n* `filename` the backup file name\n* `docker` [DockerClient](https://docker-py.readthedocs.io/en/stable/client.html#docker.client.DockerClient) object\n\nIf `.py` file is not found. The default hook is called that continue searching for `.sh` file.\nIf latter found it's called via `subprocess.check_call()`. With the same params except `docker`.\n\nBy default `cobra` copies and packs the content of a volume. \nTo backup database with tools like `mongodump` or `pg_dump` one may use `before_build` hook\nand `--exclude volume-name` from the processing.\n`before_build` hook may look like this in such a case.\n\n```bash\n#!/usr/bin/env bash\n\n# Stop any containers that mangle database while dumping to have consistent dump\ndocker stop my-excellent-app\n\nMONGO_DUMP_DIR=/tmp/mongodump\nmkdir -p \"${MONGO_DUMP_DIR}\"\nmongodump --archive=\"${MONGO_DUMP_DIR}/mongo-dump-by-hook.tar.gz\" --db=test --gzip mongodb://mongo-container-name:27017\n\n# Then start them again\ndocker start my-excellent-app\n```\n\nErrors that are propagated from hooks stop farther processing. \nTo see more details please inspect e2e test sources.\n\n### Default locations\n\nTo find out paths used by cobra one can issue following. \nOn my system I have this output.\n\n```bash\n$ cobra dirs\n/home/q/.local/share/cobra/backup\n/home/q/.cache/cobra\n/home/q/.local/share/cobra/hooks\n```\n\n### Python\n\nMinimum python version is 3.7.\n\n```python\nfrom cobra.api import Api\nfrom cobra.hooks import Hooks\nfrom docker import DockerClient\n\napi = Api(gateway=DockerClient(), hooks=Hooks())\napi.backup_build()\n```\n\nMethod parameters are described in cli help `cobra backup --help` e.g.\n\n### Security notice\n\nThis code is subject to command injection vulnerabilty. There are no such a checks. \nThe caller should provide all checks on his own.\n\n## Run tests\n\n```bash\ngit clone https://github.com/yell0w4x/cobra.git\ncd cobra\n./run-tests --unit\n```\n \nThe above runs unit tests. To execute end-to-end tests run is as follows. \nNote that docker must reside in the system.\nTo install it on Ubuntu use `wget -qO- https://get.docker.com | sudo bash`. \nOn Manjaro (Arch) issue `sudo pacman -S docker`.\n\n```bash\n./run-tests --e2e --folder-id goolge-drive-folder-id --key path/to/google-service-account-key.json\n```\nor\n```bash\nGOOGLE_DRIVE_FOLDER_ID=goolge-drive-folder-id GOOGLE_SERVICE_ACC_KEY=path/to/key.json ./run-tests --e2e\n```\n\nThe tests are based on pytest. All the extra arguments are passed to pytest. \nE.g. to have verbose output use `-v` or `-vv`. To show stdout `-s`. \nTo run certain tests use `-k test_name` and etc. For details see the pytest docs.\n\n```\n./run-tests --help\nRun cobra unit and e2e tests.\n\nUsage:\n ./run-tests [OPTIONS] [EXTRA_ARGS]\n\nAll the EXTRA_ARGS are passed to pytest\n\nOptions:\n --help Show help message\n --unit Run unit tests\n --e2e Run e2e tests\n --skip-build Skip building dist files\n --folder-id FOLDER_ID Google drive folder id to use as remote storage for e2e tests. \n If not given read from GOOGLE_DRIVE_FOLDER_ID environment variable.\n --key KEY_FN Path to google service account key file in json format\n If not given read from GOOGLE_SERVICE_ACC_KEY environment variable.\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Comprehensive Backing up and Restoration Archiver. Docker volumes, bind mounts and plain directory backup automation tool.",
"version": "0.0.5",
"project_urls": {
"Bug Tracker": "https://github.com/yell0w4x/cobra/issues",
"Homepage": "https://github.com/yell0w4x/cobra"
},
"split_keywords": [
"docker",
"volumes",
"backup"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "65044595b62ad05ce82f3bcf4edb9dbdfd5af68c418152ece8f7e49bf0bb8203",
"md5": "3d775718486e13c1b43a366a68b92876",
"sha256": "b584139af6485fbd784a22223e56d9b8e495660ac33a53e00a9ef558d9ab05b7"
},
"downloads": -1,
"filename": "cobra_archiver-0.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3d775718486e13c1b43a366a68b92876",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 26125,
"upload_time": "2023-05-11T19:55:54",
"upload_time_iso_8601": "2023-05-11T19:55:54.630244Z",
"url": "https://files.pythonhosted.org/packages/65/04/4595b62ad05ce82f3bcf4edb9dbdfd5af68c418152ece8f7e49bf0bb8203/cobra_archiver-0.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ee13d4201cc2660e11bfbff687827061d1831087afe43d4360e7ef79c1b5fddc",
"md5": "04baa4280c0fb0f76d72c4640b2869e2",
"sha256": "87d37212013583c80e778d2b6e357205fa13ca7ffc6bd18c6b642a54f98a2241"
},
"downloads": -1,
"filename": "cobra-archiver-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "04baa4280c0fb0f76d72c4640b2869e2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 25177,
"upload_time": "2023-05-11T19:55:57",
"upload_time_iso_8601": "2023-05-11T19:55:57.574184Z",
"url": "https://files.pythonhosted.org/packages/ee/13/d4201cc2660e11bfbff687827061d1831087afe43d4360e7ef79c1b5fddc/cobra-archiver-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-11 19:55:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yell0w4x",
"github_project": "cobra",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "attrs",
"specs": [
[
"==",
"22.2.0"
]
]
},
{
"name": "cachetools",
"specs": [
[
"==",
"5.3.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2022.12.7"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.0.1"
]
]
},
{
"name": "docker",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.1.0"
]
]
},
{
"name": "ezenv",
"specs": [
[
"==",
"0.92"
]
]
},
{
"name": "freezegun",
"specs": [
[
"==",
"1.2.2"
]
]
},
{
"name": "google-api-core",
"specs": [
[
"==",
"2.11.0"
]
]
},
{
"name": "google-api-python-client",
"specs": [
[
"==",
"2.74.0"
]
]
},
{
"name": "google-auth",
"specs": [
[
"==",
"2.16.0"
]
]
},
{
"name": "google-auth-httplib2",
"specs": [
[
"==",
"0.1.0"
]
]
},
{
"name": "google-auth-oauthlib",
"specs": [
[
"==",
"0.8.0"
]
]
},
{
"name": "googleapis-common-protos",
"specs": [
[
"==",
"1.58.0"
]
]
},
{
"name": "httplib2",
"specs": [
[
"==",
"0.21.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.4"
]
]
},
{
"name": "iniconfig",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "oauthlib",
"specs": [
[
"==",
"3.2.2"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"23.0"
]
]
},
{
"name": "pluggy",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"4.21.12"
]
]
},
{
"name": "pyasn1",
"specs": [
[
"==",
"0.4.8"
]
]
},
{
"name": "pyasn1-modules",
"specs": [
[
"==",
"0.2.8"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.14.0"
]
]
},
{
"name": "pyparsing",
"specs": [
[
"==",
"3.0.9"
]
]
},
{
"name": "pyproject_hooks",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "pytest",
"specs": [
[
"==",
"7.2.1"
]
]
},
{
"name": "pytest-lazy-fixture",
"specs": [
[
"==",
"0.6.3"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.28.2"
]
]
},
{
"name": "requests-oauthlib",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.3.1"
]
]
},
{
"name": "rsa",
"specs": [
[
"==",
"4.9"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.16.0"
]
]
},
{
"name": "tomli",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "uritemplate",
"specs": [
[
"==",
"4.1.1"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"1.26.14"
]
]
},
{
"name": "websocket-client",
"specs": [
[
"==",
"1.5.0"
]
]
}
],
"lcname": "cobra-archiver"
}
JSON
