| Name | coverme JSON |
| Version |
0.8.0
JSON |
| download |
| home_page | |
| Summary | Lightweight and easy configurable server backup utility. |
| upload_time | 2023-11-09 08:10:20 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | Apache-2.0 |
| keywords |
backup
utility
aws
devops
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
coverme
=======
[](https://pypi.python.org/pypi/coverme)
Lightweight and easy configurable server backup utility.
- [Install](#install)
- [Basic usage](#basic-usage)
- [Questions and answers](#questions-and-answers)
- [Command line help](#command-line-help)
- [Tips on Amazon services setup](#tips-on-amazon-services-setup)
- [License](#license)
Overview
--------
`coverme` acts as a backup data aggregator, it grabs data from different sources and uploads it to Amazon Glacier or Amazon S3. More storage options will be probably available. You're always welcome to fork and make a pull request ;)
* Requires Python 2.7+ or Python 3.4+ installed
* Should work on Linux/MacOS/FreeBSD
* Supports MySQL backups via `mysqldump`
* Supports PostgreSQL backups via `pg_dump`
* Supports directory archive
* Uploading backups to Amazon S3 as private objects
* Uploading to Amazon Glacier
* Schedule backups via `crontab` or run manually
Also for Amazon services configuration you may need to install AWS command line utility `awscli`.
Install
-------
Install via pip:
```
pip install coverme
```
or if only Python 3.x pip is available:
```
pip3 install coverme
```
If you're going to use Amazon services for backup, you'll also need to set up credentials via `aws configure` or manually, see [Tips on Amazon services setup](#tips-on-amazon-services-setup).
Basic usage
-----------
Just to make sure that installation is correct run:
```
coverme --help
```
**Please note!** Examples below probably do not provide best security practices! Intermediate backups are stored in shared `/tmp` dir and backups are not encrypted before upload.
1. Define your backup rules in JSON or YAML config, e.g. `backup.yml`:
```yaml
---
defaults:
# tmpdir - base directory for temporary files
tmpdir: /tmp
# cleanup - remove archive after upload or not, default: no
cleanup: yes
# localdir - optional, directory for local backups
localdir: backups
# format - optional, default: zip
format: gztar
backups:
- type: database
url: postgres://postgres@127.0.0.1:5432/test
to: [bucket1]
name: postgres-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql
# tags - optional, no default
tags: myapp, postgres, db
- type: database
url: mysql://root@127.0.0.1/test
to: [bucket1]
name: mysql-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql
# tags - optional, no default
tags: myapp, mysql, db
- type: dir
path: /home/myapp/var/www/uploads
to: [glacier1]
name: myapp-{yyyy}-{mm}-{dd}--{HH}-{MM}
# format - optional, default: zip
format: gztar
# tags - optional, no default
tags: myapp, uploads
vaults:
bucket1:
service: s3
region: eu-west-1
# profile - optional, AWS configuration profile
# profile: coverme
name: coverme-test
glacier1:
service: glacier
region: eu-west-1
account: NNNNNNNNNNNN
name: coverme-test
...
```
2. Perform test backup using config file `backup.yml`:
```
coverme backup -c backup.yml
```
If some configutaion error happen, fix it first. For example, may you have to configure AWS credentials with [`aws configure`](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) first.
3. Add `coverme` to cron jobs:
```
crontab -e
```
Make sure to add PATH environment setting, so cron script could found `pg_dump`, `mysqldump` and other shell commands:
```
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# run coverme backups every day at 5:30
30 5 * * * /usr/local/bin/coverme backup -c /home/myapp/backup.yml
```
Real path to `coverme` may be different depending on installation, to get proper full path run:
```
which coverme
```
Usage with Docker
-----------------
Docker container is available as [rudyryk/coverme](https://cloud.docker.com/u/rudyryk/repository/docker/rudyryk/coverme).
Here's an example for using in `docker-compose.yml`:
```yaml
# ... your services
coverme:
image: rudyryk/coverme
restart: unless-stopped
depends_on:
- postgres
environment:
PGPASSWORD: ${PGPASSWORD}
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
volumes:
- ./coverme/config.yml:/etc/coverme/config.yml
```
Questions and answers
---------------------
### How to provide PostgreSQL password for `pg_dump`?
Use `~/.pgpass` file with `600` permissions, see details here:
http://www.postgresql.org/docs/current/static/libpq-pgpass.html
### How to rotate my backups?
It's easy to rotate backups using name pattern. For example, specify
`mysql/dump-{dd}.sql` as name for MySQL dump archive and you will get
sequence of files like that:
mysql/dump-01.sql (for 1st day of month)
mysql/dump-02.sql (for 2nd day of month)
...
mysql/dump-31.sql (for 31st day of month)
So you will get monthly backups rotation. Some months have 31 day while others have 30 or 28/29, but that should not be a real issue in most cases.
Command line help
-----------------
Get list of available commands:
```
$ coverme --help
Usage: coverme [OPTIONS] COMMAND [ARGS]...
Command-line interface for coverme.
Options:
--help Show this message and exit.
Commands:
backup
```
Get help for `backup` command:
```
$ coverme backup --help
Usage: coverme backup [OPTIONS]
Options:
-c, --config TEXT Backups configuration file.Specify '-' to read from
STDIN. [default: backup.yml]
--help Show this message and exit.
```
Tips on Amazon services setup
-----------------------------
### How to install `aws` Amazon command line utility?
```
pip install awscli
```
### How to set up Amazon credentials?
```
aws configure
```
or manually save credentials to `~/.aws/credentials`:
```
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
[another]
; Just a custom profile called 'another', optional section
aws_access_key_id = YOUR_ANOTHER_ACCESS_KEY
aws_secret_access_key = YOUR_ANOTHER_SECRET_KEY
```
### How to get Amazon credentials?
Use Amazon Identity and Access Management (IAM) service to manage users:
https://console.aws.amazon.com/iam/home#users
### How to create Amazon Glacier vault?
You may create vaults using wizard:
https://console.aws.amazon.com/glacier/home
### How to grant user access to Glacier?
In Amazon Identity and Access Management (IAM) panel on user's details page:
_Permissions tab_ -> _Inline Policies_ (click to expand) -> **Create User Policy**
Choose _Policy Generator_ and then:
- Select "Amazon Glacier" in dropdown
- Check required pesmissions or mark all
- Specify Glacier vault identifier, e.g. `arn:aws:glacier:eu-west-1:NNNNNNNNNNNN:vaults/coverme-test`, vault should be created first
### How to grant user access to S3?
In Amazon Identity and Access Management (IAM) panel on user's details page:
_Permissions tab_ -> _Inline Policies_ (click to expand) -> **Create User Policy**
Choose _Policy Generator_ and then:
- Select "Amazon S3" in dropdown
- Check required pesmissions or mark all
- Specify S3 bucket resources mask, e.g. `arn:aws:s3:::coverme-test/*`
License
-------
Copyright (c) 2016, Alexey Kinëv <rudy@05bit.com>
Licensed under The Apache License Version 2.0, January 2004 (Apache-2.0),
see LICENSE file for more details.
Raw data
{
"_id": null,
"home_page": "",
"name": "coverme",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "backup,utility,aws,devops",
"author": "",
"author_email": "Alexey Kin\u00ebv <rudy@05bit.com>",
"download_url": "https://files.pythonhosted.org/packages/0a/d3/9d0aef65a8a03a85f9a6df1f6e878a2a37c6898f60b9ce2813eb9ccbf282/coverme-0.8.0.tar.gz",
"platform": null,
"description": "coverme\n=======\n\n[](https://pypi.python.org/pypi/coverme)\n\nLightweight and easy configurable server backup utility.\n\n- [Install](#install)\n- [Basic usage](#basic-usage)\n- [Questions and answers](#questions-and-answers)\n- [Command line help](#command-line-help)\n- [Tips on Amazon services setup](#tips-on-amazon-services-setup)\n- [License](#license)\n\nOverview\n--------\n\n`coverme` acts as a backup data aggregator, it grabs data from different sources and uploads it to Amazon Glacier or Amazon S3. More storage options will be probably available. You're always welcome to fork and make a pull request ;)\n\n* Requires Python 2.7+ or Python 3.4+ installed\n* Should work on Linux/MacOS/FreeBSD\n* Supports MySQL backups via `mysqldump`\n* Supports PostgreSQL backups via `pg_dump`\n* Supports directory archive\n* Uploading backups to Amazon S3 as private objects\n* Uploading to Amazon Glacier\n* Schedule backups via `crontab` or run manually\n\nAlso for Amazon services configuration you may need to install AWS command line utility `awscli`.\n\nInstall\n-------\n\nInstall via pip:\n\n```\npip install coverme\n```\n\nor if only Python 3.x pip is available: \n\n```\npip3 install coverme\n```\n\nIf you're going to use Amazon services for backup, you'll also need to set up credentials via `aws configure` or manually, see [Tips on Amazon services setup](#tips-on-amazon-services-setup).\n\nBasic usage\n-----------\n\nJust to make sure that installation is correct run:\n\n```\ncoverme --help\n```\n\n**Please note!** Examples below probably do not provide best security practices! Intermediate backups are stored in shared `/tmp` dir and backups are not encrypted before upload.\n\n1. Define your backup rules in JSON or YAML config, e.g. `backup.yml`:\n\n ```yaml\n ---\n defaults:\n # tmpdir - base directory for temporary files\n tmpdir: /tmp\n # cleanup - remove archive after upload or not, default: no\n cleanup: yes\n # localdir - optional, directory for local backups\n localdir: backups\n # format - optional, default: zip\n format: gztar\n\n backups:\n - type: database\n url: postgres://postgres@127.0.0.1:5432/test\n to: [bucket1]\n name: postgres-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql\n # tags - optional, no default\n tags: myapp, postgres, db\n\n - type: database\n url: mysql://root@127.0.0.1/test\n to: [bucket1]\n name: mysql-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql\n # tags - optional, no default\n tags: myapp, mysql, db\n\n - type: dir\n path: /home/myapp/var/www/uploads\n to: [glacier1]\n name: myapp-{yyyy}-{mm}-{dd}--{HH}-{MM}\n # format - optional, default: zip\n format: gztar\n # tags - optional, no default\n tags: myapp, uploads\n\n vaults:\n bucket1:\n service: s3\n region: eu-west-1\n # profile - optional, AWS configuration profile\n # profile: coverme\n name: coverme-test\n\n glacier1:\n service: glacier\n region: eu-west-1\n account: NNNNNNNNNNNN\n name: coverme-test\n ...\n ```\n\n2. Perform test backup using config file `backup.yml`:\n\n ```\n coverme backup -c backup.yml\n ```\n\n If some configutaion error happen, fix it first. For example, may you have to configure AWS credentials with [`aws configure`](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) first.\n\n3. Add `coverme` to cron jobs:\n\n ```\n crontab -e\n ```\n\n Make sure to add PATH environment setting, so cron script could found `pg_dump`, `mysqldump` and other shell commands:\n\n ```\n SHELL=/bin/sh\n PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\n\n # run coverme backups every day at 5:30\n 30 5 * * * /usr/local/bin/coverme backup -c /home/myapp/backup.yml\n ```\n\n Real path to `coverme` may be different depending on installation, to get proper full path run:\n\n ```\n which coverme\n ```\n\nUsage with Docker\n-----------------\n\nDocker container is available as [rudyryk/coverme](https://cloud.docker.com/u/rudyryk/repository/docker/rudyryk/coverme).\n\nHere's an example for using in `docker-compose.yml`:\n\n```yaml\n # ... your services\n\n coverme:\n image: rudyryk/coverme\n restart: unless-stopped\n depends_on:\n - postgres\n environment:\n PGPASSWORD: ${PGPASSWORD}\n AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}\n AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}\n volumes:\n - ./coverme/config.yml:/etc/coverme/config.yml\n```\n\nQuestions and answers\n---------------------\n\n### How to provide PostgreSQL password for `pg_dump`?\n\nUse `~/.pgpass` file with `600` permissions, see details here: \nhttp://www.postgresql.org/docs/current/static/libpq-pgpass.html\n\n### How to rotate my backups?\n\nIt's easy to rotate backups using name pattern. For example, specify\n`mysql/dump-{dd}.sql` as name for MySQL dump archive and you will get\nsequence of files like that:\n\n mysql/dump-01.sql (for 1st day of month)\n mysql/dump-02.sql (for 2nd day of month)\n ...\n mysql/dump-31.sql (for 31st day of month)\n\nSo you will get monthly backups rotation. Some months have 31 day while others have 30 or 28/29, but that should not be a real issue in most cases.\n\nCommand line help\n-----------------\n\nGet list of available commands:\n\n```\n$ coverme --help\n\nUsage: coverme [OPTIONS] COMMAND [ARGS]...\n\n Command-line interface for coverme.\n\nOptions:\n --help Show this message and exit.\n\nCommands:\n backup\n```\n\nGet help for `backup` command:\n\n```\n$ coverme backup --help\n\nUsage: coverme backup [OPTIONS]\n\nOptions:\n -c, --config TEXT Backups configuration file.Specify '-' to read from\n STDIN. [default: backup.yml]\n --help Show this message and exit.\n```\n\n\nTips on Amazon services setup\n-----------------------------\n\n### How to install `aws` Amazon command line utility?\n\n```\npip install awscli\n```\n\n### How to set up Amazon credentials?\n\n```\naws configure\n```\n\nor manually save credentials to `~/.aws/credentials`:\n\n```\n[default]\naws_access_key_id = YOUR_ACCESS_KEY\naws_secret_access_key = YOUR_SECRET_KEY\n\n[another]\n; Just a custom profile called 'another', optional section\naws_access_key_id = YOUR_ANOTHER_ACCESS_KEY\naws_secret_access_key = YOUR_ANOTHER_SECRET_KEY\n```\n\n### How to get Amazon credentials?\n\nUse Amazon Identity and Access Management (IAM) service to manage users: \nhttps://console.aws.amazon.com/iam/home#users\n\n### How to create Amazon Glacier vault?\n\nYou may create vaults using wizard: \nhttps://console.aws.amazon.com/glacier/home\n\n### How to grant user access to Glacier?\n\nIn Amazon Identity and Access Management (IAM) panel on user's details page: \n_Permissions tab_ -> _Inline Policies_ (click to expand) -> **Create User Policy**\n\nChoose _Policy Generator_ and then:\n\n- Select \"Amazon Glacier\" in dropdown\n- Check required pesmissions or mark all\n- Specify Glacier vault identifier, e.g. `arn:aws:glacier:eu-west-1:NNNNNNNNNNNN:vaults/coverme-test`, vault should be created first\n\n### How to grant user access to S3?\n\nIn Amazon Identity and Access Management (IAM) panel on user's details page: \n_Permissions tab_ -> _Inline Policies_ (click to expand) -> **Create User Policy**\n\nChoose _Policy Generator_ and then:\n\n- Select \"Amazon S3\" in dropdown\n- Check required pesmissions or mark all\n- Specify S3 bucket resources mask, e.g. `arn:aws:s3:::coverme-test/*`\n\nLicense\n-------\n\nCopyright (c) 2016, Alexey Kin\u00ebv <rudy@05bit.com>\n\nLicensed under The Apache License Version 2.0, January 2004 (Apache-2.0),\nsee LICENSE file for more details.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Lightweight and easy configurable server backup utility.",
"version": "0.8.0",
"project_urls": {
"Bug Tracker": "https://github.com/05bit/coverme/issues",
"Homepage": "https://github.com/05bit/coverme"
},
"split_keywords": [
"backup",
"utility",
"aws",
"devops"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b1328bdffab6d4c8a6f820c810cd036841105827dfc6a316a8870f1ba8d2ab8c",
"md5": "d1a765dd51c9286d62bbed324193a932",
"sha256": "d683d5584a125a959c1df6831fac71b7531f68cc76904b934269f62b95e4d016"
},
"downloads": -1,
"filename": "coverme-0.8.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d1a765dd51c9286d62bbed324193a932",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 14142,
"upload_time": "2023-11-09T08:10:19",
"upload_time_iso_8601": "2023-11-09T08:10:19.134930Z",
"url": "https://files.pythonhosted.org/packages/b1/32/8bdffab6d4c8a6f820c810cd036841105827dfc6a316a8870f1ba8d2ab8c/coverme-0.8.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0ad39d0aef65a8a03a85f9a6df1f6e878a2a37c6898f60b9ce2813eb9ccbf282",
"md5": "9933fdafafc0c069ff9caf0cac27b2e9",
"sha256": "7f2a72b70ba01ee0aae85cecbde8a57ef315f1b287b96fc7d3cb9245bd00d8c6"
},
"downloads": -1,
"filename": "coverme-0.8.0.tar.gz",
"has_sig": false,
"md5_digest": "9933fdafafc0c069ff9caf0cac27b2e9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 13906,
"upload_time": "2023-11-09T08:10:20",
"upload_time_iso_8601": "2023-11-09T08:10:20.702062Z",
"url": "https://files.pythonhosted.org/packages/0a/d3/9d0aef65a8a03a85f9a6df1f6e878a2a37c6898f60b9ce2813eb9ccbf282/coverme-0.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-09 08:10:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "05bit",
"github_project": "coverme",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "coverme"
}
