# backupdirs3
A tool to monitor a directory for changes, create a backup in the form of a zip file, and upload it to an AWS S3 bucket.
## Prerequisites
* `AWS credentials`: Configure AWS credentials to enable S3 uploads.
## How to run
```
pip install backupdirs3
backupdirs3 -h
```
## Help
```
usage: backupdirs3 [-h] [-v] [-c CONFIG_FILE] [-m MONITORED_DIR] [-s S3_BUCKET] [-n NODE_NAME] [-b BACKUP_NAME]
[-l LOCAL_BACKUP_DIR] [-k] [-d DELAY_BEFORE_UPLOAD] [-i INCLUDE_FILES] [-x EXCLUDE_FILES] [-1]
This tool monitors a directory for changes, create a backup in the form of a zip file, and upload it to an AWS S3 bucket.
options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-c CONFIG_FILE, --config-file CONFIG_FILE
yaml config file, mutually exclusive with other command line options
default: /etc/backupdirs3/config.yaml
-m MONITORED_DIR, --monitored-dir MONITORED_DIR
dir to monitor for changes
default: /etc/backupdirs3
-s S3_BUCKET, --s3-bucket S3_BUCKET
aws s3 bucket to upload backup zip files
default: backupdirs3-bucket
-n NODE_NAME, --node-name NODE_NAME
node name to use as prefix for backup file
default: thinkpad-e16gen1
-b BACKUP_NAME, --backup-name BACKUP_NAME
app name to use as suffix for backup file
default: backup
-l LOCAL_BACKUP_DIR, --local-backup-dir LOCAL_BACKUP_DIR
local dir to store backup zip files before upload
default: /tmp
-k, --keep-local-backups
do not delete backup zip files after upload to s3
default: False
-d DELAY_BEFORE_UPLOAD, --delay-before-upload DELAY_BEFORE_UPLOAD
seconds to wait after the last file update event before starting upload, valid range: [1..60]
default: 10
-i INCLUDE_FILES, --include-files INCLUDE_FILES
files to include into backup. Can specify multiple times. If not specified, ALL files are included.
default: []
-x EXCLUDE_FILES, --exclude-files EXCLUDE_FILES
files to exclude from backup. Can specify multiple times. If not specified, NO files are excluded.
default: []
-1, --one-time-run perform backup and exit without further monitoring.
default: False
```
## Configuration
By default, the tool looks for its configuration file at `/etc/backupdirs3/config.yaml`.
The settings specified in the default config file are used as defaults and can be overridden by command-line options.
> [!IMPORTANT]
>
> When custom config file is specified via `-c` `--config-file`
> * no other command-line parameters are allowed
> * only settings from the specified config file are used
> * default config is ignored
```bash
backupdirs3 -c ./config.yaml
```
### Example Configuration File Explained
`config.yaml`:
```yaml
# Config file for the backup directory monitoring tool
# The directory to monitor for changes
# Must be an existing directory and cannot be the root directory ('/')
# User must have read permissions to all files within this dir
monitored_dir: "/etc/backupdirs3"
# AWS S3 bucket where the backup files will be uploaded
# This bucket must already exist, and the script should have the necessary permissions to upload to it
s3_bucket: "backupdirs3-s3-bucket"
# The name of the node (usually the machine's hostname) used in naming the backup files
# Optional. If specified must only contain lowercase letters, numbers, dots, and hyphens
node_name: "your-node-name"
# A custom name to be appended to the backup file
# Optional. If specified must only contain lowercase letters, numbers, underscores, and hyphens
backup_name: "backup"
# Directory to store the local backup zip files before uploading to S3
# Must be an existing directory; defaults to system temporary directory
local_backup_dir: "/tmp"
# Delay in seconds after the last detected change before the backup process starts
# This is used to debounce rapid file changes and ensure the backup process isn't triggered too often
# Valid values are between 1 and 60 seconds
delay_before_upload: 10
# Whether to keep the local backup zip files after they are uploaded to S3
# Set to true if you want to retain the backups locally; false to delete them after upload
keep_local_backups: false
# List of file patterns to include in the backup.
# Patterns can be specified using Unix-style wildcards (e.g., "*.txt" to include all text files).
# If not specified, all files in the monitored directory are included by default.
include_files:
- "*.n3c"
# List of file patterns to exclude from the backup.
# Use Unix-style wildcards (e.g., "*.log" to exclude all log files).
# If not specified, no files are excluded by default.
exclude_files:
- "*.bak"
- "*.bkp"
# Do not monitor changes, perform one-time backup and exit immediately.
# It makes more sense to specify this flag as a command line parameter rather than in config file.
# Defaults to false.
one_time_run: false
```
## Future improvements
1. include/exclude filters for files inside dir [DONE]
2. encrypt zip archive before upload?
3. one-time backup without monitor loop [DONE]
4. backup to local dir only without s3 upload?
Raw data
{
"_id": null,
"home_page": "https://github.com/ludenus/backupdirs3",
"name": "backupdirs3",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "backup, aws, s3, directory-monitor",
"author": "Andrey Andreyev",
"author_email": "x0153362@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c6/9d/e62bfcc49cf6dcfd787aa1d712bafe656a8837d75a4dbafbf41de2478427/backupdirs3-0.3.10.tar.gz",
"platform": null,
"description": "# backupdirs3\n\nA tool to monitor a directory for changes, create a backup in the form of a zip file, and upload it to an AWS S3 bucket.\n\n## Prerequisites\n* `AWS credentials`: Configure AWS credentials to enable S3 uploads.\n\n\n## How to run\n```\npip install backupdirs3\n\nbackupdirs3 -h\n\n```\n\n## Help\n```\nusage: backupdirs3 [-h] [-v] [-c CONFIG_FILE] [-m MONITORED_DIR] [-s S3_BUCKET] [-n NODE_NAME] [-b BACKUP_NAME]\n [-l LOCAL_BACKUP_DIR] [-k] [-d DELAY_BEFORE_UPLOAD] [-i INCLUDE_FILES] [-x EXCLUDE_FILES] [-1]\n\nThis tool monitors a directory for changes, create a backup in the form of a zip file, and upload it to an AWS S3 bucket.\n\noptions:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n -c CONFIG_FILE, --config-file CONFIG_FILE\n yaml config file, mutually exclusive with other command line options \n default: /etc/backupdirs3/config.yaml\n -m MONITORED_DIR, --monitored-dir MONITORED_DIR\n dir to monitor for changes \n default: /etc/backupdirs3\n -s S3_BUCKET, --s3-bucket S3_BUCKET\n aws s3 bucket to upload backup zip files \n default: backupdirs3-bucket\n -n NODE_NAME, --node-name NODE_NAME\n node name to use as prefix for backup file \n default: thinkpad-e16gen1\n -b BACKUP_NAME, --backup-name BACKUP_NAME\n app name to use as suffix for backup file \n default: backup\n -l LOCAL_BACKUP_DIR, --local-backup-dir LOCAL_BACKUP_DIR\n local dir to store backup zip files before upload \n default: /tmp\n -k, --keep-local-backups\n do not delete backup zip files after upload to s3 \n default: False\n -d DELAY_BEFORE_UPLOAD, --delay-before-upload DELAY_BEFORE_UPLOAD\n seconds to wait after the last file update event before starting upload, valid range: [1..60] \n default: 10\n -i INCLUDE_FILES, --include-files INCLUDE_FILES\n files to include into backup. Can specify multiple times. If not specified, ALL files are included. \n default: []\n -x EXCLUDE_FILES, --exclude-files EXCLUDE_FILES\n files to exclude from backup. Can specify multiple times. If not specified, NO files are excluded. \n default: []\n -1, --one-time-run perform backup and exit without further monitoring. \n default: False\n```\n\n## Configuration\nBy default, the tool looks for its configuration file at `/etc/backupdirs3/config.yaml`. \nThe settings specified in the default config file are used as defaults and can be overridden by command-line options.\n\n> [!IMPORTANT]\n>\n> When custom config file is specified via `-c` `--config-file`\n> * no other command-line parameters are allowed\n> * only settings from the specified config file are used\n> * default config is ignored\n\n```bash\nbackupdirs3 -c ./config.yaml\n```\n\n### Example Configuration File Explained\n`config.yaml`:\n```yaml\n# Config file for the backup directory monitoring tool\n\n# The directory to monitor for changes\n# Must be an existing directory and cannot be the root directory ('/')\n# User must have read permissions to all files within this dir\nmonitored_dir: \"/etc/backupdirs3\"\n\n# AWS S3 bucket where the backup files will be uploaded\n# This bucket must already exist, and the script should have the necessary permissions to upload to it\ns3_bucket: \"backupdirs3-s3-bucket\"\n\n# The name of the node (usually the machine's hostname) used in naming the backup files\n# Optional. If specified must only contain lowercase letters, numbers, dots, and hyphens\nnode_name: \"your-node-name\"\n\n# A custom name to be appended to the backup file\n# Optional. If specified must only contain lowercase letters, numbers, underscores, and hyphens\nbackup_name: \"backup\"\n\n# Directory to store the local backup zip files before uploading to S3\n# Must be an existing directory; defaults to system temporary directory\nlocal_backup_dir: \"/tmp\"\n\n# Delay in seconds after the last detected change before the backup process starts\n# This is used to debounce rapid file changes and ensure the backup process isn't triggered too often\n# Valid values are between 1 and 60 seconds\ndelay_before_upload: 10\n\n# Whether to keep the local backup zip files after they are uploaded to S3\n# Set to true if you want to retain the backups locally; false to delete them after upload\nkeep_local_backups: false\n\n# List of file patterns to include in the backup.\n# Patterns can be specified using Unix-style wildcards (e.g., \"*.txt\" to include all text files).\n# If not specified, all files in the monitored directory are included by default.\ninclude_files:\n - \"*.n3c\"\n\n# List of file patterns to exclude from the backup.\n# Use Unix-style wildcards (e.g., \"*.log\" to exclude all log files).\n# If not specified, no files are excluded by default.\nexclude_files:\n - \"*.bak\"\n - \"*.bkp\"\n\n# Do not monitor changes, perform one-time backup and exit immediately.\n# It makes more sense to specify this flag as a command line parameter rather than in config file.\n# Defaults to false.\none_time_run: false\n```\n\n## Future improvements\n\n1. include/exclude filters for files inside dir [DONE]\n2. encrypt zip archive before upload?\n3. one-time backup without monitor loop [DONE]\n4. backup to local dir only without s3 upload?\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "This tool monitors a directory for changes, create a backup in the form of a zip file, and upload it to an AWS S3 bucket.",
"version": "0.3.10",
"project_urls": {
"Documentation": "https://github.com/ludenus/backupdirs3",
"Homepage": "https://github.com/ludenus/backupdirs3",
"Repository": "https://github.com/ludenus/backupdirs3"
},
"split_keywords": [
"backup",
" aws",
" s3",
" directory-monitor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ee7db2d8b26a9982c6e30c6bbc1b265239dc6445b7e73cc98d8242741b7b9d28",
"md5": "0148401061bba0e3e281bfd02b23a7cc",
"sha256": "fb6d2c2c50d2b350f3a54a475248d0424da2b5528f4bbe11919ab352c0488a65"
},
"downloads": -1,
"filename": "backupdirs3-0.3.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0148401061bba0e3e281bfd02b23a7cc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 8534,
"upload_time": "2024-09-23T19:36:06",
"upload_time_iso_8601": "2024-09-23T19:36:06.413041Z",
"url": "https://files.pythonhosted.org/packages/ee/7d/b2d8b26a9982c6e30c6bbc1b265239dc6445b7e73cc98d8242741b7b9d28/backupdirs3-0.3.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c69de62bfcc49cf6dcfd787aa1d712bafe656a8837d75a4dbafbf41de2478427",
"md5": "d1d25c5c805f419d9669cda844fbe5bc",
"sha256": "ea60de0366a6a4f206b5572a687903635e7bc04842d0f13ce6ba139fea9feadd"
},
"downloads": -1,
"filename": "backupdirs3-0.3.10.tar.gz",
"has_sig": false,
"md5_digest": "d1d25c5c805f419d9669cda844fbe5bc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 7365,
"upload_time": "2024-09-23T19:36:07",
"upload_time_iso_8601": "2024-09-23T19:36:07.666877Z",
"url": "https://files.pythonhosted.org/packages/c6/9d/e62bfcc49cf6dcfd787aa1d712bafe656a8837d75a4dbafbf41de2478427/backupdirs3-0.3.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-23 19:36:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ludenus",
"github_project": "backupdirs3",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "backupdirs3"
}
JSON
