[](https://opensource.org/licenses/GPL-3.0)
[](http://isitmaintained.com/project/netinvent/npbackup "Percentage of issues still open")
[](https://github.com/netinvent/npbackup/releases/latest)
[](https://github.com/netinvent/npbackup/actions/workflows/pylint-windows.yaml)
[](https://github.com/netinvent/npbackup/actions/workflows/pylint-linux.yaml)
# NPBackup
A secure and efficient file backup solution that fits both system administrators (CLI) and end users (GUI)
Works on x64 **Linux** , **NAS** solutions based on arm/arm64, **Windows** x64 and x86 and MacOS X.
## Features
- Data deduplication and fast zstd compression
- Client side data encryption
- Wide storage backend support
- local files
- SFTP
- High performance HTTP REST server
- Amazon S3/Minio/Wasabi
- Blackblaze B2
- Microsoft Azure Blob Storage
- Google Cloud Storage
- OpenStack Swift
- Alibaba Cloud (Aliyun) Object Storage System (OSS)
- Resume on interrupted backups
- Full CLI interface for scheduled task usage
- Checks for recent backups before launching a backup
- End User GUI
- Backups create, list, viewer and restore
- Full configuration interface
- Internationalization support (en, fr as of feb 2023)
- Performance
- Backup process and IO priority settings
- Upload / download speed limits
- Remote connectivity concurrency settings
- Comes with full exclusion lists for Linux and Windows
- First class prometheus support
- Restic results metric generatioion
- Grafana dashboard included
- node_exporter file collector support
- Optional push gateway metrics uploading
- First class Windows support
- VSS snapshots
- Automatic cloud file exclusions (reparse points)
- Windows pre-built executables
- Windows installer
- Additional security
- repository uri / password, http metrics and upgrade server passwords are AES-256 encrypted
- Encrypted data viewing requires additional password
- AES-256 keys can't be guessed in executables thanks to Nuitka Commercial compiler
- Easy configuration via YAML file (or via GUI)
- Remote automatic self upgrade capacity
- Included upgrade server ready to run in production
## About
So, a new backup solution out of nowhere, packed with too much features for it's own good ? Not really !
NPBackup relies on the well known [restic](https://restic.net) backup program, which has been battle proven for years.
While restic is a fanstastic program, NPBackup expands restic by offering a wider set of features.
## Quickstart
You may install npbackup via PyPI or use the pre-built executables.
### Prebuilt executables
On linux, copy `npbackup` executable to `/usr/local/bin` and make it executable via `chmod +x /usr/local/bin/npbackup`. Any distribution with glibc >= 2.17 should do.
On Windows, you can directly execute `npbackup.exe` or use `NPBackupInstaller.exe` to install NPBackup into program files and create a run schedule.
The x64 binary is compatible with Windows 10+. The x86 binary is compatible with windows Vista and higher. On those old systems, you might need to install Visual C runtime 2015.
### PyPI installation
If you don't want to use the pre-built executables, you can install via pip with `pip install npbackup`.
Python requirement: 3.6+
Note that if you want to use the GUI, you'll also need to install tkinter via `yum install python-tkinter` or `apt install python3-tk`.
### Setup
Copy the example config from model `examples/npbackup.conf.dist` into the directory where npbackup is installed.
Also copy the `excludes` directory if you plan to use the prefilled bigger exclusion lists for your backups.
You can adjust the parameters directly in the file, or via a config GUI by launching `npbackup --config-file=npbackup.conf --config-gui`
Once configured, you can launch manual backups via `npbackup --backup`. Those can be scheduled.
Windows schedule is created automatically by the installer program. On Linux, you'll have to create a cronjob or a systemd timer.
Since NPBackup is configured to only proceed with backups when no recent backups are detected, you should consider scheduling npbackup executions quite often.
The default schedule should be somewhere around 15 minutes.
You can use `npbackup --list` or the GUI to list backups.
The GUI allows an end user to check current backups & restore files.rom backups:
The YAML configuration file encrypts sensible data so the end user doesn't have to know repository URI or password.
## Quickstart GUI
Just run the npbackup executable and configure it.
Prebuilt binaries can be found [here](https://github.com/netinvent/npbackup/releases)
Main minimalistic interface allows to:
- List current backups
- Launch a manual backup
- See if last backup is recent enough
Restore window allows to browse through backups and select what files to restore.
Configuration allows to edit the YAML configuration files directly as end user
Backup process is interactive when GUI is used
**Security**
NPBackup' security model relies on symmetric encryption of all sensible data that allows to access a repository.
In order to achieve this, NPBackup contains an AES-KEY that can be set:
- at compile time
- at run time via an AES-KEY file
Please note that right clicking on "<encrypted data>" in the configuration GUI will allow to decrypt that data, by prompting a backup admin password.
That password is set at compile-time and should be different depending on the organization.
This allows a system admin to see repo URI and passwords, without leaving this information available on the computer.
The configuration file should never be world readable, as one could change the backup admin password, allowing to decrypt other parts of the conf file.
## The difficulty of laptop backups
As a matter of fact, laptop backups are the hardest. No one can predict when a laptop is on, and if it has access to internet.
Creating a backup strategy in those cases isn't a simple task.
NPBackup solves this by checking every 15 minutes if a backup newer than 24h exists.
If so, nothing is done. In the case no recent backup exists, a backup is immediately launched.
OF course, both time options are configurable.
In order to avoid sluggish user experience while backing up, process and io priority can be lowered.
Once done, NPBackup can send backup results in Prometheus format directly to a push gateway, for prometheus to catch them.
## A good server backup solution
Server backups can be achieved by setting up a scheduled task / cron job.
Of course, since NPBackup supports pre-exec and post-exec commands, it can be used to backup various services like virtual hosts or databases where snapshot/dump operations are required.
When run on a server, prometheus support can be shifted from a push gateway to a file, which will be picked up by node_exporter file collector.:
## Monitoring
NPBackup includes full prometheus support, including grafana dashboard.
On servers, we'll configure a prometheus file that gets written on each backup, and later can be collected by node_exporter.
On laptops, since we might be away from our usual network, we'll push the backup metrics to a remote push gateway which laters gets collected by prometheus itself.
The current NPBackup dashboard:
## End user expericence
While admin user experience is important, NPBackup also offers a GUI for end user experience, allowing to list all backup contents, navigate and restore files, without the need of an admin. The end user can also check if they have a recent backup completed, and launch backups manually if needed.
## Security
NPBackup inherits all security measures of it's backup backend (currently restic with AES-256 client side encryption including metadata), append only mode REST server backend.
On top of those, NPBackup itself encrypts sensible information like the repo uri and password, as well as the metrics http username and password.
This ensures that end users can restore data without the need to know any password, without compromising a secret. Note that in order to use this function, one needs to use the compiled version of NPBackup, so AES-256 keys are never exposed. Internally, NPBackup never directly uses the AES-256 key, so even a memory dump won't be enough to get the key.
## Upgrade server
NPBackup comes with integrated auto upgrade function that will run regardless of program failures, in order to lessen the maintenance burden.
The upgrade server runs a python asgi web server with integrated HTTP basic authentication, that can further be put behind an SSL proxy like HaProxy.
## Compilation
In order to fully protect the AES key that is needed to support NPBackup, one can compile the program with Nuitka.
Compiling needs restic binary for the target platform in `RESTIC_SOURCE_FILES` folder, files must be named `restic_{version}_{platform}_{arch}[.extension]` like provided by restic.net or [github](github.com/restic/restic)
Linux binaries need to be made executable in the `RESTIC_SOURCE_FILES` folder.
You'll need to change the default AES key in `secrets.py`, see the documentation in the file itself.
Compile options are available in `compile.py`.
We maintain a special 32 bit binary for Windows 7 which allows to backup those old machines until they get replaced.
We also compile our linux target on RHEL 7 in order to be compatible with reasonably old distributions (>= glibc 2.17).
arm and arm64 builds are compiled on Debian stretch for use with glibc > 2.24.
Additionnaly, arm builds are compiled without GUI support since they're supposed to fit on smaller devices like NAS / Raspberries.
## Smart shield, antivirus and reputation
Official binaries for Windows provided by NetInvent are signed with a certificate, allowing to gain trust and reputation in antivirus analysis.
Also, official binaries are compiled using Nuitka Commercial grade, which is more secure in storing secrets.
Pre-compiled builds for Windows have been code signed with NetInvent's EV certificate, using [windows_tools.signtool](github.com/netinvent/windows_tools)
Signing on a Windows machine with Windows SDK installed:
```
from windows_tools.signtool import SignTool
signer = SignTool()
signer.sign(r"c:\path\to\executable", bitness=64)
```
## Misc
NPBackup supports internationalization and automatically detects system's locale.
Still, locale can be overrided via an environment variable, eg on Linux:
```
export NPBACKUP_LOCALE=en
```
On Windows:
```
set NPBACKUP_LOCALE=en
```
## Special thanks
- Thanks to the Restic Team without which this program would not be possible
- Thanks to https://github.com/Krutyi-4el who packaged i18nice internationalization for us
- Special thanks to the BTS SIO 2nd year class of 2022 at Lycee Marillac / Perpignan who volunteered as GUI Q&A team
Raw data
{
"_id": null,
"home_page": "https://github.com/netinvent/npbackup",
"name": "npbackup",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "shell,backup,deduplication,compression,prometheus,windows,linux,gui,cli",
"author": "NetInvent - Orsiris de Jong",
"author_email": "contact@netinvent.fr",
"download_url": "https://files.pythonhosted.org/packages/55/a9/ac8bcf6ccc3334d3e3d7485fe642b8d28193b3dd6251efc89e5142d4ca6e/npbackup-2.2.1.tar.gz",
"platform": null,
"description": "[](https://opensource.org/licenses/GPL-3.0)\r\n[](http://isitmaintained.com/project/netinvent/npbackup \"Percentage of issues still open\")\r\n[](https://github.com/netinvent/npbackup/releases/latest)\r\n[](https://github.com/netinvent/npbackup/actions/workflows/pylint-windows.yaml)\r\n[](https://github.com/netinvent/npbackup/actions/workflows/pylint-linux.yaml)\r\n\r\n# NPBackup\r\n\r\nA secure and efficient file backup solution that fits both system administrators (CLI) and end users (GUI)\r\n\r\nWorks on x64 **Linux** , **NAS** solutions based on arm/arm64, **Windows** x64 and x86 and MacOS X.\r\n\r\n## Features\r\n\r\n- Data deduplication and fast zstd compression\r\n- Client side data encryption\r\n- Wide storage backend support\r\n - local files\r\n - SFTP\r\n - High performance HTTP REST server\r\n - Amazon S3/Minio/Wasabi\r\n - Blackblaze B2\r\n - Microsoft Azure Blob Storage\r\n - Google Cloud Storage\r\n - OpenStack Swift\r\n - Alibaba Cloud (Aliyun) Object Storage System (OSS)\r\n- Resume on interrupted backups\r\n- Full CLI interface for scheduled task usage\r\n - Checks for recent backups before launching a backup\r\n- End User GUI\r\n - Backups create, list, viewer and restore\r\n - Full configuration interface\r\n - Internationalization support (en, fr as of feb 2023)\r\n- Performance\r\n - Backup process and IO priority settings\r\n - Upload / download speed limits\r\n - Remote connectivity concurrency settings\r\n- Comes with full exclusion lists for Linux and Windows\r\n- First class prometheus support\r\n - Restic results metric generatioion\r\n - Grafana dashboard included\r\n - node_exporter file collector support\r\n - Optional push gateway metrics uploading\r\n- First class Windows support\r\n - VSS snapshots\r\n - Automatic cloud file exclusions (reparse points)\r\n - Windows pre-built executables\r\n - Windows installer\r\n- Additional security\r\n - repository uri / password, http metrics and upgrade server passwords are AES-256 encrypted\r\n - Encrypted data viewing requires additional password\r\n - AES-256 keys can't be guessed in executables thanks to Nuitka Commercial compiler\r\n- Easy configuration via YAML file (or via GUI)\r\n- Remote automatic self upgrade capacity\r\n - Included upgrade server ready to run in production\r\n\r\n## About\r\n\r\nSo, a new backup solution out of nowhere, packed with too much features for it's own good ? Not really !\r\n\r\nNPBackup relies on the well known [restic](https://restic.net) backup program, which has been battle proven for years.\r\nWhile restic is a fanstastic program, NPBackup expands restic by offering a wider set of features.\r\n\r\n## Quickstart\r\n\r\nYou may install npbackup via PyPI or use the pre-built executables.\r\n\r\n### Prebuilt executables\r\nOn linux, copy `npbackup` executable to `/usr/local/bin` and make it executable via `chmod +x /usr/local/bin/npbackup`. Any distribution with glibc >= 2.17 should do.\r\n\r\nOn Windows, you can directly execute `npbackup.exe` or use `NPBackupInstaller.exe` to install NPBackup into program files and create a run schedule.\r\nThe x64 binary is compatible with Windows 10+. The x86 binary is compatible with windows Vista and higher. On those old systems, you might need to install Visual C runtime 2015.\r\n\r\n### PyPI installation\r\n\r\nIf you don't want to use the pre-built executables, you can install via pip with `pip install npbackup`.\r\n\r\nPython requirement: 3.6+\r\n\r\nNote that if you want to use the GUI, you'll also need to install tkinter via `yum install python-tkinter` or `apt install python3-tk`.\r\n\r\n### Setup\r\n\r\nCopy the example config from model `examples/npbackup.conf.dist` into the directory where npbackup is installed. \r\nAlso copy the `excludes` directory if you plan to use the prefilled bigger exclusion lists for your backups.\r\n\r\nYou can adjust the parameters directly in the file, or via a config GUI by launching `npbackup --config-file=npbackup.conf --config-gui`\r\n\r\nOnce configured, you can launch manual backups via `npbackup --backup`. Those can be scheduled.\r\nWindows schedule is created automatically by the installer program. On Linux, you'll have to create a cronjob or a systemd timer.\r\n\r\nSince NPBackup is configured to only proceed with backups when no recent backups are detected, you should consider scheduling npbackup executions quite often.\r\nThe default schedule should be somewhere around 15 minutes.\r\n\r\nYou can use `npbackup --list` or the GUI to list backups.\r\n\r\nThe GUI allows an end user to check current backups & restore files.rom backups:\r\n\r\nThe YAML configuration file encrypts sensible data so the end user doesn't have to know repository URI or password.\r\n\r\n## Quickstart GUI\r\n\r\nJust run the npbackup executable and configure it.\r\nPrebuilt binaries can be found [here](https://github.com/netinvent/npbackup/releases)\r\n\r\n\r\n\r\nMain minimalistic interface allows to: \r\n - List current backups\r\n - Launch a manual backup\r\n - See if last backup is recent enough\r\n\r\n\r\n\r\nRestore window allows to browse through backups and select what files to restore.\r\n\r\n\r\n\r\nConfiguration allows to edit the YAML configuration files directly as end user\r\n\r\n\r\n\r\nBackup process is interactive when GUI is used\r\n\r\n**Security**\r\nNPBackup' security model relies on symmetric encryption of all sensible data that allows to access a repository. \r\nIn order to achieve this, NPBackup contains an AES-KEY that can be set:\r\n - at compile time\r\n - at run time via an AES-KEY file\r\n\r\nPlease note that right clicking on \"<encrypted data>\" in the configuration GUI will allow to decrypt that data, by prompting a backup admin password.\r\nThat password is set at compile-time and should be different depending on the organization.\r\n\r\nThis allows a system admin to see repo URI and passwords, without leaving this information available on the computer.\r\n\r\nThe configuration file should never be world readable, as one could change the backup admin password, allowing to decrypt other parts of the conf file.\r\n\r\n## The difficulty of laptop backups\r\n\r\nAs a matter of fact, laptop backups are the hardest. No one can predict when a laptop is on, and if it has access to internet.\r\nCreating a backup strategy in those cases isn't a simple task.\r\n\r\nNPBackup solves this by checking every 15 minutes if a backup newer than 24h exists.\r\nIf so, nothing is done. In the case no recent backup exists, a backup is immediately launched.\r\nOF course, both time options are configurable.\r\nIn order to avoid sluggish user experience while backing up, process and io priority can be lowered.\r\nOnce done, NPBackup can send backup results in Prometheus format directly to a push gateway, for prometheus to catch them.\r\n\r\n## A good server backup solution\r\n\r\nServer backups can be achieved by setting up a scheduled task / cron job.\r\n\r\nOf course, since NPBackup supports pre-exec and post-exec commands, it can be used to backup various services like virtual hosts or databases where snapshot/dump operations are required.\r\nWhen run on a server, prometheus support can be shifted from a push gateway to a file, which will be picked up by node_exporter file collector.:\r\n\r\n## Monitoring\r\n\r\nNPBackup includes full prometheus support, including grafana dashboard.\r\nOn servers, we'll configure a prometheus file that gets written on each backup, and later can be collected by node_exporter.\r\n\r\nOn laptops, since we might be away from our usual network, we'll push the backup metrics to a remote push gateway which laters gets collected by prometheus itself.\r\n\r\nThe current NPBackup dashboard:\r\n\r\n\r\n## End user expericence\r\n\r\nWhile admin user experience is important, NPBackup also offers a GUI for end user experience, allowing to list all backup contents, navigate and restore files, without the need of an admin. The end user can also check if they have a recent backup completed, and launch backups manually if needed.\r\n\r\n## Security\r\n\r\nNPBackup inherits all security measures of it's backup backend (currently restic with AES-256 client side encryption including metadata), append only mode REST server backend.\r\n\r\nOn top of those, NPBackup itself encrypts sensible information like the repo uri and password, as well as the metrics http username and password. \r\nThis ensures that end users can restore data without the need to know any password, without compromising a secret. Note that in order to use this function, one needs to use the compiled version of NPBackup, so AES-256 keys are never exposed. Internally, NPBackup never directly uses the AES-256 key, so even a memory dump won't be enough to get the key.\r\n\r\n## Upgrade server\r\n\r\nNPBackup comes with integrated auto upgrade function that will run regardless of program failures, in order to lessen the maintenance burden. \r\nThe upgrade server runs a python asgi web server with integrated HTTP basic authentication, that can further be put behind an SSL proxy like HaProxy.\r\n\r\n## Compilation\r\n\r\nIn order to fully protect the AES key that is needed to support NPBackup, one can compile the program with Nuitka.\r\nCompiling needs restic binary for the target platform in `RESTIC_SOURCE_FILES` folder, files must be named `restic_{version}_{platform}_{arch}[.extension]` like provided by restic.net or [github](github.com/restic/restic)\r\nLinux binaries need to be made executable in the `RESTIC_SOURCE_FILES` folder.\r\n\r\nYou'll need to change the default AES key in `secrets.py`, see the documentation in the file itself.\r\n\r\nCompile options are available in `compile.py`.\r\n\r\nWe maintain a special 32 bit binary for Windows 7 which allows to backup those old machines until they get replaced.\r\n\r\nWe also compile our linux target on RHEL 7 in order to be compatible with reasonably old distributions (>= glibc 2.17).\r\n\r\narm and arm64 builds are compiled on Debian stretch for use with glibc > 2.24.\r\nAdditionnaly, arm builds are compiled without GUI support since they're supposed to fit on smaller devices like NAS / Raspberries.\r\n\r\n## Smart shield, antivirus and reputation\r\n\r\nOfficial binaries for Windows provided by NetInvent are signed with a certificate, allowing to gain trust and reputation in antivirus analysis. \r\nAlso, official binaries are compiled using Nuitka Commercial grade, which is more secure in storing secrets. \r\n\r\nPre-compiled builds for Windows have been code signed with NetInvent's EV certificate, using [windows_tools.signtool](github.com/netinvent/windows_tools) \r\nSigning on a Windows machine with Windows SDK installed:\r\n```\r\nfrom windows_tools.signtool import SignTool\r\nsigner = SignTool()\r\nsigner.sign(r\"c:\\path\\to\\executable\", bitness=64)\r\n```\r\n\r\n## Misc\r\n\r\nNPBackup supports internationalization and automatically detects system's locale.\r\nStill, locale can be overrided via an environment variable, eg on Linux:\r\n```\r\nexport NPBACKUP_LOCALE=en\r\n```\r\nOn Windows:\r\n```\r\nset NPBACKUP_LOCALE=en\r\n```\r\n\r\n## Special thanks\r\n\r\n- Thanks to the Restic Team without which this program would not be possible\r\n- Thanks to https://github.com/Krutyi-4el who packaged i18nice internationalization for us\r\n- Special thanks to the BTS SIO 2nd year class of 2022 at Lycee Marillac / Perpignan who volunteered as GUI Q&A team\r\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "One fits all solution for deduplicated and compressed backups on servers and laptops",
"version": "2.2.1",
"project_urls": {
"Homepage": "https://github.com/netinvent/npbackup"
},
"split_keywords": [
"shell",
"backup",
"deduplication",
"compression",
"prometheus",
"windows",
"linux",
"gui",
"cli"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13d3573f2874943270d505347d11d92065ab19af5ab6999d7d537024d380279c",
"md5": "0da88ca6b7b62ebedafe493572a8f378",
"sha256": "b01d6bd425dad7c23bff5299dd65ca295af2fe2458034e99d79d305b05e387e0"
},
"downloads": -1,
"filename": "npbackup-2.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0da88ca6b7b62ebedafe493572a8f378",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 100835,
"upload_time": "2023-09-07T23:05:01",
"upload_time_iso_8601": "2023-09-07T23:05:01.211405Z",
"url": "https://files.pythonhosted.org/packages/13/d3/573f2874943270d505347d11d92065ab19af5ab6999d7d537024d380279c/npbackup-2.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "55a9ac8bcf6ccc3334d3e3d7485fe642b8d28193b3dd6251efc89e5142d4ca6e",
"md5": "90881b11d5a5b817e19c6a41f5422bfe",
"sha256": "927b15448afba81f16e153122f72f1b929534d5526a9be69bd78dfe55d5d9f5e"
},
"downloads": -1,
"filename": "npbackup-2.2.1.tar.gz",
"has_sig": false,
"md5_digest": "90881b11d5a5b817e19c6a41f5422bfe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 96525,
"upload_time": "2023-09-07T23:05:03",
"upload_time_iso_8601": "2023-09-07T23:05:03.676673Z",
"url": "https://files.pythonhosted.org/packages/55/a9/ac8bcf6ccc3334d3e3d7485fe642b8d28193b3dd6251efc89e5142d4ca6e/npbackup-2.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-07 23:05:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "netinvent",
"github_project": "npbackup",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "npbackup"
}
JSON
