| Name | greenbone-feed-sync JSON |
| Version |
24.1.1
JSON |
| download |
| home_page | |
| Summary | A tool for downloading the Greenbone Community Feed |
| upload_time | 2024-01-31 12:27:55 |
| maintainer | |
| docs_url | None |
| author | Björn Ricks |
| requires_python | >=3.9,<4.0 |
| license | GPL-3.0-or-later |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# greenbone-feed-sync <!-- omit in toc -->
New script for downloading the Greenbone Community Feed
- [Installation](#installation)
- [Requirements](#requirements)
- [Install using pipx](#install-using-pipx)
- [Install using pip](#install-using-pip)
- [Usage](#usage)
- [Usage on Kali Linux](#usage-on-kali-linux)
- [Command Completion](#command-completion)
- [Settings](#settings)
- [verbose](#verbose)
- [quiet](#quiet)
- [config](#config)
- [private-directory](#private-directory)
- [compression-level](#compression-level)
- [type](#type)
- [feed-url](#feed-url)
- [destination-prefix](#destination-prefix)
- [gvmd-data-destination](#gvmd-data-destination)
- [gvmd-data-url](#gvmd-data-url)
- [notus-destination](#notus-destination)
- [notus-url](#notus-url)
- [nasl-destination](#nasl-destination)
- [nasl-url](#nasl-url)
- [scap-data-destination](#scap-data-destination)
- [scap-data-url](#scap-data-url)
- [cert-data-destination](#cert-data-destination)
- [cert-data-url](#cert-data-url)
- [report-formats-destination](#report-formats-destination)
- [report-formats-url](#report-formats-url)
- [scan-configs-destination](#scan-configs-destination)
- [scan-configs-url](#scan-configs-url)
- [port-lists-destination](#port-lists-destination)
- [port-lists-url](#port-lists-url)
- [gvmd-lock-file](#gvmd-lock-file)
- [openvas-lock-file](#openvas-lock-file)
- [fail-fast](#fail-fast)
- [no-wait](#no-wait)
- [wait-interval](#wait-interval)
- [rsync-timeout](#rsync-timeout)
- [group](#group)
- [user](#user)
- [greenbone-enterprise-feed-key](#greenbone-enterprise-feed-key)
- [Config](#config-1)
- [Development](#development)
- [Maintainer](#maintainer)
- [Contributing](#contributing)
- [License](#license)
## Installation
### Requirements
Python 3.9 and later is supported.
`greenbone-feed-sync` requires the `rsync` tool being installed and available
within the `PATH`.
On Debian based Distributions like Ubuntu and Kali `rsync` can be installed via
sudo apt install rsync
### Install using pipx
You can install the latest release of **greenbone-feed-sync** from the Python
Package Index (pypi) using [pipx]
python3 -m pipx install greenbone-feed-sync
On Debian based Distributions like Ubuntu and Kali `pipx` itself can be
installed via
sudo apt install pipx
### Install using pip
**NOTE:** The `pip install` command does no longer work out-of-the-box in newer
distributions like Ubuntu 23.04 or Debian 12 because of [PEP 668](https://peps.python.org/pep-0668).
Please use the [installation via pipx](#install-using-pipx) instead.
You can install the latest release of **greenbone-feed-sync** from the
Python Package Index ([pypi]) using [pip]
python3 -m pip install greenbone-feed-sync
## Usage
Most of the time you should just run the script without any arguments to
download the new data for all necessary feed types
**NOTE:** See details about [usage on Kali Linux](#usage-on-kali-linux)
sudo greenbone-feed-sync
To get verbose progress output during the data download you might increase the
verbosity
sudo greenbone-feed-sync -vvv
If the script is run in a cron job the output can be turned off via
sudo greenbone-feed-sync --quiet
To download only a specific feed content the `--type` argument can be used
sudo greenbone-feed-sync --type nvt
Run `--help` to get information about all possible types and additional argument
options
greenbone-feed-sync --help
## Usage on Kali Linux
When running `greenbone-feed-sync` as root user, for example via sudo, the
actual user and group of the process are changed to the `gvm` user and group via
[seteuid](https://docs.python.org/3/library/os.html#os.seteuid). This is done to
ensure that [`gvmd`](https://github.com/greenbone/gvmd) and
[`openvas-scanner`](https://github.com/greenbone/openvas-scanner) can read the
downloaded file contents.
When using the Greenbone Community Edition installed via packages on Kali Linux
a different user and group are used. They are both named `_gvm` instead.
Therefore the [group](#group) and [user](#user) settings need to be adjusted.
This can be done by using a [config file](#config-1).
sudo mkdir /etc/gvm
sudo chmod +r /etc/gvm
cat <<EOF | sudo tee /etc/gvm/greenbone-feed-sync.toml
[greenbone-feed-sync]
user="_gvm"
group="_gvm"
EOF
sudo chmod +r /etc/gvm/greenbone-feed-sync.toml
## Command Completion
`greenbone-feed-sync` comes with support for command line completion in bash
and zsh.
Setup for bash:
```bash
echo "source ~/.greenbone-feed-sync-complete.bash" >> ~/.bashrc
greenbone-feed-sync --print-completion bash > ~/.greenbone-feed-sync-complete.bash
```
Alternatively, you can use the result of the completion command directly with
the eval function of your bash shell:
```bash
eval "$(greenbone-feed-sync --print-completion bash)"
```
Setup for zsh:
```zsh
echo 'fpath=("$HOME/.zsh.d" $fpath)' >> ~/.zsh
mkdir -p ~/.zsh.d/
greenbone-feed-sync --print-completion zsh > ~/.zsh.d/_greenbone_feed_sync
```
## Settings
The `greenbone-feed-sync` script is adjustable for all kind of purposes and very
flexible which content gets downloaded. Most likely you will never need to
adjust the settings because the defaults will suffice. Changing the settings
is only required for experts and testing purposes.
### verbose
| Name | Value |
| -------------------- | -------------------------------------------- |
| CLI Argument | `--verbose, -v` |
| Config Variable | verbose |
| Environment Variable | `GREENBONE_FEED_SYNC_VERBOSE` |
| Default Value | 2 |
| Description | Log verbosity. `-vvv` for maximum verbosity. |
### quiet
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------- |
| CLI Argument | `--quiet` |
| Config Variable | |
| Environment Variable | |
| Default Value | |
| Description | Disable all log output. Same as setting `verbose` or `GREENBONE_FEED_SYNC_VERBOSE` to 0 |
### config
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------- |
| CLI Argument | `--config, -c` |
| Config Variable | |
| Environment Variable | |
| Default Value | `~/.config/greenbone-feed-sync.toml` and `/etc/gvm/greenbone-feed-sync.toml` |
| Description | TOML config file to load settings from. |
### private-directory
| Name | Value |
| -------------------- | ------------------------------------------------------------------------------------ |
| CLI Argument | `--private-directory` |
| Config Variable | private-directory |
| Environment Variable | `GREENBONE_FEED_SYNC_PRIVATE_DIRECTORY` |
| Default Value | |
| Description | (Sub-)Directory to exclude from the sync which will never get deleted automatically. |
### compression-level
| Name | Value |
| -------------------- | ----------------------------------------------------------------------- |
| CLI Argument | `--compression-level` |
| Config Variable | compression-level |
| Environment Variable | `GREENBONE_FEED_SYNC_COMPRESSION_LEVEL` |
| Default Value | 9 |
| Description | rsync compression level 0-9. (0 - no compression, 9 - high compression) |
### type
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--type` |
| Config Variable | |
| Environment Variable | |
| Default Value | all |
| Description | Specifies which feed data should be downloaded. Possible values are `all`, `nvt`/`nvts`, `gvmd-data`, `scap`, `cert`, `notus`, `nasl`, `report-format`/`report-formats`, `scan-config`/`scan-configs` or `port-list`/`port-lists`. |
### feed-url
| Name | Value |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | |
| Config Variable | feed-url |
| Environment Variable | `GREENBONE_FEED_SYNC_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community` |
| Description | URL to download the feed data from. Other URLs will be relative to this URL by default. For example using `rsync://example.com` as feed url the notus url will be `rsync://example.com/vulnerability-feed/22.04/vt-data/notus/`. |
### destination-prefix
| Name | Value |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | |
| Config Variable | destination-prefix |
| Environment Variable | `GREENBONE_FEED_SYNC_DESTINATION_PREFIX` |
| Default Value | `/var/lib/` |
| Description | Directory prefix to use for default feed data download destinations. Other download destinations will be relative to this path by default. For example using `/opt/lib` as destination prefix will change the default of the notus destination to `/opt/lib/notus`. |
### gvmd-data-destination
| Name | Value |
| -------------------- | ------------------------------------------- |
| CLI Argument | `--gvmd-data-destination` |
| Config Variable | gvmd-data-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_DATA_DESTINATION` |
| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/` |
| Description | Destination of the downloaded gvmd data. |
### gvmd-data-url
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------------- |
| CLI Argument | `--gvmd-data-url` |
| Config Variable | gvmd-data-url |
| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_DATA_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/` |
| Description | URL to download the gvmd data from. This includes scan-configs, report-formats and port-lists. |
### notus-destination
| Name | Value |
| -------------------- | ----------------------------------------- |
| CLI Argument | `--notus-destination` |
| Config Variable | notus-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_NOTUS_DESTINATION` |
| Default Value | `/var/lib/notus` |
| Description | Destination of the downloaded notus data. |
### notus-url
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------- |
| CLI Argument | `--notus-url` |
| Config Variable | notus-url |
| Environment Variable | `GREENBONE_FEED_SYNC_NOTUS_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/notus/` |
| Description | URL to download the notus data from. |
### nasl-destination
| Name | Value |
| -------------------- | ---------------------------------------- |
| CLI Argument | `--nasl-destination` |
| Config Variable | nasl-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_NASL_DESTINATION` |
| Default Value | `/var/lib/openvas/plugins` |
| Description | Destination of the downloaded nasl data. |
### nasl-url
| Name | Value |
| -------------------- | --------------------------------------------------------------------------------------- |
| CLI Argument | `--nasl-url` |
| Config Variable | nasl-url |
| Environment Variable | `GREENBONE_FEED_SYNC_NASL_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/nasl/` |
| Description | URL to download the nasl data from. |
### scap-data-destination
| Name | Value |
| -------------------- | ------------------------------------------- |
| CLI Argument | `--scap-data-destination` |
| Config Variable | scap-data-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_SCAP_DATA_DESTINATION` |
| Default Value | `/var/lib/gvm/scap-data` |
| Description | Destination of the downloaded SCAP data. |
### scap-data-url
| Name | Value |
| -------------------- | ----------------------------------------------------------------------------------- |
| CLI Argument | `--scap-data-url` |
| Config Variable | scap-data-url |
| Environment Variable | `GREENBONE_FEED_SYNC_SCAP_DATA_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/scap-data` |
| Description | URL to download the SCAP data from. |
### cert-data-destination
| Name | Value |
| -------------------- | ------------------------------------------- |
| CLI Argument | `--cert-data-destination` |
| Config Variable | cert-data-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_CERT_DATA_DESTINATION` |
| Default Value | `/var/lib/gvm/cert-data` |
| Description | Destination of the downloaded CERT data. |
### cert-data-url
| Name | Value |
| -------------------- | ----------------------------------------------------------------------------------- |
| CLI Argument | `--cert-data-url` |
| Config Variable | cert-data-url |
| Environment Variable | `GREENBONE_FEED_SYNC_CERT_DATA_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/cert-data` |
| Description | URL to download the CERT data from. |
### report-formats-destination
| Name | Value |
| -------------------- | ----------------------------------------------------- |
| CLI Argument | `--report-formats-destination` |
| Config Variable | report-formats-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_REPORT_FORMATS_DESTINATION` |
| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/report-formats` |
| Description | Destination of the downloaded report format data. |
### report-formats-url
| Name | Value |
| -------------------- | ------------------------------------------------------------------------------- |
| CLI Argument | `--report-formats-url` |
| Config Variable | report-formats-url |
| Environment Variable | `GREENBONE_FEED_SYNC_REPORT_FORMATS_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/report-formats` |
| Description | URL to download the report format data from. |
### scan-configs-destination
| Name | Value |
| -------------------- | --------------------------------------------------- |
| CLI Argument | `--scan-configs-destination` |
| Config Variable | scan-configs-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_SCAN_CONFIGS_DESTINATION` |
| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/scan-configs` |
| Description | Destination of the downloaded scan config data. |
### scan-configs-url
| Name | Value |
| -------------------- | ----------------------------------------------------------------------------- |
| CLI Argument | `--scan-configs-url` |
| Config Variable | scan-configs-url |
| Environment Variable | `GREENBONE_FEED_SYNC_SCAN_CONFIGS_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/scan-configs` |
| Description | URL to download the scan config data from. |
### port-lists-destination
| Name | Value |
| -------------------- | ------------------------------------------------- |
| CLI Argument | `--port-lists-destination` |
| Config Variable | port-lists-destination |
| Environment Variable | `GREENBONE_FEED_SYNC_PORT_LISTS_DESTINATION` |
| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/port-lists` |
| Description | Destination of the downloaded port list data. |
### port-lists-url
| Name | Value |
| -------------------- | --------------------------------------------------------------------------- |
| CLI Argument | `--port-lists-url` |
| Config Variable | port-lists-url |
| Environment Variable | `GREENBONE_FEED_SYNC_PORT_LISTS_URL` |
| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/port-lists` |
| Description | URL to download the port list data from. |
### gvmd-lock-file
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--gvmd-lock-file` |
| Config Variable | gvmd-lock-file |
| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_LOCK_FILE` |
| Default Value | `/var/lib/gvm/feed-update.lock` |
| Description | File to use for locking the feed synchronization for data loaded by the gvmd daemon. Used to avoid that more then one process accesses the feed data at the same time. |
### openvas-lock-file
| Name | Value |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--openvas-lock-file` |
| Config Variable | openvas-lock-file |
| Environment Variable | `GREENBONE_FEED_SYNC_OPENVAS_LOCK_FILE` |
| Default Value | `/var/lib/openvas/feed-update.lock` |
| Description | File to use for locking the feed synchronization for data loaded by the openvas scanner. Used to avoid that more then one process accesses the feed data at the same time. |
### fail-fast
| Name | Value |
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--fail-fast, --failfast` |
| Config Variable | fail-fast |
| Environment Variable | `GREENBONE_FEED_SYNC_FAIL_FAST` |
| Default Value | false |
| Description | Stop after a first error has occurred. Otherwise the script tries to download additional data if specified. |
### no-wait
| Name | Value |
| -------------------- | ------------------------------------------------- |
| CLI Argument | `--no-wait` |
| Config Variable | no-wait |
| Environment Variable | `GREENBONE_FEED_SYNC_NO_WAIT` |
| Default Value | false |
| Description | Fail directly if the lock file can't be acquired. |
### wait-interval
| Name | Value |
| -------------------- | ------------------------------------------------------------------------------------ |
| CLI Argument | `--wait-interval` |
| Config Variable | wait-interval |
| Environment Variable | `GREENBONE_FEED_SYNC_LOCK_WAIT_INTERVAL` |
| Default Value | 5 |
| Description | Time to wait in seconds after failed lock attempt before re-trying to lock the file. |
### rsync-timeout
| Name | Value |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--rsync-timeout` |
| Config Variable | rsync-timeout |
| Environment Variable | `GREENBONE_FEED_SYNC_RSYNC_TIMEOUT` |
| Default Value | |
| Description | Maximum I/O timeout in seconds used for rsync. If no data is transferred for the specified time then rsync will exit. By default no timeout is set and the rsync default will be used. |
### group
| Name | Value |
| -------------------- | ---------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--group` |
| Config Variable | group |
| Environment Variable | `GREENBONE_FEED_SYNC_GROUP` |
| Default Value | gvm |
| Description | If the greenbone-feed-sync script is run as root, the effective group is changed to this group name or ID. |
### user
| Name | Value |
| -------------------- | -------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--user` |
| Config Variable | user |
| Environment Variable | `GREENBONE_FEED_SYNC_USER` |
| Default Value | gvm |
| Description | If the greenbone-feed-sync script is run as root, the effective user is changed to this user name or ID. |
### greenbone-enterprise-feed-key
| Name | Value |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI Argument | `--greenbone-enterprise-feed-key` |
| Config Variable | greenbone-enterprise-feed-key |
| Environment Variable | `GREENBONE_FEED_SYNC_ENTERPRISE_FEED_KEY` |
| Default Value | /etc/gvm/greenbone-enterprise-feed-key |
| Description | File to read the Greenbone Enterprise Feed key from. The key gives access to additional vulnerability tests for enterprise software among other advantages. See [Greenbone Enterprise Feed and Greenbone Community Feed in Comparison](https://www.greenbone.net/en/feed-comparison/) for more details. The default URLs are adjusted according to the data in the key. If the key file does not exist it is ignored. |
## Config
It is possible to use a config file for loading the settings of the
`greenbone-feed-sync` script. The config file uses the [TOML] format. Without
explicitly passing a config file, `greenbone-feed-sync` tries to load
`~/.config/greenbone-feed-sync.toml` and if that file doesn't exist afterwards
`/etc/gvm/greenbone-feed-sync.toml`.
Example:
```toml
[greenbone-feed-sync]
destination-prefix = "/opt/greenbone-feed"
lock-file = "/opt/greenbone-feed.lock"
no-wait = true
```
## Development
**greenbone-feed-sync** uses [poetry] for its own dependency management and
build process.
First install poetry via pipx
python3 -m pipx install poetry
Afterwards run
poetry install
in the checkout directory of **greenbone-feed-sync** (the directory containing
the `pyproject.toml` file) to install all dependencies including the packages
only required for development.
Afterwards activate the git hooks for auto-formatting and linting via
[autohooks].
poetry run autohooks activate
Validate the activated git hooks by running
poetry run autohooks check
## Maintainer
This project is maintained by [Greenbone AG][Greenbone Networks]
## Contributing
Your contributions are highly appreciated. Please
[create a pull request](https://github.com/greenbone/greeenbon-feed-sync/pulls)
on GitHub. Bigger changes need to be discussed with the development team via the
[issues section at GitHub](https://github.com/greenbone/greenbone-feed-sync/issues)
first.
## License
Copyright (C) 2022-2024 [Greenbone AG][Greenbone Networks]
Licensed under the [GNU General Public License v3.0 or later](LICENSE).
[Greenbone Networks]: https://www.greenbone.net/
[poetry]: https://python-poetry.org/
[pip]: https://pip.pypa.io/
[pipx]: https://pypa.github.io/pipx/
[autohooks]: https://github.com/greenbone/autohooks
[TOML]: https://toml.io/
[pypi]: https://pypi.org
Raw data
{
"_id": null,
"home_page": "",
"name": "greenbone-feed-sync",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Bj\u00f6rn Ricks",
"author_email": "bjoern.ricks@greenbone.net",
"download_url": "https://files.pythonhosted.org/packages/bb/70/3fe268a899cf53066f70507ff616ff8c9ce6f461a77ec2559b369886eefc/greenbone_feed_sync-24.1.1.tar.gz",
"platform": null,
"description": "\n\n# greenbone-feed-sync <!-- omit in toc -->\n\nNew script for downloading the Greenbone Community Feed\n\n- [Installation](#installation)\n - [Requirements](#requirements)\n - [Install using pipx](#install-using-pipx)\n - [Install using pip](#install-using-pip)\n- [Usage](#usage)\n- [Usage on Kali Linux](#usage-on-kali-linux)\n- [Command Completion](#command-completion)\n- [Settings](#settings)\n - [verbose](#verbose)\n - [quiet](#quiet)\n - [config](#config)\n - [private-directory](#private-directory)\n - [compression-level](#compression-level)\n - [type](#type)\n - [feed-url](#feed-url)\n - [destination-prefix](#destination-prefix)\n - [gvmd-data-destination](#gvmd-data-destination)\n - [gvmd-data-url](#gvmd-data-url)\n - [notus-destination](#notus-destination)\n - [notus-url](#notus-url)\n - [nasl-destination](#nasl-destination)\n - [nasl-url](#nasl-url)\n - [scap-data-destination](#scap-data-destination)\n - [scap-data-url](#scap-data-url)\n - [cert-data-destination](#cert-data-destination)\n - [cert-data-url](#cert-data-url)\n - [report-formats-destination](#report-formats-destination)\n - [report-formats-url](#report-formats-url)\n - [scan-configs-destination](#scan-configs-destination)\n - [scan-configs-url](#scan-configs-url)\n - [port-lists-destination](#port-lists-destination)\n - [port-lists-url](#port-lists-url)\n - [gvmd-lock-file](#gvmd-lock-file)\n - [openvas-lock-file](#openvas-lock-file)\n - [fail-fast](#fail-fast)\n - [no-wait](#no-wait)\n - [wait-interval](#wait-interval)\n - [rsync-timeout](#rsync-timeout)\n - [group](#group)\n - [user](#user)\n - [greenbone-enterprise-feed-key](#greenbone-enterprise-feed-key)\n- [Config](#config-1)\n- [Development](#development)\n- [Maintainer](#maintainer)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Installation\n\n### Requirements\n\nPython 3.9 and later is supported.\n\n`greenbone-feed-sync` requires the `rsync` tool being installed and available\nwithin the `PATH`.\n\nOn Debian based Distributions like Ubuntu and Kali `rsync` can be installed via\n\n sudo apt install rsync\n\n### Install using pipx\n\nYou can install the latest release of **greenbone-feed-sync** from the Python\nPackage Index (pypi) using [pipx]\n\n python3 -m pipx install greenbone-feed-sync\n\nOn Debian based Distributions like Ubuntu and Kali `pipx` itself can be\ninstalled via\n\n sudo apt install pipx\n\n### Install using pip\n\n**NOTE:** The `pip install` command does no longer work out-of-the-box in newer\ndistributions like Ubuntu 23.04 or Debian 12 because of [PEP 668](https://peps.python.org/pep-0668).\nPlease use the [installation via pipx](#install-using-pipx) instead.\n\nYou can install the latest release of **greenbone-feed-sync** from the\nPython Package Index ([pypi]) using [pip]\n\n python3 -m pip install greenbone-feed-sync\n\n## Usage\n\nMost of the time you should just run the script without any arguments to\ndownload the new data for all necessary feed types\n\n**NOTE:** See details about [usage on Kali Linux](#usage-on-kali-linux)\n\n sudo greenbone-feed-sync\n\nTo get verbose progress output during the data download you might increase the\nverbosity\n\n sudo greenbone-feed-sync -vvv\n\n\nIf the script is run in a cron job the output can be turned off via\n\n sudo greenbone-feed-sync --quiet\n\n\nTo download only a specific feed content the `--type` argument can be used\n\n sudo greenbone-feed-sync --type nvt\n\nRun `--help` to get information about all possible types and additional argument\noptions\n\n greenbone-feed-sync --help\n\n## Usage on Kali Linux\n\nWhen running `greenbone-feed-sync` as root user, for example via sudo, the\nactual user and group of the process are changed to the `gvm` user and group via\n[seteuid](https://docs.python.org/3/library/os.html#os.seteuid). This is done to\nensure that [`gvmd`](https://github.com/greenbone/gvmd) and\n[`openvas-scanner`](https://github.com/greenbone/openvas-scanner) can read the\ndownloaded file contents.\n\nWhen using the Greenbone Community Edition installed via packages on Kali Linux\na different user and group are used. They are both named `_gvm` instead.\nTherefore the [group](#group) and [user](#user) settings need to be adjusted.\nThis can be done by using a [config file](#config-1).\n\n sudo mkdir /etc/gvm\n sudo chmod +r /etc/gvm\n cat <<EOF | sudo tee /etc/gvm/greenbone-feed-sync.toml\n [greenbone-feed-sync]\n user=\"_gvm\"\n group=\"_gvm\"\n EOF\n sudo chmod +r /etc/gvm/greenbone-feed-sync.toml\n\n## Command Completion\n\n`greenbone-feed-sync` comes with support for command line completion in bash\nand zsh.\n\nSetup for bash:\n\n```bash\necho \"source ~/.greenbone-feed-sync-complete.bash\" >> ~/.bashrc\ngreenbone-feed-sync --print-completion bash > ~/.greenbone-feed-sync-complete.bash\n```\n\nAlternatively, you can use the result of the completion command directly with\nthe eval function of your bash shell:\n\n```bash\neval \"$(greenbone-feed-sync --print-completion bash)\"\n```\n\nSetup for zsh:\n\n```zsh\necho 'fpath=(\"$HOME/.zsh.d\" $fpath)' >> ~/.zsh\nmkdir -p ~/.zsh.d/\ngreenbone-feed-sync --print-completion zsh > ~/.zsh.d/_greenbone_feed_sync\n```\n\n## Settings\n\nThe `greenbone-feed-sync` script is adjustable for all kind of purposes and very\nflexible which content gets downloaded. Most likely you will never need to\nadjust the settings because the defaults will suffice. Changing the settings\nis only required for experts and testing purposes.\n\n### verbose\n\n| Name | Value |\n| -------------------- | -------------------------------------------- |\n| CLI Argument | `--verbose, -v` |\n| Config Variable | verbose |\n| Environment Variable | `GREENBONE_FEED_SYNC_VERBOSE` |\n| Default Value | 2 |\n| Description | Log verbosity. `-vvv` for maximum verbosity. |\n\n### quiet\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------- |\n| CLI Argument | `--quiet` |\n| Config Variable | |\n| Environment Variable | |\n| Default Value | |\n| Description | Disable all log output. Same as setting `verbose` or `GREENBONE_FEED_SYNC_VERBOSE` to 0 |\n\n### config\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------- |\n| CLI Argument | `--config, -c` |\n| Config Variable | |\n| Environment Variable | |\n| Default Value | `~/.config/greenbone-feed-sync.toml` and `/etc/gvm/greenbone-feed-sync.toml` |\n| Description | TOML config file to load settings from. |\n\n### private-directory\n\n| Name | Value |\n| -------------------- | ------------------------------------------------------------------------------------ |\n| CLI Argument | `--private-directory` |\n| Config Variable | private-directory |\n| Environment Variable | `GREENBONE_FEED_SYNC_PRIVATE_DIRECTORY` |\n| Default Value | |\n| Description | (Sub-)Directory to exclude from the sync which will never get deleted automatically. |\n\n### compression-level\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------------------------- |\n| CLI Argument | `--compression-level` |\n| Config Variable | compression-level |\n| Environment Variable | `GREENBONE_FEED_SYNC_COMPRESSION_LEVEL` |\n| Default Value | 9 |\n| Description | rsync compression level 0-9. (0 - no compression, 9 - high compression) |\n\n### type\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--type` |\n| Config Variable | |\n| Environment Variable | |\n| Default Value | all |\n| Description | Specifies which feed data should be downloaded. Possible values are `all`, `nvt`/`nvts`, `gvmd-data`, `scap`, `cert`, `notus`, `nasl`, `report-format`/`report-formats`, `scan-config`/`scan-configs` or `port-list`/`port-lists`. |\n\n### feed-url\n\n| Name | Value |\n| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | |\n| Config Variable | feed-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community` |\n| Description | URL to download the feed data from. Other URLs will be relative to this URL by default. For example using `rsync://example.com` as feed url the notus url will be `rsync://example.com/vulnerability-feed/22.04/vt-data/notus/`. |\n\n### destination-prefix\n\n| Name | Value |\n| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | |\n| Config Variable | destination-prefix |\n| Environment Variable | `GREENBONE_FEED_SYNC_DESTINATION_PREFIX` |\n| Default Value | `/var/lib/` |\n| Description | Directory prefix to use for default feed data download destinations. Other download destinations will be relative to this path by default. For example using `/opt/lib` as destination prefix will change the default of the notus destination to `/opt/lib/notus`. |\n\n### gvmd-data-destination\n\n| Name | Value |\n| -------------------- | ------------------------------------------- |\n| CLI Argument | `--gvmd-data-destination` |\n| Config Variable | gvmd-data-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_DATA_DESTINATION` |\n| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/` |\n| Description | Destination of the downloaded gvmd data. |\n\n### gvmd-data-url\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------------- |\n| CLI Argument | `--gvmd-data-url` |\n| Config Variable | gvmd-data-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_DATA_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/` |\n| Description | URL to download the gvmd data from. This includes scan-configs, report-formats and port-lists. |\n\n### notus-destination\n\n| Name | Value |\n| -------------------- | ----------------------------------------- |\n| CLI Argument | `--notus-destination` |\n| Config Variable | notus-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_NOTUS_DESTINATION` |\n| Default Value | `/var/lib/notus` |\n| Description | Destination of the downloaded notus data. |\n\n### notus-url\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------- |\n| CLI Argument | `--notus-url` |\n| Config Variable | notus-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_NOTUS_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/notus/` |\n| Description | URL to download the notus data from. |\n\n### nasl-destination\n\n| Name | Value |\n| -------------------- | ---------------------------------------- |\n| CLI Argument | `--nasl-destination` |\n| Config Variable | nasl-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_NASL_DESTINATION` |\n| Default Value | `/var/lib/openvas/plugins` |\n| Description | Destination of the downloaded nasl data. |\n\n### nasl-url\n\n| Name | Value |\n| -------------------- | --------------------------------------------------------------------------------------- |\n| CLI Argument | `--nasl-url` |\n| Config Variable | nasl-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_NASL_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/nasl/` |\n| Description | URL to download the nasl data from. |\n\n### scap-data-destination\n\n| Name | Value |\n| -------------------- | ------------------------------------------- |\n| CLI Argument | `--scap-data-destination` |\n| Config Variable | scap-data-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_SCAP_DATA_DESTINATION` |\n| Default Value | `/var/lib/gvm/scap-data` |\n| Description | Destination of the downloaded SCAP data. |\n\n### scap-data-url\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------------------------------------- |\n| CLI Argument | `--scap-data-url` |\n| Config Variable | scap-data-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_SCAP_DATA_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/scap-data` |\n| Description | URL to download the SCAP data from. |\n\n### cert-data-destination\n\n| Name | Value |\n| -------------------- | ------------------------------------------- |\n| CLI Argument | `--cert-data-destination` |\n| Config Variable | cert-data-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_CERT_DATA_DESTINATION` |\n| Default Value | `/var/lib/gvm/cert-data` |\n| Description | Destination of the downloaded CERT data. |\n\n### cert-data-url\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------------------------------------- |\n| CLI Argument | `--cert-data-url` |\n| Config Variable | cert-data-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_CERT_DATA_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/vulnerability-feed/22.04/cert-data` |\n| Description | URL to download the CERT data from. |\n\n### report-formats-destination\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------- |\n| CLI Argument | `--report-formats-destination` |\n| Config Variable | report-formats-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_REPORT_FORMATS_DESTINATION` |\n| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/report-formats` |\n| Description | Destination of the downloaded report format data. |\n\n### report-formats-url\n\n| Name | Value |\n| -------------------- | ------------------------------------------------------------------------------- |\n| CLI Argument | `--report-formats-url` |\n| Config Variable | report-formats-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_REPORT_FORMATS_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/report-formats` |\n| Description | URL to download the report format data from. |\n\n### scan-configs-destination\n\n| Name | Value |\n| -------------------- | --------------------------------------------------- |\n| CLI Argument | `--scan-configs-destination` |\n| Config Variable | scan-configs-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_SCAN_CONFIGS_DESTINATION` |\n| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/scan-configs` |\n| Description | Destination of the downloaded scan config data. |\n\n### scan-configs-url\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------------------------------- |\n| CLI Argument | `--scan-configs-url` |\n| Config Variable | scan-configs-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_SCAN_CONFIGS_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/scan-configs` |\n| Description | URL to download the scan config data from. |\n\n### port-lists-destination\n\n| Name | Value |\n| -------------------- | ------------------------------------------------- |\n| CLI Argument | `--port-lists-destination` |\n| Config Variable | port-lists-destination |\n| Environment Variable | `GREENBONE_FEED_SYNC_PORT_LISTS_DESTINATION` |\n| Default Value | `/var/lib/gvm/data-objects/gvmd/22.04/port-lists` |\n| Description | Destination of the downloaded port list data. |\n\n### port-lists-url\n\n| Name | Value |\n| -------------------- | --------------------------------------------------------------------------- |\n| CLI Argument | `--port-lists-url` |\n| Config Variable | port-lists-url |\n| Environment Variable | `GREENBONE_FEED_SYNC_PORT_LISTS_URL` |\n| Default Value | `rsync://feed.community.greenbone.net/community/data-feed/22.04/port-lists` |\n| Description | URL to download the port list data from. |\n\n### gvmd-lock-file\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--gvmd-lock-file` |\n| Config Variable | gvmd-lock-file |\n| Environment Variable | `GREENBONE_FEED_SYNC_GVMD_LOCK_FILE` |\n| Default Value | `/var/lib/gvm/feed-update.lock` |\n| Description | File to use for locking the feed synchronization for data loaded by the gvmd daemon. Used to avoid that more then one process accesses the feed data at the same time. |\n\n### openvas-lock-file\n\n| Name | Value |\n| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--openvas-lock-file` |\n| Config Variable | openvas-lock-file |\n| Environment Variable | `GREENBONE_FEED_SYNC_OPENVAS_LOCK_FILE` |\n| Default Value | `/var/lib/openvas/feed-update.lock` |\n| Description | File to use for locking the feed synchronization for data loaded by the openvas scanner. Used to avoid that more then one process accesses the feed data at the same time. |\n\n### fail-fast\n\n| Name | Value |\n| -------------------- | ----------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--fail-fast, --failfast` |\n| Config Variable | fail-fast |\n| Environment Variable | `GREENBONE_FEED_SYNC_FAIL_FAST` |\n| Default Value | false |\n| Description | Stop after a first error has occurred. Otherwise the script tries to download additional data if specified. |\n\n### no-wait\n\n| Name | Value |\n| -------------------- | ------------------------------------------------- |\n| CLI Argument | `--no-wait` |\n| Config Variable | no-wait |\n| Environment Variable | `GREENBONE_FEED_SYNC_NO_WAIT` |\n| Default Value | false |\n| Description | Fail directly if the lock file can't be acquired. |\n\n### wait-interval\n\n| Name | Value |\n| -------------------- | ------------------------------------------------------------------------------------ |\n| CLI Argument | `--wait-interval` |\n| Config Variable | wait-interval |\n| Environment Variable | `GREENBONE_FEED_SYNC_LOCK_WAIT_INTERVAL` |\n| Default Value | 5 |\n| Description | Time to wait in seconds after failed lock attempt before re-trying to lock the file. |\n\n### rsync-timeout\n\n| Name | Value |\n| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--rsync-timeout` |\n| Config Variable | rsync-timeout |\n| Environment Variable | `GREENBONE_FEED_SYNC_RSYNC_TIMEOUT` |\n| Default Value | |\n| Description | Maximum I/O timeout in seconds used for rsync. If no data is transferred for the specified time then rsync will exit. By default no timeout is set and the rsync default will be used. |\n\n### group\n\n| Name | Value |\n| -------------------- | ---------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--group` |\n| Config Variable | group |\n| Environment Variable | `GREENBONE_FEED_SYNC_GROUP` |\n| Default Value | gvm |\n| Description | If the greenbone-feed-sync script is run as root, the effective group is changed to this group name or ID. |\n\n### user\n\n| Name | Value |\n| -------------------- | -------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--user` |\n| Config Variable | user |\n| Environment Variable | `GREENBONE_FEED_SYNC_USER` |\n| Default Value | gvm |\n| Description | If the greenbone-feed-sync script is run as root, the effective user is changed to this user name or ID. |\n\n### greenbone-enterprise-feed-key\n\n| Name | Value |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CLI Argument | `--greenbone-enterprise-feed-key` |\n| Config Variable | greenbone-enterprise-feed-key |\n| Environment Variable | `GREENBONE_FEED_SYNC_ENTERPRISE_FEED_KEY` |\n| Default Value | /etc/gvm/greenbone-enterprise-feed-key |\n| Description | File to read the Greenbone Enterprise Feed key from. The key gives access to additional vulnerability tests for enterprise software among other advantages. See [Greenbone Enterprise Feed and Greenbone Community Feed in Comparison](https://www.greenbone.net/en/feed-comparison/) for more details. The default URLs are adjusted according to the data in the key. If the key file does not exist it is ignored. |\n\n## Config\n\nIt is possible to use a config file for loading the settings of the\n`greenbone-feed-sync` script. The config file uses the [TOML] format. Without\nexplicitly passing a config file, `greenbone-feed-sync` tries to load\n`~/.config/greenbone-feed-sync.toml` and if that file doesn't exist afterwards\n`/etc/gvm/greenbone-feed-sync.toml`.\n\nExample:\n\n```toml\n[greenbone-feed-sync]\ndestination-prefix = \"/opt/greenbone-feed\"\nlock-file = \"/opt/greenbone-feed.lock\"\nno-wait = true\n```\n\n## Development\n\n**greenbone-feed-sync** uses [poetry] for its own dependency management and\nbuild process.\n\nFirst install poetry via pipx\n\n python3 -m pipx install poetry\n\nAfterwards run\n\n poetry install\n\nin the checkout directory of **greenbone-feed-sync** (the directory containing\nthe `pyproject.toml` file) to install all dependencies including the packages\nonly required for development.\n\nAfterwards activate the git hooks for auto-formatting and linting via\n[autohooks].\n\n poetry run autohooks activate\n\nValidate the activated git hooks by running\n\n poetry run autohooks check\n\n## Maintainer\n\nThis project is maintained by [Greenbone AG][Greenbone Networks]\n\n## Contributing\n\nYour contributions are highly appreciated. Please\n[create a pull request](https://github.com/greenbone/greeenbon-feed-sync/pulls)\non GitHub. Bigger changes need to be discussed with the development team via the\n[issues section at GitHub](https://github.com/greenbone/greenbone-feed-sync/issues)\nfirst.\n\n## License\n\nCopyright (C) 2022-2024 [Greenbone AG][Greenbone Networks]\n\nLicensed under the [GNU General Public License v3.0 or later](LICENSE).\n\n[Greenbone Networks]: https://www.greenbone.net/\n[poetry]: https://python-poetry.org/\n[pip]: https://pip.pypa.io/\n[pipx]: https://pypa.github.io/pipx/\n[autohooks]: https://github.com/greenbone/autohooks\n[TOML]: https://toml.io/\n[pypi]: https://pypi.org\n\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "A tool for downloading the Greenbone Community Feed",
"version": "24.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2e84ac4eab38ce3b73e7e9b053eded4c1976fdd80e1c3065bd448c56975b04b3",
"md5": "ca400c5d1af81ee37ab9b06e66df2c6f",
"sha256": "184c0529f236a02270784d8b1cef4fb14e3dfa98fc61e0f3187c897852202e41"
},
"downloads": -1,
"filename": "greenbone_feed_sync-24.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ca400c5d1af81ee37ab9b06e66df2c6f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<4.0",
"size": 32125,
"upload_time": "2024-01-31T12:27:53",
"upload_time_iso_8601": "2024-01-31T12:27:53.179667Z",
"url": "https://files.pythonhosted.org/packages/2e/84/ac4eab38ce3b73e7e9b053eded4c1976fdd80e1c3065bd448c56975b04b3/greenbone_feed_sync-24.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bb703fe268a899cf53066f70507ff616ff8c9ce6f461a77ec2559b369886eefc",
"md5": "a1d457378ad392e6ee8aed048d4b49fb",
"sha256": "e4e28cf303fd76a6128249748d2e5f355f00aee1f9cb0545ae98b09167997296"
},
"downloads": -1,
"filename": "greenbone_feed_sync-24.1.1.tar.gz",
"has_sig": false,
"md5_digest": "a1d457378ad392e6ee8aed048d4b49fb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<4.0",
"size": 59954,
"upload_time": "2024-01-31T12:27:55",
"upload_time_iso_8601": "2024-01-31T12:27:55.175844Z",
"url": "https://files.pythonhosted.org/packages/bb/70/3fe268a899cf53066f70507ff616ff8c9ce6f461a77ec2559b369886eefc/greenbone_feed_sync-24.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-31 12:27:55",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "greenbone-feed-sync"
}
