# TVM Multi Scan Exporter
Tenable Vulnerability Management offers
the [Scan Export API](https://pytenable.readthedocs.io/en/stable/api/io/scans.html#tenable.io.scans.ScansAPI.export) to
export vulnerabilities from individual scans,
and [Vulnerability Exports API](https://pytenable.readthedocs.io/en/stable/api/io/exports.html#tenable.io.exports.api.ExportsAPI.vulns)
to export vulnerabilities from all the scans, after they have been collapsed into an asset entity. This library allows
the users of Tenable VM to gather vulnerabilities from their scan data, before the vulnerabilities are collapsed into
an Asset entity by using existing public APIs. We call this TVM Multi Scan Exporter.
> ⚠️ **Disclaimer**
> Export aggregation is performed entirely on your machine.
> Please make sure your system has enough memory and disk space to complete the process successfully.
Users can create the following types of aggregated scan exports.
1. `CSV` - A single CSV file will be created in the same directory of the script you write.
2. `JSON` - A single JSON file will be created in the same directory of the script you write.
3. `Parquet` - A single Apache Parquet file will be created in the same directory of the script you write.
4. `Write to DB` - The result of the export will be written to an external database
- Users need to provide database credentials to a real DB, and the results of the exports are dumped into it. The
database needs to be created by the user before using this library. A table will be automatically created.
- > ⚠️ **Disclaimer**
Make sure the columns stay the same each time you use this export option. If the schema changes between exports,
the library will raise an error.
## How to use this library.
### Optional Prerequisites
If you're writing to an external database, you need to install the driver for the database you are using.
For running MS SQL Server on the Mac, for example, you would want to do the following. You can pick the version
compatible with the version of the DB you're running.
```
brew install msodbcsql17
```
For Linux, you may
follow [this](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16&tabs=alpine18-install%2Calpine17-install%2Cdebian8-install%2Credhat7-13-install%2Crhel7-offline)
guide to install the driver.
This library can be configured in two ways
1. As a Method Argument.
2. Using a `TOML` Configuration file. (This is the recommended approach)
### 1. Configuring TvmMultiScanExporter Programmatically
This library provides a single class, `TvmMultiScanExporter`, with a single method, `export`, which accepts a `Config`
object containing all the necessary export configuration.
Below is a detailed overview of the supported configuration options available in this library.
#### Config
1. `scan_name` (str) Scan Name to export. A case-insensitive contains check will be run on this value. This is a *
*mandatory** field.
2. `allowed_memory_gb` (int) Allowed memory for the Duck DB process on the client. This is a **mandatory** field.
3. `workers` (int) Parallelism for running exports. Defaults to 1. This value should be between 1 and 10.
4. `window` (`Window`) Window to export scans.
5. `columns` (`List[str]`) Columns to include in the export. These are header values from CSV Scan Exports. It already
defaults to a common list of fields.
6. `output_location` (str) The folder where the export files will be downloaded. This is an **optional** field.
The default value is the current working directory.
7. `export_type`: (`ExportType`) Type of export. The supported values are: `Csv()`, `Json()`, `Parquet()`,
and `WriteToDatabase()`. The `WriteToDatabase` object takes in the config to connect to an external database. When
using `WriteToDatabase()`, users need to configure the `EXTERNAL_DB_PASSWORD` environment variable.
#### Window
1. `since` (int) Unix Timestamp of the start time of the window to export scans. Defaults to 24 hours in the past.
2. `upto` (int) Unix timestamp of the end time of the window. Defaults to current time.
#### WriteToDatabase
1. `user` (str): The username for connecting to the external database.
2. `host_address` (str): The server name or IP address of the external database.
3. `port` (int): The port number used to connect to the database.
4. `database_name` (str): The name of the database. This should already exist within the database.
5. `table_name` (str): The name of the table where exported data will be written. This table may or may not already
exist,
but the schema of any existing table should match the schema of the export (i.e., the columns being exported).
6. `engine` (DatabaseEngine): The type of database engine. Currently, we support MS SQL Server by default.
7. `write_batch_size` (int): The number of rows included in each batch. The default is set to 100.
Here's how you would use it.
1. Create an instance of `TenableScanAggregator` after setting these environmental variables: `TVM_ACCESS_KEY`
and `TVM_SECRET_KEY`
```python
exporter = TvmMultiScanExporter()
```
From here, all you need to do is to call the `export` method with the required `Config`. Here are some examples:
##### How to create a CSV / JSON / Parquet
```python
exporter.export(
Config(
scan_name="nessus-agent-1-asset-hour-9",
allowed_memory_gb=12,
export_type=Csv(), # or Json() or Parquet()
workers=1
)
)
```
##### How to create a "Write-to-External-DB Export"
When using the `WriteToDatabase` export type, you must provide the database password as an environment variable. The
library will automatically look for the following environment variable when running the script:
- `EXTERNAL_DB_PASSWORD`: The password for connecting to the external database.
Make sure to set this environment variable before executing the script to ensure a successful connection.
```python
exporter.export(
Config(
scan_name="nessus-agent-1-asset-hour-9",
allowed_memory_gb=12,
export_type=WriteToDatabase(
user="your-db-username",
host_address="db-server-name",
port=1433,
database_name="your-db-name",
table_name="your-table-name",
),
workers=1
)
)
```
### 2. Configuring TenableScanAggregator with a Configuration file (Recommended)
In this case, the Python file will be the smallest. All necessary information can be provided to the library using a
file named `config.toml` placed in the same directory as the script. It follows a similar structure as the `Config`
class. Here's an example `config.toml` file.
Your Tenable Vulnerability Management access and secret keys can be configured as environment variables. The library
will look for the following variables when you run the script.
1. `TVM_ACCESS_KEY` TVM Access Key
2. `TVM_SECRET_KEY` TVM Secret Key
The name and location of this configuration file can also be customized, as detailed in the following sections of this
document.
When using the `write_to_database` export type, you need to provide the database password as an environment variable.
The library will look for the additional environmental variable: `EXTERNAL_DB_PASSWORD` when you run the script.
> ⚠️ **Note**
> The configuration in the `toml` file can be overridden by providing a `Config` object to the `export` method.
```toml
# Required fields
scan_name = "" # scan name to filter by
allowed_memory_gb = 12 # tweak as necessary
# Optional fields: uncomment as needed
# workers = 1
# columns = ["Asset UUID", "IP Address", "Plugin ID", "Severity"]
# output_location = "path/to/download/folder"
# Uncomment for CSV / JSON / Parquet export types
#export_type = "csv"
#export_type = "json"
#export_type = "parquet"
# Keep the following object to write to database. Comment out if not needed
[export_type]
type = "write_to_database"
user = "your-db-username"
host_address = "db-server-name"
port = 1433
database_name = "your-db-name"
table_name = "your-table-name"
engine = "MsSqlServer" # Optional; defaults to MsSqlServer
write_batch_size = 500 # Optional; defaults to 100
# uncomment the following object if you want to override the window
#[window]
#since = 1742898712
#upto = 1710086402
```
If the config file is placed in a different directory / under a different name, the path to the same can be provided as
a property to the `TvmMultiScanExporter` object.
```
project/
├── run.py ← your script file
├── config/
│ └── settings.toml
```
```python
TvmMultiScanExporter(config_path="config/settings.toml").export()
```
Examples for both the approaches can be found inside the `examples` directory in this repo.
## Configuring `tvm_url` for Different TVM Environments
The `TvmMultiScanExporter` class accepts an optional `tvm_url` property, allowing users to configure the library for
different TVM environments. By default, the library uses the standard TVM URL (cloud.tenable.com),
but this can be overridden as needed.
Example: Configuring `tvm_url`
You can specify the tvm_url when creating an instance of `TvmMultiScanExporter`:
```python
from tvm_multi_scan_exporter.aggregated_scan_exporter import TvmMultiScanExporter
exporter = TvmMultiScanExporter(tvm_url="https://custom-tvm-environment.example.com")
```
Raw data
{
"_id": null,
"home_page": null,
"name": "tvm-multi-scan-exporter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "tenable, tenable vulnerability management",
"author": "Tenable, Inc.",
"author_email": "Exports Team <exports-integration@tenable.com>",
"download_url": "https://files.pythonhosted.org/packages/46/b4/5ea3d11454bbc05e4560cc4a5dea478f7abf0916260fbc01616f4ffa37c1/tvm_multi_scan_exporter-0.1.1.tar.gz",
"platform": null,
"description": "# TVM Multi Scan Exporter\n\nTenable Vulnerability Management offers\nthe [Scan Export API](https://pytenable.readthedocs.io/en/stable/api/io/scans.html#tenable.io.scans.ScansAPI.export) to\nexport vulnerabilities from individual scans,\nand [Vulnerability Exports API](https://pytenable.readthedocs.io/en/stable/api/io/exports.html#tenable.io.exports.api.ExportsAPI.vulns)\nto export vulnerabilities from all the scans, after they have been collapsed into an asset entity. This library allows\nthe users of Tenable VM to gather vulnerabilities from their scan data, before the vulnerabilities are collapsed into\nan Asset entity by using existing public APIs. We call this TVM Multi Scan Exporter.\n\n> \u26a0\ufe0f **Disclaimer** \n> Export aggregation is performed entirely on your machine. \n> Please make sure your system has enough memory and disk space to complete the process successfully.\n\nUsers can create the following types of aggregated scan exports.\n\n1. `CSV` - A single CSV file will be created in the same directory of the script you write.\n2. `JSON` - A single JSON file will be created in the same directory of the script you write.\n3. `Parquet` - A single Apache Parquet file will be created in the same directory of the script you write.\n4. `Write to DB` - The result of the export will be written to an external database\n - Users need to provide database credentials to a real DB, and the results of the exports are dumped into it. The\n database needs to be created by the user before using this library. A table will be automatically created.\n - > \u26a0\ufe0f **Disclaimer**\n Make sure the columns stay the same each time you use this export option. If the schema changes between exports,\n the library will raise an error.\n\n## How to use this library.\n\n### Optional Prerequisites\n\nIf you're writing to an external database, you need to install the driver for the database you are using.\n\nFor running MS SQL Server on the Mac, for example, you would want to do the following. You can pick the version\ncompatible with the version of the DB you're running.\n\n```\nbrew install msodbcsql17\n```\n\nFor Linux, you may\nfollow [this](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16&tabs=alpine18-install%2Calpine17-install%2Cdebian8-install%2Credhat7-13-install%2Crhel7-offline)\nguide to install the driver.\n\nThis library can be configured in two ways\n\n1. As a Method Argument.\n2. Using a `TOML` Configuration file. (This is the recommended approach)\n\n### 1. Configuring TvmMultiScanExporter Programmatically\n\nThis library provides a single class, `TvmMultiScanExporter`, with a single method, `export`, which accepts a `Config`\nobject containing all the necessary export configuration.\n\nBelow is a detailed overview of the supported configuration options available in this library.\n\n#### Config\n\n1. `scan_name` (str) Scan Name to export. A case-insensitive contains check will be run on this value. This is a *\n *mandatory** field.\n2. `allowed_memory_gb` (int) Allowed memory for the Duck DB process on the client. This is a **mandatory** field.\n3. `workers` (int) Parallelism for running exports. Defaults to 1. This value should be between 1 and 10.\n4. `window` (`Window`) Window to export scans.\n5. `columns` (`List[str]`) Columns to include in the export. These are header values from CSV Scan Exports. It already\n defaults to a common list of fields.\n6. `output_location` (str) The folder where the export files will be downloaded. This is an **optional** field.\n The default value is the current working directory.\n7. `export_type`: (`ExportType`) Type of export. The supported values are: `Csv()`, `Json()`, `Parquet()`,\n and `WriteToDatabase()`. The `WriteToDatabase` object takes in the config to connect to an external database. When\n using `WriteToDatabase()`, users need to configure the `EXTERNAL_DB_PASSWORD` environment variable.\n\n#### Window\n\n1. `since` (int) Unix Timestamp of the start time of the window to export scans. Defaults to 24 hours in the past.\n2. `upto` (int) Unix timestamp of the end time of the window. Defaults to current time.\n\n#### WriteToDatabase\n\n1. `user` (str): The username for connecting to the external database.\n2. `host_address` (str): The server name or IP address of the external database.\n3. `port` (int): The port number used to connect to the database.\n4. `database_name` (str): The name of the database. This should already exist within the database.\n5. `table_name` (str): The name of the table where exported data will be written. This table may or may not already\n exist,\n but the schema of any existing table should match the schema of the export (i.e., the columns being exported).\n6. `engine` (DatabaseEngine): The type of database engine. Currently, we support MS SQL Server by default.\n7. `write_batch_size` (int): The number of rows included in each batch. The default is set to 100.\n\nHere's how you would use it.\n\n1. Create an instance of `TenableScanAggregator` after setting these environmental variables: `TVM_ACCESS_KEY`\n and `TVM_SECRET_KEY`\n\n```python\nexporter = TvmMultiScanExporter()\n```\n\nFrom here, all you need to do is to call the `export` method with the required `Config`. Here are some examples:\n\n##### How to create a CSV / JSON / Parquet\n\n```python\nexporter.export(\n Config(\n scan_name=\"nessus-agent-1-asset-hour-9\",\n allowed_memory_gb=12,\n export_type=Csv(), # or Json() or Parquet()\n workers=1\n )\n)\n```\n\n##### How to create a \"Write-to-External-DB Export\"\n\nWhen using the `WriteToDatabase` export type, you must provide the database password as an environment variable. The\nlibrary will automatically look for the following environment variable when running the script:\n\n- `EXTERNAL_DB_PASSWORD`: The password for connecting to the external database.\n\nMake sure to set this environment variable before executing the script to ensure a successful connection.\n\n```python\nexporter.export(\n Config(\n scan_name=\"nessus-agent-1-asset-hour-9\",\n allowed_memory_gb=12,\n export_type=WriteToDatabase(\n user=\"your-db-username\",\n host_address=\"db-server-name\",\n port=1433,\n database_name=\"your-db-name\",\n table_name=\"your-table-name\",\n ),\n workers=1\n )\n)\n```\n\n### 2. Configuring TenableScanAggregator with a Configuration file (Recommended)\n\nIn this case, the Python file will be the smallest. All necessary information can be provided to the library using a\nfile named `config.toml` placed in the same directory as the script. It follows a similar structure as the `Config`\nclass. Here's an example `config.toml` file.\n\nYour Tenable Vulnerability Management access and secret keys can be configured as environment variables. The library\nwill look for the following variables when you run the script.\n\n1. `TVM_ACCESS_KEY` TVM Access Key\n2. `TVM_SECRET_KEY` TVM Secret Key\n\nThe name and location of this configuration file can also be customized, as detailed in the following sections of this\ndocument.\n\nWhen using the `write_to_database` export type, you need to provide the database password as an environment variable.\nThe library will look for the additional environmental variable: `EXTERNAL_DB_PASSWORD` when you run the script.\n\n> \u26a0\ufe0f **Note** \n> The configuration in the `toml` file can be overridden by providing a `Config` object to the `export` method.\n\n```toml\n# Required fields\nscan_name = \"\" # scan name to filter by\nallowed_memory_gb = 12 # tweak as necessary\n\n# Optional fields: uncomment as needed\n\n# workers = 1\n# columns = [\"Asset UUID\", \"IP Address\", \"Plugin ID\", \"Severity\"]\n\n# output_location = \"path/to/download/folder\"\n\n# Uncomment for CSV / JSON / Parquet export types\n#export_type = \"csv\"\n#export_type = \"json\"\n#export_type = \"parquet\"\n\n# Keep the following object to write to database. Comment out if not needed\n[export_type]\ntype = \"write_to_database\"\nuser = \"your-db-username\"\nhost_address = \"db-server-name\"\nport = 1433\ndatabase_name = \"your-db-name\"\ntable_name = \"your-table-name\"\nengine = \"MsSqlServer\" # Optional; defaults to MsSqlServer\nwrite_batch_size = 500 # Optional; defaults to 100\n\n# uncomment the following object if you want to override the window\n#[window]\n#since = 1742898712\n#upto = 1710086402\n\n```\n\nIf the config file is placed in a different directory / under a different name, the path to the same can be provided as\na property to the `TvmMultiScanExporter` object.\n\n```\nproject/\n\u251c\u2500\u2500 run.py \u2190 your script file\n\u251c\u2500\u2500 config/\n\u2502 \u2514\u2500\u2500 settings.toml\n\n```\n\n```python\nTvmMultiScanExporter(config_path=\"config/settings.toml\").export()\n```\n\nExamples for both the approaches can be found inside the `examples` directory in this repo.\n\n## Configuring `tvm_url` for Different TVM Environments\n\nThe `TvmMultiScanExporter` class accepts an optional `tvm_url` property, allowing users to configure the library for\ndifferent TVM environments. By default, the library uses the standard TVM URL (cloud.tenable.com),\nbut this can be overridden as needed.\n\nExample: Configuring `tvm_url`\nYou can specify the tvm_url when creating an instance of `TvmMultiScanExporter`:\n\n```python\nfrom tvm_multi_scan_exporter.aggregated_scan_exporter import TvmMultiScanExporter\n\nexporter = TvmMultiScanExporter(tvm_url=\"https://custom-tvm-environment.example.com\")\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python library to aggregate Scan Exports into a single export",
"version": "0.1.1",
"project_urls": {
"Bug Tracker": "https://github.com/tenable/tvm_multi_scan_exporter/issues",
"Homepage": "https://github.com/tenable/tvm_multi_scan_exporter/blob/main/README.md"
},
"split_keywords": [
"tenable",
" tenable vulnerability management"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7b96aba6d379c06c6143e6f38ace86e5919e7b40af17306d2d1d633cdb45d74a",
"md5": "415b7412fa713ed645c5e1bd139ccd37",
"sha256": "14aef900f4981435fa12ddda6fc28b773210e46587d8292b2cd92268f8ee303f"
},
"downloads": -1,
"filename": "tvm_multi_scan_exporter-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "415b7412fa713ed645c5e1bd139ccd37",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 20129,
"upload_time": "2025-07-09T07:17:37",
"upload_time_iso_8601": "2025-07-09T07:17:37.085494Z",
"url": "https://files.pythonhosted.org/packages/7b/96/aba6d379c06c6143e6f38ace86e5919e7b40af17306d2d1d633cdb45d74a/tvm_multi_scan_exporter-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "46b45ea3d11454bbc05e4560cc4a5dea478f7abf0916260fbc01616f4ffa37c1",
"md5": "d9976348a09bf8ebd648c8dadd63bdbe",
"sha256": "b064b790d8efc0d6c39856d5871fa39d7d732024a8f7e45078c2f0e308456243"
},
"downloads": -1,
"filename": "tvm_multi_scan_exporter-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "d9976348a09bf8ebd648c8dadd63bdbe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 26740,
"upload_time": "2025-07-09T07:17:38",
"upload_time_iso_8601": "2025-07-09T07:17:38.034599Z",
"url": "https://files.pythonhosted.org/packages/46/b4/5ea3d11454bbc05e4560cc4a5dea478f7abf0916260fbc01616f4ffa37c1/tvm_multi_scan_exporter-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-09 07:17:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tenable",
"github_project": "tvm_multi_scan_exporter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tvm-multi-scan-exporter"
}
JSON
