# Netbox Network Importer
Tool to analyze and synchronize an existing network with a Netbox (SOT).
The main use cases for the network importer:
- Import an existing network into a Netbox
- ~~Check the differences between the running network and the Netbox~~
---
## Install
1. Clone repository
- `git clone git@gitlab.cesnet.cz:701/done/netbox_network_importer.git`
2. Go into the repository
- `cd netbox_network_importer `
3. Create `virtualenv`
- `python3 -m venv venv`
3. Activate it
- `source venv/bin/activate`
3. Install the tool
- `pip install .`
---
## Pre-requisite
To operate, the Netbox Network Importer is dependent on the following items:
- Access to Netbox API
- Access to Network Devices via SSH
- Created Custom Field on Netbox
```json
{
"display": "Ignore Importer",
"content_types": [
"dcim.interface",
],
"type": {
"value": "boolean",
"label": "Boolean (true/false)"
},
"object_type": null,
"data_type": "boolean",
"name": "ignore_importer",
"label": "Ignore Importer",
"description": "Will be ignored when processed by Network Importer",
"required": true,
"filter_logic": {
"value": "exact",
"label": "Exact"
},
"default": false,
"weight": 100,
}
```
---
## Inventory
A device inventory must be available in NetBox.
To be able to connect to the device the following information needs to be defined :
- Primary ip address
- Platform (must be a one of the following ['iosxr', 'ios', 'iosxe'])
---
## Configuration file
The information to connect to NetBox must be provided via the configuration file. The configuration file below presents all available options that can be provided to control the behavior of the Netbox Network Importer.
Netbox Network Importer tries to find the configuration file at:
- `/home/<user>/.config/netbox_network_importer/config.yml`
- `/etc/netbox_network_importer/config.yml`
- Or you can set path to configuration file like:
- `netbox_network_importer --configs path_to_dir_with_config/ synchronize`
### Complete Configuration Example
```yaml
---
netbox:
NETBOX_API_TOKEN: "your_api_token_here"
NETBOX_INSTANCE_URL: "https://your-netbox-instance.com/"
NETBOX_GRAPHQL_URL: "https://your-netbox-instance.com/graphql/"
tacacs:
TACACS_USERNAME: "your_username"
TACACS_PASSWORD: "your_password"
config:
# Logging Configuration
LOG_DIR: "logs" # Directory for log files
LOG_LEVEL: "INFO" # Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
# Output Configuration
OUTPUT_DIR: "outputs" # Directory for result files (JSON/HTML)
# Network Behavior
SUPPRESS_CONNECTION_ERRORS: true # Suppress verbose NAPALM/NETCONF/SSH logs
NORNIR_WORKERS: 20 # Number of parallel workers (default: 20)
# Result Filtering
FILTER_EMPTY_UPDATE_RESULTS: true # Filter UPDATE actions with empty diff
FILTER_EMPTY_TASK_RESULTS: true # Filter tasks with no results
# Output Formatting
USE_NORNIR_RICH_PRINT: true # Use rich formatting (true) vs plain text (false)
```
### Configuration Options Explained
#### Logging Options
- **`LOG_LEVEL`**: Controls verbosity (DEBUG for detailed info, CRITICAL for errors only)
- **`LOG_DIR`**: Where log files are stored (creates automatically)
- **`SUPPRESS_CONNECTION_ERRORS`**: When `true`, hides verbose network library logs
#### Performance Options
- **`NORNIR_WORKERS`**: Number of devices processed in parallel (adjust based on your system)
#### Output Control
- **`FILTER_EMPTY_UPDATE_RESULTS`**: Hides UPDATE operations with no actual changes
- **`FILTER_EMPTY_TASK_RESULTS`**: Hides tasks that produced no results
- **`USE_NORNIR_RICH_PRINT`**: Choose between colored rich output or plain text
#### Log Files Generated
- **`logs/output.log`**: Main application log (rotated at 5MB, kept 10 days)
- **`logs/errors.log`**: Error-only log (rotated at 5MB, kept 30 days)
- **`logs/debug.log`**: Detailed debug log (only when LOG_LEVEL=DEBUG, kept 3 days)
---
## Execute
The Netbox Network Importer runs only in apply mode and provides detailed logging and result filtering.
- In apply mode, the NETBOX will be updated with:
- Device serial numbers
- Network interfaces
- IP addresses
- VLANs
- Link Aggregation Groups (LAGs)
- Interface bandwidths
### Apply Mode
The Netbox Network Importer will attempt to create/update or delete all elements in the Netbox that do not match what has been observed in the network.
#### Device Filtering Rules
It will ignore devices on certain circumstances:
- Device `status` must be `Active`, otherwise the device is skipped
- Device `primary_ip` must be properly set, otherwise the device is skipped
- Device `platform` must include one of the following values `[ios, iosxr, iosxe]`, otherwise the device is skipped
#### Interface Filtering Rules
- If interface has attribute `Ignore Importer` set to `True`, then the processing on the Interface will be skipped and ignored
#### Result Filtering (Configurable)
- **Empty Updates**: When `FILTER_EMPTY_UPDATE_RESULTS: true`, UPDATE actions with no actual changes are hidden from output
- **Empty Tasks**: When `FILTER_EMPTY_TASK_RESULTS: true`, tasks that produce no results are hidden from output
- **Output Format**: When `USE_NORNIR_RICH_PRINT: true`, uses colored rich formatting; when `false`, uses plain text
**Execution Examples**:
- `netbox_network_importer synchronize` # runs the importer on all active devices
- `netbox_network_importer synchronize -d Rxx` # runs importer on device Rxx from Netbox
- `netbox_network_importer synchronize -p iosxe` # runs importer on all devices with platform iosxe
- `netbox_network_importer synchronize -d Rxx -d Ryy` # runs importer on devices Rxx and Ryy
**Logging and Output**:
- Real-time progress is shown in the console
- Detailed logs are written to `logs/output.log`
- Errors are separately logged to `logs/errors.log`
- Results are saved as JSON and HTML in the `outputs/` directory
---
### Help
```
~/netbox_network_importer$ netbox_network_importer --help
Usage: netbox_network_importer [OPTIONS] COMMAND [ARGS]...
Options:
-c, --configs PATH path to folder with configurations
--help Show this message and exit.
Commands:
pyats Connect to device using pyats and pynetbox and print parsed command
synchronize Run set of Nornir task to update data in Netbox from network devices
synchronize-device Synchronize specific device data in Netbox from network
~/netbox_network_importer$ netbox_network_importer synchronize --help
Usage: netbox_network_importer synchronize [OPTIONS]
Run set of Nornir task to update data in Netbox from network devices
Args:
devices (str): device name filter, can be used multiple times
platforms (str): platform name filter, can be used multiple times
Returns:
dict: Dictionary of hosts, it's actions and results
Options:
-d, --devices TEXT Run on specified devices
-p, --platforms TEXT Run on specified platforms
--help Show this message and exit.
```
### Troubleshooting
#### Configuration Issues
- Check that `config.yml` exists in one of the expected locations
- Verify NETBOX_API_TOKEN has proper permissions
- Ensure NETBOX_INSTANCE_URL is accessible
#### Logging Issues
- Check that `LOG_DIR` path is writable
- Use `LOG_LEVEL: "DEBUG"` for detailed troubleshooting
- Set `SUPPRESS_CONNECTION_ERRORS: false` to see network connection details
#### Performance Issues
- Adjust `NORNIR_WORKERS` (reduce for slower systems, increase for faster ones)
- Use result filtering to reduce output volume
- Monitor log files for bottlenecks
---
Raw data
{
"_id": null,
"home_page": "https://gitlab.cesnet.cz/701/done/netbox_network_importer",
"name": "netbox-network-importer",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.6",
"maintainer_email": null,
"keywords": "netbox, network",
"author": "Jan Krupa",
"author_email": "jan.krupa@cesnet.cz",
"download_url": "https://files.pythonhosted.org/packages/40/01/4a717d32926305cebaec9b0dc8c0f0f260ecff707b3117c87703a98d8ddd/netbox_network_importer-1.1.7.tar.gz",
"platform": null,
"description": "# Netbox Network Importer\n\nTool to analyze and synchronize an existing network with a Netbox (SOT).\n\nThe main use cases for the network importer: \n - Import an existing network into a Netbox\n - ~~Check the differences between the running network and the Netbox~~\n\n---\n## Install\n1. Clone repository\n - `git clone git@gitlab.cesnet.cz:701/done/netbox_network_importer.git`\n2. Go into the repository\n - `cd netbox_network_importer `\n3. Create `virtualenv`\n - `python3 -m venv venv`\n3. Activate it\n - `source venv/bin/activate`\n3. Install the tool\n - `pip install .`\n---\n## Pre-requisite\nTo operate, the Netbox Network Importer is dependent on the following items:\n- Access to Netbox API\n- Access to Network Devices via SSH\n- Created Custom Field on Netbox\n```json\n {\n \"display\": \"Ignore Importer\",\n \"content_types\": [\n \"dcim.interface\",\n ],\n \"type\": {\n \"value\": \"boolean\",\n \"label\": \"Boolean (true/false)\"\n },\n \"object_type\": null,\n \"data_type\": \"boolean\",\n \"name\": \"ignore_importer\",\n \"label\": \"Ignore Importer\",\n \"description\": \"Will be ignored when processed by Network Importer\",\n \"required\": true,\n \"filter_logic\": {\n \"value\": \"exact\",\n \"label\": \"Exact\"\n },\n \"default\": false,\n \"weight\": 100,\n }\n```\n---\n## Inventory\nA device inventory must be available in NetBox. \n\nTo be able to connect to the device the following information needs to be defined :\n- Primary ip address\n- Platform (must be a one of the following ['iosxr', 'ios', 'iosxe'])\n---\n## Configuration file\nThe information to connect to NetBox must be provided via the configuration file. The configuration file below presents all available options that can be provided to control the behavior of the Netbox Network Importer.\n\nNetbox Network Importer tries to find the configuration file at:\n- `/home/<user>/.config/netbox_network_importer/config.yml` \n- `/etc/netbox_network_importer/config.yml`\n- Or you can set path to configuration file like:\n - `netbox_network_importer --configs path_to_dir_with_config/ synchronize`\n\n### Complete Configuration Example\n```yaml\n---\nnetbox:\n NETBOX_API_TOKEN: \"your_api_token_here\"\n NETBOX_INSTANCE_URL: \"https://your-netbox-instance.com/\"\n NETBOX_GRAPHQL_URL: \"https://your-netbox-instance.com/graphql/\"\ntacacs:\n TACACS_USERNAME: \"your_username\"\n TACACS_PASSWORD: \"your_password\"\nconfig:\n # Logging Configuration\n LOG_DIR: \"logs\" # Directory for log files\n LOG_LEVEL: \"INFO\" # Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL\n \n # Output Configuration \n OUTPUT_DIR: \"outputs\" # Directory for result files (JSON/HTML)\n \n # Network Behavior\n SUPPRESS_CONNECTION_ERRORS: true # Suppress verbose NAPALM/NETCONF/SSH logs\n NORNIR_WORKERS: 20 # Number of parallel workers (default: 20)\n \n # Result Filtering\n FILTER_EMPTY_UPDATE_RESULTS: true # Filter UPDATE actions with empty diff\n FILTER_EMPTY_TASK_RESULTS: true # Filter tasks with no results\n \n # Output Formatting\n USE_NORNIR_RICH_PRINT: true # Use rich formatting (true) vs plain text (false)\n```\n\n### Configuration Options Explained\n\n#### Logging Options\n- **`LOG_LEVEL`**: Controls verbosity (DEBUG for detailed info, CRITICAL for errors only)\n- **`LOG_DIR`**: Where log files are stored (creates automatically)\n- **`SUPPRESS_CONNECTION_ERRORS`**: When `true`, hides verbose network library logs\n\n#### Performance Options \n- **`NORNIR_WORKERS`**: Number of devices processed in parallel (adjust based on your system)\n\n#### Output Control\n- **`FILTER_EMPTY_UPDATE_RESULTS`**: Hides UPDATE operations with no actual changes\n- **`FILTER_EMPTY_TASK_RESULTS`**: Hides tasks that produced no results\n- **`USE_NORNIR_RICH_PRINT`**: Choose between colored rich output or plain text\n\n#### Log Files Generated\n- **`logs/output.log`**: Main application log (rotated at 5MB, kept 10 days)\n- **`logs/errors.log`**: Error-only log (rotated at 5MB, kept 30 days) \n- **`logs/debug.log`**: Detailed debug log (only when LOG_LEVEL=DEBUG, kept 3 days)\n---\n## Execute\nThe Netbox Network Importer runs only in apply mode and provides detailed logging and result filtering.\n\n- In apply mode, the NETBOX will be updated with:\n - Device serial numbers\n - Network interfaces\n - IP addresses\n - VLANs \n - Link Aggregation Groups (LAGs)\n - Interface bandwidths\n\n### Apply Mode\nThe Netbox Network Importer will attempt to create/update or delete all elements in the Netbox that do not match what has been observed in the network.\n\n#### Device Filtering Rules\nIt will ignore devices on certain circumstances:\n- Device `status` must be `Active`, otherwise the device is skipped\n- Device `primary_ip` must be properly set, otherwise the device is skipped\n- Device `platform` must include one of the following values `[ios, iosxr, iosxe]`, otherwise the device is skipped\n\n#### Interface Filtering Rules\n- If interface has attribute `Ignore Importer` set to `True`, then the processing on the Interface will be skipped and ignored\n\n#### Result Filtering (Configurable)\n- **Empty Updates**: When `FILTER_EMPTY_UPDATE_RESULTS: true`, UPDATE actions with no actual changes are hidden from output\n- **Empty Tasks**: When `FILTER_EMPTY_TASK_RESULTS: true`, tasks that produce no results are hidden from output\n- **Output Format**: When `USE_NORNIR_RICH_PRINT: true`, uses colored rich formatting; when `false`, uses plain text\n\n**Execution Examples**:\n- `netbox_network_importer synchronize` # runs the importer on all active devices\n- `netbox_network_importer synchronize -d Rxx` # runs importer on device Rxx from Netbox\n- `netbox_network_importer synchronize -p iosxe` # runs importer on all devices with platform iosxe\n- `netbox_network_importer synchronize -d Rxx -d Ryy` # runs importer on devices Rxx and Ryy\n\n**Logging and Output**:\n- Real-time progress is shown in the console\n- Detailed logs are written to `logs/output.log`\n- Errors are separately logged to `logs/errors.log`\n- Results are saved as JSON and HTML in the `outputs/` directory\n---\n\n### Help\n```\n~/netbox_network_importer$ netbox_network_importer --help\nUsage: netbox_network_importer [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n -c, --configs PATH path to folder with configurations\n --help Show this message and exit.\n\nCommands:\n pyats Connect to device using pyats and pynetbox and print parsed command\n synchronize Run set of Nornir task to update data in Netbox from network devices\n synchronize-device Synchronize specific device data in Netbox from network\n\n~/netbox_network_importer$ netbox_network_importer synchronize --help\nUsage: netbox_network_importer synchronize [OPTIONS]\n\n Run set of Nornir task to update data in Netbox from network devices\n\n Args: \n devices (str): device name filter, can be used multiple times\n platforms (str): platform name filter, can be used multiple times\n Returns: \n dict: Dictionary of hosts, it's actions and results\n\nOptions:\n -d, --devices TEXT Run on specified devices\n -p, --platforms TEXT Run on specified platforms\n --help Show this message and exit.\n```\n\n### Troubleshooting\n\n#### Configuration Issues\n- Check that `config.yml` exists in one of the expected locations\n- Verify NETBOX_API_TOKEN has proper permissions\n- Ensure NETBOX_INSTANCE_URL is accessible\n\n#### Logging Issues\n- Check that `LOG_DIR` path is writable\n- Use `LOG_LEVEL: \"DEBUG\"` for detailed troubleshooting\n- Set `SUPPRESS_CONNECTION_ERRORS: false` to see network connection details\n\n#### Performance Issues\n- Adjust `NORNIR_WORKERS` (reduce for slower systems, increase for faster ones)\n- Use result filtering to reduce output volume\n- Monitor log files for bottlenecks\n---\n",
"bugtrack_url": null,
"license": null,
"summary": "Poll data from devices and store them into Netbox",
"version": "1.1.7",
"project_urls": {
"Homepage": "https://gitlab.cesnet.cz/701/done/netbox_network_importer"
},
"split_keywords": [
"netbox",
" network"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "40014a717d32926305cebaec9b0dc8c0f0f260ecff707b3117c87703a98d8ddd",
"md5": "e43ee4d33b2c1ae4d7cefc8a25897450",
"sha256": "6d13774adda6cc5e650fb24050182afad207012beb22140da98871f96a7c00d9"
},
"downloads": -1,
"filename": "netbox_network_importer-1.1.7.tar.gz",
"has_sig": false,
"md5_digest": "e43ee4d33b2c1ae4d7cefc8a25897450",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.6",
"size": 40443,
"upload_time": "2025-10-21T06:31:54",
"upload_time_iso_8601": "2025-10-21T06:31:54.807048Z",
"url": "https://files.pythonhosted.org/packages/40/01/4a717d32926305cebaec9b0dc8c0f0f260ecff707b3117c87703a98d8ddd/netbox_network_importer-1.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-21 06:31:54",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "netbox-network-importer"
}
JSON