# Firewall Testing-Framework
A framework for testing and troubleshooting firewall Layer 3-4 rulesets.
**WARNING**: This project is still in the conception-phase.
----
## Goal / Why?
When having to administer IT infrastructure and networks - we will often have multiple firewalls in place.
Maintaining these might be time-consuming. You might also face some challenges:
* **Troubleshooting & Analysis**:
Even for senior network engineers it can be a challenge to find the source of an unexpected block/accept in large rulesets that are distributed across multiple systems and firewall vendors.
Infrastructure-as-Code does help to keep the rulesets in a consistent state - but it does not solve the issue of having to manually analyze/troubleshoot existing rulesets.
This project wants to provide one interface for simulating traffic over multiple firewall systems.
* **Automated Regression-Tests**:
Why would you want to do ruleset-regression-tests?
* If you utilize Infrastructure-as-Code and change-reviews for updating your ruleset you might want to
* You may want/need to periodically verify that the currently active rulesets actually allow/deny the traffic you expect
This can be a tedious task - you might overlook some edge-case.
* Especially when a ruleset is administered by teams of engineers over a long time period - it can be a challenge to:
* detect configuration errors/mistakes before they can be exploited
* make sure the design-choices for the ruleset are adhered to
How do regression-tests work?
* You define test-cases that simulate traffic over one or multiple firewalls
* You assert that the traffic was allowed/denied/rejected
* You might even want to assert that the traffic took a specific outbound route or was NATed to a specific IP
This way you can continuously extend these test-cases and easily verify that the currently active rulesets comply with them.
----
## Idea
Take a look at this topology:
<img src="https://raw.githubusercontent.com/O-X-L/firewall-testing-framework/refs/heads/latest/docs/source/_static/img/topology.svg" max-width="700"></img>
The flow is planned to be:
1. Either:
* manually pull the current config from the existing firewalls
* or utilize existing `pull-plugins` to do so (p.e. via API)
2. The vendor-specific configuration gets parsed by `translation-plugins` which output a standardized firewall config-schema.
3. The user provides a high-level `topology-config`
4. If automated tests should be run: The user needs to provide a `test-traffic config`
Else the user has the option to enter an interactive shell where traffic can be sent manually
5. The `firewall/network simulator`
* parses the provided config
* generates the network-topology
* finds where the packet originates from (or notifies the user if more information is required)
* finds the route the packet should take
* tests the traffic against the rulesets of firewalls that are hops of that route
Thanks already go to @MikPisula, the creator of the [MikPisula/packet-simulator](https://github.com/MikPisula/packet-simulator) for creating a simulator for netfilter (IPTables/NFTables) firewalls.
Also thanks to the [go-ftw (Web Application Firewall Testing Framework) project](https://github.com/coreruleset/go-ftw) that inspired us to support regression-tests.
----
## Principles
* **Strict separation of vendor-specific plugins** from the core traffic-simulator.
Plugins CAN be used to pull the current configuration (rulesets, interfaces, routes) from a firewall system, but admins should always be able to manually provide this information.
Some might not want to trust some 'nice-to-have' tool with access to their firewalls.
* The user should be able to choose the **output verbosity**.
We want to provide full transparency (*show every rule the traffic interacts with*) but if not required (*p.e in automated/CI-mode*) it should be brief.
----
## Contribute
Contributions are welcome (:
What would be helpful for now:
* Feel free to discuss the ideas and roadmap for this project with us: [GitHub discussions](https://github.com/O-X-L/firewall-testing-framework/discussions) or [contact us directly](mailto://contact@oxl.at)
* Open [issues](https://github.com/O-X-L/firewall-testing-framework/issues) if you think you have found a problem with the existing code (be aware that it might not yet be in a usable state)
* Please do not post any generic AI-slop.. thanks.
* Be friendly and respectful
----
## Roadmap
### 2025
**Core Simulator**:
* Generating Layer 3 Topology
* Generating multiple Firewalls
* Detect Firewall-chaining (one firewall routes to another one - p.e. over VPN)
* Run modes:
* Basic interactive shell
* Automated/CI mode
* Defining basic config-schema (Topology, Rulesets, Tests)
* Run multiple Test-cases from config (CLI pytest-like?)
* Option to Output results to JSON
* Security Features to protect users:
* Warn before executing non-verified (code review) plugin
**Development**:
* Create Plugin Templates
* Create Guide on how to develop Plugins
**Firewall Support**:
* Netfilter (NFTables/IPTables)
* OPNsense (Plugin that parses Config-Backup-File)
----
### What will be out-of-scope for now
Why? Because we initially need to focus on building the core simulator!
* Transparent firewalls (layer 2 interception)
* Application-Level Protocols
* Connection-Tracking helpers (rules that use these CT-states)
* Non-static routing (dynamic routing, rule-based routing via fwmark and routing-table lookup)
Raw data
{
"_id": null,
"home_page": null,
"name": "firewall-testing-framework",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "firewall, rulesets, regression-testing, cicd, iac, infrastructure-as-code, network-security",
"author": null,
"author_email": "Rath Pascal <contact@oxl.at>",
"download_url": "https://files.pythonhosted.org/packages/4e/85/f07293bdf55fda9c13081206ba59e1700752cfc0d295eef9f610d5e8b493/firewall_testing_framework-0.0.1.tar.gz",
"platform": null,
"description": "# Firewall Testing-Framework\n\nA framework for testing and troubleshooting firewall Layer 3-4 rulesets.\n\n**WARNING**: This project is still in the conception-phase.\n\n----\n\n## Goal / Why?\n\nWhen having to administer IT infrastructure and networks - we will often have multiple firewalls in place.\n\nMaintaining these might be time-consuming. You might also face some challenges:\n\n* **Troubleshooting & Analysis**:\n\n Even for senior network engineers it can be a challenge to find the source of an unexpected block/accept in large rulesets that are distributed across multiple systems and firewall vendors. \n\n Infrastructure-as-Code does help to keep the rulesets in a consistent state - but it does not solve the issue of having to manually analyze/troubleshoot existing rulesets.\n\n This project wants to provide one interface for simulating traffic over multiple firewall systems.\n\n* **Automated Regression-Tests**:\n\n Why would you want to do ruleset-regression-tests?\n * If you utilize Infrastructure-as-Code and change-reviews for updating your ruleset you might want to \n * You may want/need to periodically verify that the currently active rulesets actually allow/deny the traffic you expect\n This can be a tedious task - you might overlook some edge-case.\n * Especially when a ruleset is administered by teams of engineers over a long time period - it can be a challenge to:\n * detect configuration errors/mistakes before they can be exploited\n * make sure the design-choices for the ruleset are adhered to\n\n How do regression-tests work?\n * You define test-cases that simulate traffic over one or multiple firewalls\n * You assert that the traffic was allowed/denied/rejected\n * You might even want to assert that the traffic took a specific outbound route or was NATed to a specific IP\n\n This way you can continuously extend these test-cases and easily verify that the currently active rulesets comply with them.\n\n----\n\n## Idea\n\nTake a look at this topology:\n\n<img src=\"https://raw.githubusercontent.com/O-X-L/firewall-testing-framework/refs/heads/latest/docs/source/_static/img/topology.svg\" max-width=\"700\"></img>\n\nThe flow is planned to be:\n\n1. Either:\n * manually pull the current config from the existing firewalls\n * or utilize existing `pull-plugins` to do so (p.e. via API)\n\n2. The vendor-specific configuration gets parsed by `translation-plugins` which output a standardized firewall config-schema.\n\n3. The user provides a high-level `topology-config`\n\n4. If automated tests should be run: The user needs to provide a `test-traffic config`\n\n Else the user has the option to enter an interactive shell where traffic can be sent manually\n\n5. The `firewall/network simulator`\n\n * parses the provided config\n * generates the network-topology\n * finds where the packet originates from (or notifies the user if more information is required)\n * finds the route the packet should take\n * tests the traffic against the rulesets of firewalls that are hops of that route\n\nThanks already go to @MikPisula, the creator of the [MikPisula/packet-simulator](https://github.com/MikPisula/packet-simulator) for creating a simulator for netfilter (IPTables/NFTables) firewalls.\n\nAlso thanks to the [go-ftw (Web Application Firewall Testing Framework) project](https://github.com/coreruleset/go-ftw) that inspired us to support regression-tests.\n\n----\n\n## Principles\n\n* **Strict separation of vendor-specific plugins** from the core traffic-simulator. \n\n Plugins CAN be used to pull the current configuration (rulesets, interfaces, routes) from a firewall system, but admins should always be able to manually provide this information.\n\n Some might not want to trust some 'nice-to-have' tool with access to their firewalls.\n\n* The user should be able to choose the **output verbosity**.\n\n We want to provide full transparency (*show every rule the traffic interacts with*) but if not required (*p.e in automated/CI-mode*) it should be brief.\n\n----\n\n## Contribute\n\nContributions are welcome (:\n\nWhat would be helpful for now:\n* Feel free to discuss the ideas and roadmap for this project with us: [GitHub discussions](https://github.com/O-X-L/firewall-testing-framework/discussions) or [contact us directly](mailto://contact@oxl.at)\n* Open [issues](https://github.com/O-X-L/firewall-testing-framework/issues) if you think you have found a problem with the existing code (be aware that it might not yet be in a usable state)\n\n* Please do not post any generic AI-slop.. thanks.\n* Be friendly and respectful\n\n----\n\n## Roadmap\n\n### 2025\n\n**Core Simulator**:\n* Generating Layer 3 Topology\n* Generating multiple Firewalls\n* Detect Firewall-chaining (one firewall routes to another one - p.e. over VPN)\n* Run modes:\n * Basic interactive shell\n * Automated/CI mode\n* Defining basic config-schema (Topology, Rulesets, Tests)\n* Run multiple Test-cases from config (CLI pytest-like?)\n* Option to Output results to JSON\n* Security Features to protect users:\n * Warn before executing non-verified (code review) plugin\n\n**Development**:\n* Create Plugin Templates\n* Create Guide on how to develop Plugins\n\n**Firewall Support**:\n* Netfilter (NFTables/IPTables)\n* OPNsense (Plugin that parses Config-Backup-File)\n\n----\n\n### What will be out-of-scope for now\n\nWhy? Because we initially need to focus on building the core simulator!\n\n* Transparent firewalls (layer 2 interception)\n* Application-Level Protocols\n* Connection-Tracking helpers (rules that use these CT-states)\n* Non-static routing (dynamic routing, rule-based routing via fwmark and routing-table lookup)\n",
"bugtrack_url": null,
"license": null,
"summary": "OXL Firewall Testing Framework",
"version": "0.0.1",
"project_urls": {
"Documentation": "https://github.com/O-X-L/firewall-testing-framework",
"Homepage": "https://www.oxl.at",
"Issues": "https://github.com/O-X-L/firewall-testing-framework/issues",
"Repository": "https://github.com/O-X-L/firewall-testing-framework.git"
},
"split_keywords": [
"firewall",
" rulesets",
" regression-testing",
" cicd",
" iac",
" infrastructure-as-code",
" network-security"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "78e9767da5c6712e400a7bdbc22c5952dbc6d4f9a9cdbd90747d8b81635db692",
"md5": "0239c0b4059e86a56c3176e001520609",
"sha256": "092d7342931544aa0697a02f29585e75a229ceb3dd06b185439099ff49c3d5ac"
},
"downloads": -1,
"filename": "firewall_testing_framework-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0239c0b4059e86a56c3176e001520609",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 70470,
"upload_time": "2025-07-30T21:35:07",
"upload_time_iso_8601": "2025-07-30T21:35:07.437375Z",
"url": "https://files.pythonhosted.org/packages/78/e9/767da5c6712e400a7bdbc22c5952dbc6d4f9a9cdbd90747d8b81635db692/firewall_testing_framework-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4e85f07293bdf55fda9c13081206ba59e1700752cfc0d295eef9f610d5e8b493",
"md5": "e5c5311aa50eb51e16a31a02cb571acb",
"sha256": "7a91abe01741b81240a11e469982f68190ab543d18e76e3fe2f1d3e368060b47"
},
"downloads": -1,
"filename": "firewall_testing_framework-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "e5c5311aa50eb51e16a31a02cb571acb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 68572,
"upload_time": "2025-07-30T21:35:08",
"upload_time_iso_8601": "2025-07-30T21:35:08.892084Z",
"url": "https://files.pythonhosted.org/packages/4e/85/f07293bdf55fda9c13081206ba59e1700752cfc0d295eef9f610d5e8b493/firewall_testing_framework-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-30 21:35:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "O-X-L",
"github_project": "firewall-testing-framework",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "firewall-testing-framework"
}
JSON
