# DNSMule
[](https://codecov.io/gh/joniumGit/dnsmule)
Package for rule based dependency scanning and service fingerprinting via DNS.
This package provides utilities for writing and evaluating verbose and easy to read rule definitions in _YAML_-format.
There are two builtin rule formats with more available as plugins.
## Installation
```shell
pip install dnsmule[full] dnsmule_plugins[full]
```
This will install everything available for DNSMule. You can also choose to install components as necessary.
For installing from the repo you can use:
```shell
pip install -e .
```
## Overview
The DNSMule tool is composed in the following way:
- DNSMule
- Backend
- Rules
- Rule
- Plugins
## Examples
Check out the examples in the [examples](examples) folder. They should get you up and running quickly.
## YAML Configuration
### Summary
The tool configuration is done through one or multiple rule configuration files. The file structure is defined in the
[schema file](rules/rules-schema.yml). In addition to some builtin rule types, it is possible to create new types by
registering handlers or rules programmatically.
Rules support registration per DNS record type and priority for controlling invocation order.
```yaml
version: '0.0.1'
rules:
- name: o365
priority: 10
type: dns.regex
record: txt
config:
pattern: '^MS=ms'
identification: MICROSOFT::O365
- name: ses
type: dns.regex
record: txt
config:
pattern: '^amazonses:'
identification: AMAZON::SES
```
#### Example
```python
from dnsmule.definitions import Record, Result
from dnsmule.rules import Rules, Rule
rules: Rules
...
@rules.add.A[10]
def my_scan(record: Record) -> Result:
from dnsmule.logger import get_logger
get_logger().info('Address %s', record)
return record.tag('MY::SCAN')
@rules.register('my.rule')
def create_my_rule(**arguments) -> Rule:
...
```
Here the `add` is used to directly register a new rule into the ruleset with a given priority. The `register` call
creates a new handler for rules of type `my.rule`. Any future `my.rule` creations from YAML or code would be routed to
this factory function.
See the examples folder for more examples and how to use rules in code.
### Plugins and Backends
#### Plugins
It is possible to register plugins using the YAML file:
```yaml
plugins:
- name: dnsmule_plugins.CertCheckPlugin
config:
callback: false
```
These are required to extend the `dnsmule.plugins.Plugin` class.
Plugins are evaluated and initialized before rules.
Any rules requiring a plugin should list their plugin in this block.
Plugins are only initialized once and if a plugin already exists in the receiving DNSMule instance
it will be ignored.
#### Backends
It is possible to define a single backend in a YAML file:
```yaml
backend:
name: 'dnspython.DNSPythonBackend'
config:
timeout: 5.5
resolver: 8.8.8.8
```
The backend must extend the `dnsmule.backends.Backend` class.
This declaration is ignored if this is not used in `DNSMule.load` or `DNSMule(file=file)`.
## Editor Support
#### Type Hints and JSON Schema (IntelliJ IDEA, PyCharm, etc.)
It is possible to register the schema file as a custom JSON schema in IntelliJ editors.
This will give access to typehints and schema validation inside rule files and is especially nice for dynamic rule
definitions as you get full editor features for python inside the snippets.
- settings...
- Languages & Frameworks > Schemas and DTDs > JSON Schema Mappings
- Add a new mapping with the schema file and specified file or pattern
This is configured in the schema using the custom intellij language injection tag:
```yml
x-intellij-language-injection:
language: Python
prefix: |+0
code hints go here
check the schema for more info
suffix: ''
```
Currently, this supports `dns.regex` pattern regex language injection and `dns.dynamic` rule code language injection.
Type hints and quick documentation are available.
## Builtin Rules
#### Regex rules
Regex rules can be defined with either one `pattern` or multiple `patterns`.
An example is in the following snippet:
```yml
rules:
- name: test
type: dns.regex
record: txt
config:
pattern: '^.*\.hello_world\.'
identification: HELLO::WORLD
flags:
- UNICODE
- DOTALL
- name: generic_verification
type: dns.regex
record: txt
priority: 10
description: Generic Site Regex Collection
config:
patterns:
- '^(.+)(?:-(?:site|domain))?-verification='
- '^(.+)(?:site|domain)verification'
- '^(.+)_verify_'
- '^(\w+)-code:'
group: 1
```
The full definition and additional info is available from the schema file, examples, and code.
#### Dynamic Rules
Dynamic rules are defined as code snippets with one or two methods
An init method that is invoked once after creation
```python
def init() -> None:
add_rule(...)
```
A process function that is invoked once for each record
```python
def process(record: Record) -> Result:
add_rule(...)
return record.result()
```
Both of these functions have access to the following rule creation method:
```python
def add_rule(
record_type: Union[str, int, RRType],
rule_type: str,
name: str,
*,
priority: int = 0,
**options,
) -> None:
"""
:param record_type: Valid DNS record type as text, int, or type
:param rule_type: Valid rule type factory e.g. dns.regex
:param name: Name of the created rule
:param priority: Priority for the created rule, default 0
:param options: Any additional options for the rule factory
"""
```
The only globals passed to these methods are:
- \_\_builtins\_\_
- RRType, Record, Result, Domain, Tag, Config
- The Config contains the `config` property passed to the rule from YAML
- add_rule
- Any additional globals created by the code itself
When the code is exec'd the result is inspected for:
- init function without parameters
- process function with a single parameter
Some notes:
- The init function is invoked exactly once.
- The process function is invoked exactly once for every single Record.
- Any rules created from the init method will be invoked for every suitable record.
- Any rules created from the process method will be invoked for suitable records found after creation.
- Creating DynamicRules from init or process is considered undefined behaviour and care should be taken
- The user should call init manually and include fail-safes for only calling it once
- The add_rule callback might not be available so you need to pass it manually to the rule
## Other
### Example server
The repo has a `Dockerfile` for easily running the tool using an example server in Docker:
```shell
$ ./build-image
$ ./run-server
```
### Notice
This package is under active development.
### Additional
- RnD Scripts under [scripts](scripts)
- Example server under [server](server)
- Examples for coding under [examples](examples)
Raw data
{
"_id": null,
"home_page": "https://github.com/joniumGit/dnsmule",
"name": "dnsmule",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "DNS,Dependency,DNSMule",
"author": "joniumGit",
"author_email": "52005121+joniumGit@users.noreply.github.com",
"download_url": "https://files.pythonhosted.org/packages/50/5d/79f6cc520157be3e8e6bcb9014213fdad266632e354119f19445e79d1e84/dnsmule-0.5.0.tar.gz",
"platform": null,
"description": "# DNSMule\n\n[](https://codecov.io/gh/joniumGit/dnsmule)\n\nPackage for rule based dependency scanning and service fingerprinting via DNS.\n\nThis package provides utilities for writing and evaluating verbose and easy to read rule definitions in _YAML_-format.\nThere are two builtin rule formats with more available as plugins.\n\n## Installation\n\n```shell\npip install dnsmule[full] dnsmule_plugins[full]\n```\n\nThis will install everything available for DNSMule. You can also choose to install components as necessary.\n\nFor installing from the repo you can use:\n\n```shell\npip install -e .\n```\n\n## Overview\n\nThe DNSMule tool is composed in the following way:\n\n- DNSMule\n - Backend\n - Rules\n - Rule\n - Plugins\n\n## Examples\n\nCheck out the examples in the [examples](examples) folder. They should get you up and running quickly.\n\n## YAML Configuration\n\n### Summary\n\nThe tool configuration is done through one or multiple rule configuration files. The file structure is defined in the\n[schema file](rules/rules-schema.yml). In addition to some builtin rule types, it is possible to create new types by\nregistering handlers or rules programmatically.\n\nRules support registration per DNS record type and priority for controlling invocation order.\n\n```yaml\nversion: '0.0.1'\nrules:\n - name: o365\n priority: 10\n type: dns.regex\n record: txt\n config:\n pattern: '^MS=ms'\n identification: MICROSOFT::O365\n - name: ses\n type: dns.regex\n record: txt\n config:\n pattern: '^amazonses:'\n identification: AMAZON::SES\n```\n\n#### Example\n\n```python\nfrom dnsmule.definitions import Record, Result\nfrom dnsmule.rules import Rules, Rule\n\nrules: Rules\n\n...\n\n\n@rules.add.A[10]\ndef my_scan(record: Record) -> Result:\n from dnsmule.logger import get_logger\n get_logger().info('Address %s', record)\n return record.tag('MY::SCAN')\n\n\n@rules.register('my.rule')\ndef create_my_rule(**arguments) -> Rule:\n ...\n```\n\nHere the `add` is used to directly register a new rule into the ruleset with a given priority. The `register` call\ncreates a new handler for rules of type `my.rule`. Any future `my.rule` creations from YAML or code would be routed to\nthis factory function.\n\nSee the examples folder for more examples and how to use rules in code.\n\n### Plugins and Backends\n\n#### Plugins\n\nIt is possible to register plugins using the YAML file:\n\n```yaml\nplugins:\n - name: dnsmule_plugins.CertCheckPlugin\n config:\n callback: false\n```\n\nThese are required to extend the `dnsmule.plugins.Plugin` class.\nPlugins are evaluated and initialized before rules.\nAny rules requiring a plugin should list their plugin in this block.\nPlugins are only initialized once and if a plugin already exists in the receiving DNSMule instance\nit will be ignored.\n\n#### Backends\n\nIt is possible to define a single backend in a YAML file:\n\n```yaml\nbackend:\n name: 'dnspython.DNSPythonBackend'\n config:\n timeout: 5.5\n resolver: 8.8.8.8\n```\n\nThe backend must extend the `dnsmule.backends.Backend` class.\nThis declaration is ignored if this is not used in `DNSMule.load` or `DNSMule(file=file)`.\n\n## Editor Support\n\n#### Type Hints and JSON Schema (IntelliJ IDEA, PyCharm, etc.)\n\nIt is possible to register the schema file as a custom JSON schema in IntelliJ editors.\nThis will give access to typehints and schema validation inside rule files and is especially nice for dynamic rule\ndefinitions as you get full editor features for python inside the snippets.\n\n- settings...\n- Languages & Frameworks > Schemas and DTDs > JSON Schema Mappings\n- Add a new mapping with the schema file and specified file or pattern\n\nThis is configured in the schema using the custom intellij language injection tag:\n\n```yml\nx-intellij-language-injection:\n language: Python\n prefix: |+0\n code hints go here\n check the schema for more info\n suffix: ''\n```\n\nCurrently, this supports `dns.regex` pattern regex language injection and `dns.dynamic` rule code language injection.\nType hints and quick documentation are available.\n\n## Builtin Rules\n\n#### Regex rules\n\nRegex rules can be defined with either one `pattern` or multiple `patterns`.\nAn example is in the following snippet:\n\n```yml\nrules:\n - name: test\n type: dns.regex\n record: txt\n config:\n pattern: '^.*\\.hello_world\\.'\n identification: HELLO::WORLD\n flags:\n - UNICODE\n - DOTALL\n - name: generic_verification\n type: dns.regex\n record: txt\n priority: 10\n description: Generic Site Regex Collection\n config:\n patterns:\n - '^(.+)(?:-(?:site|domain))?-verification='\n - '^(.+)(?:site|domain)verification'\n - '^(.+)_verify_'\n - '^(\\w+)-code:'\n group: 1\n```\n\nThe full definition and additional info is available from the schema file, examples, and code.\n\n#### Dynamic Rules\n\nDynamic rules are defined as code snippets with one or two methods\n\nAn init method that is invoked once after creation\n\n```python\ndef init() -> None:\n add_rule(...)\n```\n\nA process function that is invoked once for each record\n\n```python\ndef process(record: Record) -> Result:\n add_rule(...)\n return record.result()\n```\n\nBoth of these functions have access to the following rule creation method:\n\n```python\ndef add_rule(\n record_type: Union[str, int, RRType],\n rule_type: str,\n name: str,\n *,\n priority: int = 0,\n **options,\n) -> None:\n \"\"\"\n :param record_type: Valid DNS record type as text, int, or type\n :param rule_type: Valid rule type factory e.g. dns.regex\n :param name: Name of the created rule\n :param priority: Priority for the created rule, default 0\n :param options: Any additional options for the rule factory\n \"\"\"\n```\n\nThe only globals passed to these methods are:\n\n- \\_\\_builtins\\_\\_\n- RRType, Record, Result, Domain, Tag, Config\n - The Config contains the `config` property passed to the rule from YAML\n- add_rule\n- Any additional globals created by the code itself\n\nWhen the code is exec'd the result is inspected for:\n\n- init function without parameters\n- process function with a single parameter\n\nSome notes:\n\n- The init function is invoked exactly once.\n- The process function is invoked exactly once for every single Record.\n- Any rules created from the init method will be invoked for every suitable record.\n- Any rules created from the process method will be invoked for suitable records found after creation.\n- Creating DynamicRules from init or process is considered undefined behaviour and care should be taken\n - The user should call init manually and include fail-safes for only calling it once\n - The add_rule callback might not be available so you need to pass it manually to the rule\n\n## Other\n\n### Example server\n\nThe repo has a `Dockerfile` for easily running the tool using an example server in Docker:\n\n```shell\n$ ./build-image\n$ ./run-server\n```\n\n### Notice\n\nThis package is under active development.\n\n### Additional\n\n- RnD Scripts under [scripts](scripts)\n- Example server under [server](server)\n- Examples for coding under [examples](examples)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Rule based dependency scanning and service fingerprinting via DNS",
"version": "0.5.0",
"project_urls": {
"Bug Reports": "https://github.com/joniumGit/dnsmule/issues",
"Homepage": "https://github.com/joniumGit/dnsmule",
"Source": "https://github.com/joniumGit/dnsmule"
},
"split_keywords": [
"dns",
"dependency",
"dnsmule"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "11b2f4b464ca611da63ef7a84bacefbaead514b76247ce20ccb616527e6677dd",
"md5": "097b16ce28c70fdb013ed8df8a81c4d0",
"sha256": "399b5e8c9a1c365c21876708e9692ad373b41ee1125eeb8ee7780b817f833010"
},
"downloads": -1,
"filename": "dnsmule-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "097b16ce28c70fdb013ed8df8a81c4d0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 29120,
"upload_time": "2023-05-13T17:40:52",
"upload_time_iso_8601": "2023-05-13T17:40:52.361547Z",
"url": "https://files.pythonhosted.org/packages/11/b2/f4b464ca611da63ef7a84bacefbaead514b76247ce20ccb616527e6677dd/dnsmule-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "505d79f6cc520157be3e8e6bcb9014213fdad266632e354119f19445e79d1e84",
"md5": "51ba747b835bacd0755fa128aa525652",
"sha256": "e4962aabd48eda9cf8c623d4364774a76835b77bbfef9f8daa25c1b0d68fba83"
},
"downloads": -1,
"filename": "dnsmule-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "51ba747b835bacd0755fa128aa525652",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 23628,
"upload_time": "2023-05-13T17:40:54",
"upload_time_iso_8601": "2023-05-13T17:40:54.201675Z",
"url": "https://files.pythonhosted.org/packages/50/5d/79f6cc520157be3e8e6bcb9014213fdad266632e354119f19445e79d1e84/dnsmule-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-13 17:40:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "joniumGit",
"github_project": "dnsmule",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dnsmule"
}
JSON
