# pySigma InsightIDR Backend
## Overview
This is the Rapid7 [InsightIDR](https://www.rapid7.com/products/insightidr/) backend for [pySigma](https://github.com/SigmaHQ/pySigma), capable of converting Sigma rules into [Log Entry Query Language (LEQL)](https://docs.rapid7.com/insightidr/use-a-search-language) queries compatible with the InsightIDR SIEM. It provides the package `sigma.backends.insight_idr` with the `InsightIDRBackend` class.
Further, it contains the processing pipeline `sigma.pipelines.insight_idr`, which performs field mapping and error handling. The InsightIDR pipeline is the automatic/default processing pipeline for the InsightIDR backend.
## Installation
The pySigma InsightIDR Backend is available on Python Package Index (PyPI) here: https://pypi.org/project/pysigma-backend-insightidr/. It can be installed using pip with the command: `pip install pysigma-backend-insightidr`.
## Rule Support
The InsightIDR backend supports the following log entry/rule types:
* Process start events
* DNS query events
* Web proxy events
* Firewall events
* Ingress authentication events
## Output Format Support
It supports the following output formats which can be used for log search, custom alerts, dashboards, and reporting:
* **default**: queries output in the InsightIDR "Simple" format*
* **leql_advanced_search**: queries in the "Advanced" format**
* **leql_detection_definition**: queries matching the LEQL detection rule logic format roughly matching what is shown in the InsightIDR Detection Rules -> Detection Rule -> Rule Logic screen***
\*Ideal for use in custom alerts.
\*\*Ideal for use with [InsightIDR4Py](https://github.com/mbabinski/InsightIDR4Py), a module offering streamlined access to the Rapid7 LogSearch API.
\*\*\*Conceptual only - these queries are not usable within the InsightIDR interfaces mentioned above.
Sigma rules using the Sigma endswith modifier uses a regular expression for pattern matching, as LEQL contains no IENDS-WITH or IENDS-WITH-ANY modifier.
## Usage example
### Sigma CLI
You can quickly convert a single rule or rules in a directory structure using Sigma CLI. You can use:
`sigma convert -t insightidr -f leql_advanced_search -s ~/sigma/rules` where -t is the target query language, -f is the desired output format, and -s is the Sigma rule or rules directory you wish to convert.
### Stand-alone Script
The following example script demonstrates how you can use the InsightIDR backend to generate advanced LEQL queries for the following Sigma rules:
* [Webshell Detection With Command Line Keywords](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_webshell_detection.yml)
* [Windows Cmd Delete File](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_cmd_delete.yml)
* [Suspicious Rundll32 Activity](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_susp_rundll32_activity.yml)
```python
# demonstrates basic usage of InsightIDR backend
from sigma.collection import SigmaCollection
from sigma.backends.insight_idr import insight_idr
# create pipeline and backend
insight_idr_backend = insight_idr.InsightIDRBackend()
# load a ruleset
process_start_rules = [r"C:\SigmaRules\rules\windows\process_creation\proc_creation_win_webshell_detection.yml",
r"C:\SigmaRules\rules\windows\process_creation\proc_creation_win_cmd_delete.yml",
r"C:\SigmaRules\rules\windows\process_creation\proc_creation_win_susp_rundll32_activity.yml"]
process_start_rule_collection = SigmaCollection.load_ruleset(process_start_rules)
# convert the rules
for rule in process_start_rule_collection.rules:
print(rule.title + " conversion:")
print(insight_idr_backend.convert_rule(rule, "leql_advanced_search")[0])
print("\n")
```
with resulting output:
```
Webshell Detection With Command Line Keywords conversion:
where((parent_process.exe_path=/(.*\\w3wp\.exe$|.*\\php\-cgi\.exe$|.*\\nginx\.exe$|.*\\httpd\.exe$)/i OR parent_process.exe_path ICONTAINS-ANY ["\apache", "\tomcat"]) AND ((process.exe_path=/(.*\\net\.exe$|.*\\net1\.exe$)/i) AND (process.cmd_line ICONTAINS-ANY [" user ", " use ", " group "]) OR process.exe_path=/.*\\ping\.exe$/i AND process.cmd_line ICONTAINS " -n " OR process.cmd_line ICONTAINS-ANY ["&cd&echo", "cd /d "] OR process.exe_path=/.*\\wmic\.exe$/i AND process.cmd_line ICONTAINS " /node:" OR process.exe_path=/(.*\\whoami\.exe$|.*\\systeminfo\.exe$|.*\\quser\.exe$|.*\\ipconfig\.exe$|.*\\pathping\.exe$|.*\\tracert\.exe$|.*\\netstat\.exe$|.*\\schtasks\.exe$|.*\\vssadmin\.exe$|.*\\wevtutil\.exe$|.*\\tasklist\.exe$)/i OR process.cmd_line ICONTAINS-ANY [" Test-NetConnection ", "dir \"]))
Windows Cmd Delete File conversion:
where(process.cmd_line ICONTAINS-ALL ["del ", "/f"] OR process.cmd_line ICONTAINS-ALL ["rmdir", "/s", "/q"])
Suspicious Rundll32 Activity conversion:
where(process.cmd_line ICONTAINS-ALL ["javascript:", ".RegisterXLL"] OR process.cmd_line ICONTAINS-ALL ["url.dll", "OpenURL"] OR process.cmd_line ICONTAINS-ALL ["url.dll", "OpenURLA"] OR process.cmd_line ICONTAINS-ALL ["url.dll", "FileProtocolHandler"] OR process.cmd_line ICONTAINS-ALL ["zipfldr.dll", "RouteTheCall"] OR process.cmd_line ICONTAINS-ALL ["shell32.dll", "Control_RunDLL"] OR process.cmd_line ICONTAINS-ALL ["shell32.dll", "ShellExec_RunDLL"] OR process.cmd_line ICONTAINS-ALL ["mshtml.dll", "PrintHTML"] OR process.cmd_line ICONTAINS-ALL ["advpack.dll", "LaunchINFSection"] OR process.cmd_line ICONTAINS-ALL ["advpack.dll", "RegisterOCX"] OR process.cmd_line ICONTAINS-ALL ["ieadvpack.dll", "LaunchINFSection"] OR process.cmd_line ICONTAINS-ALL ["ieadvpack.dll", "RegisterOCX"] OR process.cmd_line ICONTAINS-ALL ["ieframe.dll", "OpenURL"] OR process.cmd_line ICONTAINS-ALL ["shdocvw.dll", "OpenURL"] OR process.cmd_line ICONTAINS-ALL ["syssetup.dll", "SetupInfObjectInstallAction"] OR process.cmd_line ICONTAINS-ALL ["setupapi.dll", "InstallHinfSection"] OR process.cmd_line ICONTAINS-ALL ["pcwutl.dll", "LaunchApplication"] OR process.cmd_line ICONTAINS-ALL ["dfshim.dll", "ShOpenVerbApplication"])
```
## Limitations and Constraints
This backend is in a preliminary stage, and does not support all Sigma rule types or InsightIDR event sources/logset types. Attempting to convert rule types other than the types listed above will result in an error.
Additionally, certain selection fields listed below are not supported within the following Sigma rule types:
Process start events
* CurrentDirectory
* IntegrityLevel
* imphash
* LogonId
DNS query events
* ProcessId
* QueryStatus
* QueryResults
Web proxy events
* c-uri-extension
* c-uri-stem
* c-useragent
* cs-referrer
* cs-version
* sc-status
Finally, Sigma rules using selection conditions based on aggregate functions like count() are deprecated within pySigma and are not supported.
Note that [sigma-cli](https://github.com/SigmaHQ/sigma-cli) contains swithces, ```--skip-unsupported``` and ```--fail-unsupported``` that allow the user to skip rules that cannot be supported by the backend.
## Authorship and Maintenance
This backend was authored and is currently maintained by [Micah Babinski](https://github.com/mbabinski/) with generous assistance from [Thomas Patzke](https://github.com/thomaspatzke). Suggestions and collaboration are welcomed in any form.
Raw data
{
"_id": null,
"home_page": "https://github.com/SigmaHQ/pySigma-backend-insightidr",
"name": "pysigma-backend-insightidr",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Micah Babinski",
"author_email": "m.babinski.88@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/12/f7/382c1dc2f3d6b04cd0cf68919b24b45f62226a37151ac3ec4d572b0cceee/pysigma_backend_insightidr-0.2.4.tar.gz",
"platform": null,
"description": "# pySigma InsightIDR Backend\n\n\n\n\n\n## Overview\nThis is the Rapid7 [InsightIDR](https://www.rapid7.com/products/insightidr/) backend for [pySigma](https://github.com/SigmaHQ/pySigma), capable of converting Sigma rules into [Log Entry Query Language (LEQL)](https://docs.rapid7.com/insightidr/use-a-search-language) queries compatible with the InsightIDR SIEM. It provides the package `sigma.backends.insight_idr` with the `InsightIDRBackend` class.\nFurther, it contains the processing pipeline `sigma.pipelines.insight_idr`, which performs field mapping and error handling. The InsightIDR pipeline is the automatic/default processing pipeline for the InsightIDR backend.\n\n## Installation\nThe pySigma InsightIDR Backend is available on Python Package Index (PyPI) here: https://pypi.org/project/pysigma-backend-insightidr/. It can be installed using pip with the command: `pip install pysigma-backend-insightidr`.\n\n## Rule Support\nThe InsightIDR backend supports the following log entry/rule types:\n\n* Process start events\n* DNS query events\n* Web proxy events\n* Firewall events\n* Ingress authentication events\n\n## Output Format Support\nIt supports the following output formats which can be used for log search, custom alerts, dashboards, and reporting:\n\n* **default**: queries output in the InsightIDR \"Simple\" format*\n* **leql_advanced_search**: queries in the \"Advanced\" format**\n* **leql_detection_definition**: queries matching the LEQL detection rule logic format roughly matching what is shown in the InsightIDR Detection Rules -> Detection Rule -> Rule Logic screen***\n\n\\*Ideal for use in custom alerts. \n\\*\\*Ideal for use with [InsightIDR4Py](https://github.com/mbabinski/InsightIDR4Py), a module offering streamlined access to the Rapid7 LogSearch API. \n\\*\\*\\*Conceptual only - these queries are not usable within the InsightIDR interfaces mentioned above. \n\nSigma rules using the Sigma endswith modifier uses a regular expression for pattern matching, as LEQL contains no IENDS-WITH or IENDS-WITH-ANY modifier.\n\n## Usage example\n\n### Sigma CLI\nYou can quickly convert a single rule or rules in a directory structure using Sigma CLI. You can use:\n`sigma convert -t insightidr -f leql_advanced_search -s ~/sigma/rules` where -t is the target query language, -f is the desired output format, and -s is the Sigma rule or rules directory you wish to convert.\n\n### Stand-alone Script\nThe following example script demonstrates how you can use the InsightIDR backend to generate advanced LEQL queries for the following Sigma rules:\n* [Webshell Detection With Command Line Keywords](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_webshell_detection.yml)\n* [Windows Cmd Delete File](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_cmd_delete.yml)\n* [Suspicious Rundll32 Activity](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_susp_rundll32_activity.yml)\n```python\n# demonstrates basic usage of InsightIDR backend\nfrom sigma.collection import SigmaCollection\nfrom sigma.backends.insight_idr import insight_idr\n\n# create pipeline and backend\ninsight_idr_backend = insight_idr.InsightIDRBackend()\n\n# load a ruleset\nprocess_start_rules = [r\"C:\\SigmaRules\\rules\\windows\\process_creation\\proc_creation_win_webshell_detection.yml\",\n r\"C:\\SigmaRules\\rules\\windows\\process_creation\\proc_creation_win_cmd_delete.yml\",\n r\"C:\\SigmaRules\\rules\\windows\\process_creation\\proc_creation_win_susp_rundll32_activity.yml\"]\n\nprocess_start_rule_collection = SigmaCollection.load_ruleset(process_start_rules)\n\n# convert the rules\nfor rule in process_start_rule_collection.rules:\n print(rule.title + \" conversion:\")\n print(insight_idr_backend.convert_rule(rule, \"leql_advanced_search\")[0])\n print(\"\\n\")\n```\n\nwith resulting output:\n```\nWebshell Detection With Command Line Keywords conversion:\nwhere((parent_process.exe_path=/(.*\\\\w3wp\\.exe$|.*\\\\php\\-cgi\\.exe$|.*\\\\nginx\\.exe$|.*\\\\httpd\\.exe$)/i OR parent_process.exe_path ICONTAINS-ANY [\"\\apache\", \"\\tomcat\"]) AND ((process.exe_path=/(.*\\\\net\\.exe$|.*\\\\net1\\.exe$)/i) AND (process.cmd_line ICONTAINS-ANY [\" user \", \" use \", \" group \"]) OR process.exe_path=/.*\\\\ping\\.exe$/i AND process.cmd_line ICONTAINS \" -n \" OR process.cmd_line ICONTAINS-ANY [\"&cd&echo\", \"cd /d \"] OR process.exe_path=/.*\\\\wmic\\.exe$/i AND process.cmd_line ICONTAINS \" /node:\" OR process.exe_path=/(.*\\\\whoami\\.exe$|.*\\\\systeminfo\\.exe$|.*\\\\quser\\.exe$|.*\\\\ipconfig\\.exe$|.*\\\\pathping\\.exe$|.*\\\\tracert\\.exe$|.*\\\\netstat\\.exe$|.*\\\\schtasks\\.exe$|.*\\\\vssadmin\\.exe$|.*\\\\wevtutil\\.exe$|.*\\\\tasklist\\.exe$)/i OR process.cmd_line ICONTAINS-ANY [\" Test-NetConnection \", \"dir \\\"]))\n\nWindows Cmd Delete File conversion:\nwhere(process.cmd_line ICONTAINS-ALL [\"del \", \"/f\"] OR process.cmd_line ICONTAINS-ALL [\"rmdir\", \"/s\", \"/q\"])\n\nSuspicious Rundll32 Activity conversion:\nwhere(process.cmd_line ICONTAINS-ALL [\"javascript:\", \".RegisterXLL\"] OR process.cmd_line ICONTAINS-ALL [\"url.dll\", \"OpenURL\"] OR process.cmd_line ICONTAINS-ALL [\"url.dll\", \"OpenURLA\"] OR process.cmd_line ICONTAINS-ALL [\"url.dll\", \"FileProtocolHandler\"] OR process.cmd_line ICONTAINS-ALL [\"zipfldr.dll\", \"RouteTheCall\"] OR process.cmd_line ICONTAINS-ALL [\"shell32.dll\", \"Control_RunDLL\"] OR process.cmd_line ICONTAINS-ALL [\"shell32.dll\", \"ShellExec_RunDLL\"] OR process.cmd_line ICONTAINS-ALL [\"mshtml.dll\", \"PrintHTML\"] OR process.cmd_line ICONTAINS-ALL [\"advpack.dll\", \"LaunchINFSection\"] OR process.cmd_line ICONTAINS-ALL [\"advpack.dll\", \"RegisterOCX\"] OR process.cmd_line ICONTAINS-ALL [\"ieadvpack.dll\", \"LaunchINFSection\"] OR process.cmd_line ICONTAINS-ALL [\"ieadvpack.dll\", \"RegisterOCX\"] OR process.cmd_line ICONTAINS-ALL [\"ieframe.dll\", \"OpenURL\"] OR process.cmd_line ICONTAINS-ALL [\"shdocvw.dll\", \"OpenURL\"] OR process.cmd_line ICONTAINS-ALL [\"syssetup.dll\", \"SetupInfObjectInstallAction\"] OR process.cmd_line ICONTAINS-ALL [\"setupapi.dll\", \"InstallHinfSection\"] OR process.cmd_line ICONTAINS-ALL [\"pcwutl.dll\", \"LaunchApplication\"] OR process.cmd_line ICONTAINS-ALL [\"dfshim.dll\", \"ShOpenVerbApplication\"])\n```\n\n## Limitations and Constraints\nThis backend is in a preliminary stage, and does not support all Sigma rule types or InsightIDR event sources/logset types. Attempting to convert rule types other than the types listed above will result in an error.\n\nAdditionally, certain selection fields listed below are not supported within the following Sigma rule types:\n\nProcess start events\n* CurrentDirectory\n* IntegrityLevel\n* imphash\n* LogonId\n\nDNS query events\n* ProcessId\n* QueryStatus\n* QueryResults\n\nWeb proxy events\n* c-uri-extension\n* c-uri-stem\n* c-useragent\n* cs-referrer\n* cs-version\n* sc-status\n\nFinally, Sigma rules using selection conditions based on aggregate functions like count() are deprecated within pySigma and are not supported.\n\nNote that [sigma-cli](https://github.com/SigmaHQ/sigma-cli) contains swithces, ```--skip-unsupported``` and ```--fail-unsupported``` that allow the user to skip rules that cannot be supported by the backend.\n\n## Authorship and Maintenance\nThis backend was authored and is currently maintained by [Micah Babinski](https://github.com/mbabinski/) with generous assistance from [Thomas Patzke](https://github.com/thomaspatzke). Suggestions and collaboration are welcomed in any form.\n",
"bugtrack_url": null,
"license": "LGPL-2.1-only",
"summary": "pySigma Rapid7 InsightIDR backend",
"version": "0.2.4",
"project_urls": {
"Homepage": "https://github.com/SigmaHQ/pySigma-backend-insightidr",
"Repository": "https://github.com/SigmaHQ/pySigma-backend-insightidr"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3b2864efcf4b1df7e572cb38194895557d28b342bc882376e43d21f328850606",
"md5": "aec0eb2947c093583d8ef6d9faa72905",
"sha256": "2341165e08373437f02e37521b5ce2a4cd1a07747d6cb27f759c6493ff56c8b4"
},
"downloads": -1,
"filename": "pysigma_backend_insightidr-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "aec0eb2947c093583d8ef6d9faa72905",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 19276,
"upload_time": "2024-09-30T21:44:36",
"upload_time_iso_8601": "2024-09-30T21:44:36.624403Z",
"url": "https://files.pythonhosted.org/packages/3b/28/64efcf4b1df7e572cb38194895557d28b342bc882376e43d21f328850606/pysigma_backend_insightidr-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "12f7382c1dc2f3d6b04cd0cf68919b24b45f62226a37151ac3ec4d572b0cceee",
"md5": "b17d791759a65c107d251e1a537a4394",
"sha256": "7ac24cbff4e9ffbdb94ac5d94ddc69fee8d2e3eb8e0d5db7b6509f16f346750a"
},
"downloads": -1,
"filename": "pysigma_backend_insightidr-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "b17d791759a65c107d251e1a537a4394",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 19986,
"upload_time": "2024-09-30T21:44:37",
"upload_time_iso_8601": "2024-09-30T21:44:37.856724Z",
"url": "https://files.pythonhosted.org/packages/12/f7/382c1dc2f3d6b04cd0cf68919b24b45f62226a37151ac3ec4d572b0cceee/pysigma_backend_insightidr-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-30 21:44:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SigmaHQ",
"github_project": "pySigma-backend-insightidr",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pysigma-backend-insightidr"
}
JSON
