# DetectIQ
DetectIQ is an AI-powered security rule management platform that helps create, analyze, and optimize detection rules across multiple security platforms. It can be used with the provided UI, or just with Python scripts using the self contained `detectiq/core` module. See examples in the [examples](examples/) directory for more information.
[](https://www.python.org/downloads/)
[](https://www.gnu.org/licenses/lgpl-2.1)
[]()
- [Quickstart](#quickstart)
- [Current Features](#current-features)
- [Road Map](#road-map)
- [Screenshots](#screenshots)
- [Using as a Package](#using-as-a-package)
- [Environment Configuration](#environment-configuration)
- [Development](#development)
- [Contributing](#contributing)
- [License](#license)
- [Support & Community](#support--community)
- [Acknowledgments](#acknowledgments)
> ⚠️ **IMPORTANT DISCLAIMER**
>
> This project is currently a **Proof of Concept** and is under active development:
> - Features are incomplete and actively being developed
> - Bugs and breaking changes are expected
> - Project structure and APIs may change significantly
> - Documentation may be outdated or incomplete
> - Not recommended for production use at this time
> - Security features are still being implemented
>
> We welcome all feedback and contributions, but please use at your own risk!
## Quickstart
To get started, run the commands below. For more information, refer to the [docs](docs/README.md)!
**Step 1.** Clone the repository.
```bash
git clone https://github.com/AttackIQ/DetectIQ.git
```
**Step 2.** Set your environment variables (using [`.env.example`](.env.example) as a template).
```bash
cp .env.example .env
```
**Step 3.** Run the provided `start.sh` script and pass `install` as an argument.
```bash
bash start.sh install
```
**Step 4.** Run the provided `start.sh` script and pass `run` as an argument.
```bash
bash start.sh run
```
**Step 5.** Use your favorite browser to navigate to [http://localhost:3000](http://localhost:3000).
## Current Features
### AI-Powered Detection
- Create and optimize detection rules using OpenAI's LLM models
- Intelligent rule suggestions based on context and best practices
- Automated rule validation and testing
- Upload malware samples and PCAP files for static analysis, automatically adding context for YARA and Snort rule creation
- LLM Rule creation analysis and detection logic returned in the rule creation response
### Rule Repository Integration
- Enhanced by community-tested repositories:
- SigmaHQ Core Ruleset
- YARA-Forge Rules
- Snort3 Community Ruleset
- Automatically check and update repositories with rule changes
- Vectorize rules for efficient similarity comparison for more context-aware rule creation engine
### Static Analysis Integration
- Automated file analysis for YARA rules
- PCAP analysis for Snort rule creation
- Implicit log analysis for Sigma rule optimization (Explicit Analysis Coming Soon)
### Multi-Platform Integration
- Automatic Sigma rule translation to various SIEM queries using `pySigma` and `SigmAIQ` wrapper
- Seamlessly create Splunk Enterprise Security correlation rules from Sigma rules
## Road Map
- [ ] Custom/local LLM models, embeddings, and vector stores
- [ ] More integrations with SIEMs such as Elastic and Microsoft XDR
- [ ] Explicit log analysis for Sigma rule optimization
- [ ] Rule testing and validation
- [ ] Rule searching, e.g. "Do I have a rule in place that can detect this?"
- [ ] Deployment tracking and workflow automation
- [ ] Rule management UI Enhancements
- [ ] Authentication and Authorization
- [ ] Project refactoring for production readiness
- [ ] Chatbot (langchain agents) UI with memory
- [ ] Docker containerization and deployment
- [ ] Rule management without OpenAI requirements
- [ ] More non-webapp examples
## Screenshots
### Rule Dashboard with Splunk Deployment Option
<p align="center">
<em>Rule Dashboard with Splunk Deployment Option</em>
<img src="docs/images/detectiq_rules_page.png" alt="Rule Dashboard with Splunk Deployment Option"/>
</p>
### Sigma Rule Creation
<p align="center">
<em>Sigma Rule Creation from threat report snippet</em>
<img src="docs/images/detectiq_sigma_rule_creation_1.png" alt="Sigma Rule Creation"/>
<img src="docs/images/detectiq_sigma_rule_creation_2.png" alt="Sigma Rule Creation"/>
</p>
### YARA Rule Creation
<p align="center">
<em>YARA Rule Creation using file analysis from uploaded mimikatz.exe sample</em>
<img src="docs/images/detectiq_yara_rule_creation_file_1.png" alt="YARA Rule Creation"/>
<img src="docs/images/detectiq_yara_rule_creation_file_2.png" alt="YARA Rule Creation"/>
</p>
### Settings Page
<p align="center">
<em>Settings Page</em>
<img src="docs/images/detectiq_settings.png" alt="Settings Page"/>
</p>
### About Page
<p align="center">
<em>About Page</em>
<img src="docs/images/detectiq_about.png" alt="About Page"/>
</p>
## Using as a Package
DetectIQ can be installed as a Python package from PyPI:
```bash
pip install detectiq
```
This allows you to leverage DetectIQ's detection rule management capabilities in your own Python projects:
```python
import asyncio
from typing import cast
import os
# Set OpenAI API key
os.environ["OPENAI_API_KEY"] = "your-api-key"
from langchain.schema.language_model import BaseLanguageModel
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from detectiq.core.llm.yara_rules import YaraLLM
from detectiq.core.llm.toolkits.base import create_rule_agent
from detectiq.core.llm.toolkits.yara_toolkit import YaraToolkit
async def main():
# Initialize LLMs
agent_llm = cast(BaseLanguageModel, ChatOpenAI(temperature=0, model="gpt-4o"))
rule_creation_llm = cast(BaseLanguageModel, ChatOpenAI(temperature=0, model="gpt-4o"))
# Initialize YARA tools
yara_llm = YaraLLM(
embedding_model=OpenAIEmbeddings(model="text-embedding-3-small"),
agent_llm=agent_llm,
rule_creation_llm=rule_creation_llm,
rule_dir="./rules",
vector_store_dir="./vectorstore",
)
# Create agent
yara_agent = create_rule_agent(
rule_type="yara",
vectorstore=yara_llm.vectordb,
rule_creation_llm=yara_llm.rule_creation_llm,
agent_llm=yara_llm.agent_llm,
toolkit_class=YaraToolkit,
)
# Create a rule
result = await yara_agent.ainvoke({"input": "Create a YARA rule to detect ransomware"})
print(result.get("output"))
if __name__ == "__main__":
asyncio.run(main())
```
### Accessing Documentation Resources
The package includes documentation resources like screenshots and images that you can use in your own applications:
```python
from detectiq.documentation import list_images, get_image_path
# List all available images
available_images = list_images()
print(available_images.keys()) # Shows names like 'rules_page', 'yara_rule_creation_1', etc.
# Get path to a specific image
rules_page_image = get_image_path('rules_page')
print(f"Rules page image path: {rules_page_image}")
# Use the image in your application
# For example, with a GUI toolkit or in documentation
```
For more detailed examples, see the [examples](examples/) directory, particularly [examples/use_documentation.py](examples/use_documentation.py).
For instructions on publishing the package to PyPI, see [PUBLISHING.md](PUBLISHING.md).
## Environment Configuration
DetectIQ uses environment variables for configuration. A comprehensive example with documentation is provided in [.env.example](.env.example).
To configure the application:
1. Copy the example file to `.env`:
```bash
cp .env.example .env
```
2. Edit the `.env` file with your specific settings:
```bash
# Required for LLM functionality
OPENAI_API_KEY=your-api-key-here
# Optional configurations
LOG_LEVEL=INFO
DEBUG=False
```
3. The same `.env` file can be used for both the web application and the examples.
## Development
DetectIQ includes a comprehensive Makefile to assist with development, testing, and publishing tasks.
### Prerequisites
Before development, ensure you have:
1. Python 3.9+ installed
2. Poetry installed
3. Required development dependencies:
```bash
make install-dev
```
This will install all development dependencies, including:
- Testing tools (pytest)
- Code quality tools (black, ruff)
- Package building tools (build, twine)
- Keyring backends (keyrings.alt) for token management
### Makefile Commands
To view all available commands:
```bash
make help
```
#### Common Development Commands
```bash
# Installation
make install/local # Complete local installation (backend + frontend)
# Running the application
make run/local # Run both backend and frontend servers
# Code quality
make format # Format Python files using black
make ruff # Run Ruff linter
make test # Run tests with coverage
# Package management
make update # Update dependencies
make version # Display current version
make version-patch # Bump patch version (0.0.X)
make version-minor # Bump minor version (0.X.0)
make version-major # Bump major version (X.0.0)
# PyPI publishing
make pypi-build # Build package for PyPI
make pypi-check # Check package with twine
make pypi-publish # Publish to PyPI
```
For more details on publishing the package, see [PUBLISHING.md](PUBLISHING.md).
## Contributing
1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request
## License
This project uses multiple licenses:
- Core Project: LGPL v2.1
- Sigma Rules: Detection Rule License (DRL)
- YARA Rules: YARAForge License
- Snort Rules: GPL with VRT License
## Support & Community
- Join our [SigmaHQ Discord](https://discord.gg/27r98bMv6c) for discussions
- Report issues via GitHub Issues
## Acknowledgments
- SigmaHQ Community
- YARA-Forge Contributors
- Snort Community
- OpenAI for GPT-4o Integration
Raw data
{
"_id": null,
"home_page": null,
"name": "detectiq",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.14,>=3.9",
"maintainer_email": null,
"keywords": "security, detection, sigma, yara, snort, llm",
"author": "AttackIQ",
"author_email": "rajesh.sharma@attackiq.com",
"download_url": "https://files.pythonhosted.org/packages/02/62/c9917a161c333de6bd3c362adc48165b50541f8578dfa719c6f8681d8b98/detectiq-0.1.5.tar.gz",
"platform": null,
"description": "# DetectIQ\nDetectIQ is an AI-powered security rule management platform that helps create, analyze, and optimize detection rules across multiple security platforms. It can be used with the provided UI, or just with Python scripts using the self contained `detectiq/core` module. See examples in the [examples](examples/) directory for more information.\n[](https://www.python.org/downloads/)\n[](https://www.gnu.org/licenses/lgpl-2.1)\n[]()\n- [Quickstart](#quickstart)\n- [Current Features](#current-features)\n- [Road Map](#road-map)\n- [Screenshots](#screenshots)\n- [Using as a Package](#using-as-a-package)\n- [Environment Configuration](#environment-configuration)\n- [Development](#development)\n- [Contributing](#contributing)\n- [License](#license)\n- [Support & Community](#support--community)\n- [Acknowledgments](#acknowledgments)\n\n> \u26a0\ufe0f **IMPORTANT DISCLAIMER**\n> \n> This project is currently a **Proof of Concept** and is under active development:\n> - Features are incomplete and actively being developed\n> - Bugs and breaking changes are expected\n> - Project structure and APIs may change significantly\n> - Documentation may be outdated or incomplete\n> - Not recommended for production use at this time\n> - Security features are still being implemented\n> \n> We welcome all feedback and contributions, but please use at your own risk!\n\n## Quickstart\nTo get started, run the commands below. For more information, refer to the [docs](docs/README.md)!\n\n**Step 1.** Clone the repository.\n```bash\ngit clone https://github.com/AttackIQ/DetectIQ.git\n```\n\n**Step 2.** Set your environment variables (using [`.env.example`](.env.example) as a template).\n```bash\ncp .env.example .env\n```\n\n**Step 3.** Run the provided `start.sh` script and pass `install` as an argument.\n```bash\nbash start.sh install\n```\n\n**Step 4.** Run the provided `start.sh` script and pass `run` as an argument.\n```bash\nbash start.sh run\n```\n\n**Step 5.** Use your favorite browser to navigate to [http://localhost:3000](http://localhost:3000).\n\n## Current Features\n### AI-Powered Detection \n- Create and optimize detection rules using OpenAI's LLM models\n- Intelligent rule suggestions based on context and best practices\n- Automated rule validation and testing \n- Upload malware samples and PCAP files for static analysis, automatically adding context for YARA and Snort rule creation\n- LLM Rule creation analysis and detection logic returned in the rule creation response\n\n### Rule Repository Integration \n- Enhanced by community-tested repositories:\n - SigmaHQ Core Ruleset\n - YARA-Forge Rules\n - Snort3 Community Ruleset\n- Automatically check and update repositories with rule changes\n- Vectorize rules for efficient similarity comparison for more context-aware rule creation engine\n\n### Static Analysis Integration \n- Automated file analysis for YARA rules\n- PCAP analysis for Snort rule creation\n- Implicit log analysis for Sigma rule optimization (Explicit Analysis Coming Soon)\n\n### Multi-Platform Integration \n- Automatic Sigma rule translation to various SIEM queries using `pySigma` and `SigmAIQ` wrapper\n- Seamlessly create Splunk Enterprise Security correlation rules from Sigma rules\n\n## Road Map\n- [ ] Custom/local LLM models, embeddings, and vector stores\n- [ ] More integrations with SIEMs such as Elastic and Microsoft XDR\n- [ ] Explicit log analysis for Sigma rule optimization\n- [ ] Rule testing and validation\n- [ ] Rule searching, e.g. \"Do I have a rule in place that can detect this?\"\n- [ ] Deployment tracking and workflow automation\n- [ ] Rule management UI Enhancements\n- [ ] Authentication and Authorization\n- [ ] Project refactoring for production readiness\n- [ ] Chatbot (langchain agents) UI with memory\n- [ ] Docker containerization and deployment\n- [ ] Rule management without OpenAI requirements\n- [ ] More non-webapp examples\n\n## Screenshots\n### Rule Dashboard with Splunk Deployment Option\n<p align=\"center\">\n <em>Rule Dashboard with Splunk Deployment Option</em>\n <img src=\"docs/images/detectiq_rules_page.png\" alt=\"Rule Dashboard with Splunk Deployment Option\"/>\n</p>\n\n### Sigma Rule Creation\n<p align=\"center\">\n <em>Sigma Rule Creation from threat report snippet</em>\n <img src=\"docs/images/detectiq_sigma_rule_creation_1.png\" alt=\"Sigma Rule Creation\"/>\n <img src=\"docs/images/detectiq_sigma_rule_creation_2.png\" alt=\"Sigma Rule Creation\"/>\n</p>\n\n### YARA Rule Creation\n<p align=\"center\">\n <em>YARA Rule Creation using file analysis from uploaded mimikatz.exe sample</em>\n <img src=\"docs/images/detectiq_yara_rule_creation_file_1.png\" alt=\"YARA Rule Creation\"/>\n <img src=\"docs/images/detectiq_yara_rule_creation_file_2.png\" alt=\"YARA Rule Creation\"/>\n</p>\n\n### Settings Page\n<p align=\"center\">\n <em>Settings Page</em>\n <img src=\"docs/images/detectiq_settings.png\" alt=\"Settings Page\"/>\n</p>\n\n### About Page\n<p align=\"center\">\n <em>About Page</em>\n <img src=\"docs/images/detectiq_about.png\" alt=\"About Page\"/>\n</p>\n\n## Using as a Package\n\nDetectIQ can be installed as a Python package from PyPI:\n\n```bash\npip install detectiq\n```\n\nThis allows you to leverage DetectIQ's detection rule management capabilities in your own Python projects:\n\n```python\nimport asyncio\nfrom typing import cast\nimport os\n\n# Set OpenAI API key\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\n\nfrom langchain.schema.language_model import BaseLanguageModel\nfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings\nfrom detectiq.core.llm.yara_rules import YaraLLM\nfrom detectiq.core.llm.toolkits.base import create_rule_agent\nfrom detectiq.core.llm.toolkits.yara_toolkit import YaraToolkit\n\nasync def main():\n # Initialize LLMs\n agent_llm = cast(BaseLanguageModel, ChatOpenAI(temperature=0, model=\"gpt-4o\"))\n rule_creation_llm = cast(BaseLanguageModel, ChatOpenAI(temperature=0, model=\"gpt-4o\"))\n \n # Initialize YARA tools\n yara_llm = YaraLLM(\n embedding_model=OpenAIEmbeddings(model=\"text-embedding-3-small\"),\n agent_llm=agent_llm,\n rule_creation_llm=rule_creation_llm,\n rule_dir=\"./rules\",\n vector_store_dir=\"./vectorstore\",\n )\n \n # Create agent\n yara_agent = create_rule_agent(\n rule_type=\"yara\",\n vectorstore=yara_llm.vectordb,\n rule_creation_llm=yara_llm.rule_creation_llm,\n agent_llm=yara_llm.agent_llm,\n toolkit_class=YaraToolkit,\n )\n \n # Create a rule\n result = await yara_agent.ainvoke({\"input\": \"Create a YARA rule to detect ransomware\"})\n print(result.get(\"output\"))\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n### Accessing Documentation Resources\n\nThe package includes documentation resources like screenshots and images that you can use in your own applications:\n\n```python\nfrom detectiq.documentation import list_images, get_image_path\n\n# List all available images\navailable_images = list_images()\nprint(available_images.keys()) # Shows names like 'rules_page', 'yara_rule_creation_1', etc.\n\n# Get path to a specific image\nrules_page_image = get_image_path('rules_page')\nprint(f\"Rules page image path: {rules_page_image}\")\n\n# Use the image in your application\n# For example, with a GUI toolkit or in documentation\n```\n\nFor more detailed examples, see the [examples](examples/) directory, particularly [examples/use_documentation.py](examples/use_documentation.py).\n\nFor instructions on publishing the package to PyPI, see [PUBLISHING.md](PUBLISHING.md).\n\n## Environment Configuration\n\nDetectIQ uses environment variables for configuration. A comprehensive example with documentation is provided in [.env.example](.env.example).\n\nTo configure the application:\n\n1. Copy the example file to `.env`:\n ```bash\n cp .env.example .env\n ```\n\n2. Edit the `.env` file with your specific settings:\n ```bash\n # Required for LLM functionality\n OPENAI_API_KEY=your-api-key-here\n \n # Optional configurations\n LOG_LEVEL=INFO\n DEBUG=False\n ```\n\n3. The same `.env` file can be used for both the web application and the examples.\n\n## Development\n\nDetectIQ includes a comprehensive Makefile to assist with development, testing, and publishing tasks.\n\n### Prerequisites\n\nBefore development, ensure you have:\n\n1. Python 3.9+ installed\n2. Poetry installed\n3. Required development dependencies:\n ```bash\n make install-dev\n ```\n\nThis will install all development dependencies, including:\n- Testing tools (pytest)\n- Code quality tools (black, ruff)\n- Package building tools (build, twine)\n- Keyring backends (keyrings.alt) for token management\n\n### Makefile Commands\n\nTo view all available commands:\n\n```bash\nmake help\n```\n\n#### Common Development Commands\n\n```bash\n# Installation\nmake install/local # Complete local installation (backend + frontend)\n\n# Running the application\nmake run/local # Run both backend and frontend servers\n\n# Code quality\nmake format # Format Python files using black\nmake ruff # Run Ruff linter\nmake test # Run tests with coverage\n\n# Package management\nmake update # Update dependencies\nmake version # Display current version\nmake version-patch # Bump patch version (0.0.X)\nmake version-minor # Bump minor version (0.X.0)\nmake version-major # Bump major version (X.0.0)\n\n# PyPI publishing\nmake pypi-build # Build package for PyPI\nmake pypi-check # Check package with twine\nmake pypi-publish # Publish to PyPI\n```\n\nFor more details on publishing the package, see [PUBLISHING.md](PUBLISHING.md).\n\n## Contributing\n1. Fork the repository\n2. Create a feature branch\n3. Commit your changes\n4. Push to the branch\n5. Create a Pull Request\n\n## License\nThis project uses multiple licenses:\n- Core Project: LGPL v2.1\n- Sigma Rules: Detection Rule License (DRL)\n- YARA Rules: YARAForge License\n- Snort Rules: GPL with VRT License\n\n## Support & Community\n- Join our [SigmaHQ Discord](https://discord.gg/27r98bMv6c) for discussions\n- Report issues via GitHub Issues\n\n## Acknowledgments\n- SigmaHQ Community\n- YARA-Forge Contributors\n- Snort Community\n- OpenAI for GPT-4o Integration\n\n",
"bugtrack_url": null,
"license": "LGPL-2.1",
"summary": "A detection engineering workbench with LLM capabilities",
"version": "0.1.5",
"project_urls": {
"Repository": "https://github.com/AttackIQ/DetectIQ"
},
"split_keywords": [
"security",
" detection",
" sigma",
" yara",
" snort",
" llm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c8a249a96f98f73ff6186a46b686225325c2fe1889fa07f40c93492c77241a33",
"md5": "ed306ddcd0c16532a4073e5cb1bc7b60",
"sha256": "822eed3f19817167ea22d49b5331e14520365cd10ec6b00e5182e9912daa398b"
},
"downloads": -1,
"filename": "detectiq-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ed306ddcd0c16532a4073e5cb1bc7b60",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.14,>=3.9",
"size": 100966,
"upload_time": "2025-03-19T05:21:34",
"upload_time_iso_8601": "2025-03-19T05:21:34.424882Z",
"url": "https://files.pythonhosted.org/packages/c8/a2/49a96f98f73ff6186a46b686225325c2fe1889fa07f40c93492c77241a33/detectiq-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0262c9917a161c333de6bd3c362adc48165b50541f8578dfa719c6f8681d8b98",
"md5": "903a96af59d7c66d29731b8b419e10b0",
"sha256": "6bccc314d3fb8cfcfac0c6e7cb40160b0c53a924354dd9b046bfe4b62171dbb0"
},
"downloads": -1,
"filename": "detectiq-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "903a96af59d7c66d29731b8b419e10b0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.14,>=3.9",
"size": 1479805,
"upload_time": "2025-03-19T05:21:37",
"upload_time_iso_8601": "2025-03-19T05:21:37.874012Z",
"url": "https://files.pythonhosted.org/packages/02/62/c9917a161c333de6bd3c362adc48165b50541f8578dfa719c6f8681d8b98/detectiq-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-19 05:21:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AttackIQ",
"github_project": "DetectIQ",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "detectiq"
}
JSON
