# GLIMPS Audit Client Library and CLI
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/astral-sh/ruff)
[](https://badge.fury.io/py/gaudit)
A comprehensive Python client library, command-line interface, and IDA Pro plugin for interacting with the GLIMPS Audit API v2.0.4. This tool enables seamless integration with GLIMPS's binary analysis platform for software composition analysis and vulnerability detection.
## Table of Contents
- [Features](#features)
- [Requirements](#requirements)
- [Installation](#installation)
- [IDA Pro Plugin](#ida-pro-plugin)
- [Quick Start](#quick-start)
- [Configuration](#configuration)
- [CLI Usage Guide](#cli-usage-guide)
- [Python Library Usage](#python-library-usage)
- [API Reference](#api-reference)
- [Examples](#examples)
- [Testing](#testing)
- [Development](#development)
- [Troubleshooting](#troubleshooting)
- [Contributing](#contributing)
- [License](#license)
- [Support](#support)
## Features
### Core Capabilities
- **Complete API Coverage**: Full implementation of GLIMPS Audit API v2.0.4
- **Triple Interface**: Python library, command-line tool, and IDA Pro plugin
- **Binary Analysis**: Submit executables (PE/ELF) for comprehensive analysis
- **Library Detection**: Identify third-party libraries and their versions
- **Dataset Management**: Create, populate, and manage custom reference datasets
- **Vulnerability Correlation**: Match binaries against known vulnerable components
- **IDA Pro Integration**: Direct integration with IDA Pro for reverse engineering workflows
### Technical Features
- **Cross-Platform Support**: Windows, macOS, and Linux compatibility
- **Secure Authentication**: JWT-based authentication with automatic token refresh
- **Comprehensive Testing**: 80%+ code coverage with unit and integration tests
- **Type Hints**: Full type annotation support for better IDE integration
- **Async Support**: Efficient handling of long-running operations
- **Configurable**: Environment variables and configuration files support
## Requirements
- Python 3.8 or higher
- pip package manager
- Active GLIMPS Audit account with API access
- For IDA Pro plugin: IDA Pro 8.5 or higher
### System Dependencies
- Operating System: Windows 10+, macOS 10.14+, or Linux (Ubuntu 18.04+)
- Network: Internet connection for API access
- Storage: ~50MB for installation
## Installation
### From PyPI (Recommended)
```bash
pip install gaudit
```
### From Source (Latest Development)
```bash
# Clone the repository
git clone https://github.com/GLIMPS/gaudit.git
cd gaudit
# Install in production mode
pip install .
# Or install in development mode (editable)
pip install -e .
```
### Development Installation
For contributors and developers:
```bash
# Clone and install with development dependencies
git clone https://github.com/GLIMPS/gaudit.git
cd gaudit
pip install -r requirements-dev.txt
pip install -e .
# Verify installation
gaudit --version
```
### Docker Installation (Alternative)
```dockerfile
FROM python:3.11-slim
RUN pip install gaudit
```
## IDA Pro Plugin
The GAudit package includes a powerful IDA Pro plugin for integrating GLIMPS Audit directly into your reverse engineering workflow.
### Plugin Installation
1. **Install the GAudit package**:
```bash
pip install gaudit
```
2. **Copy the plugin file to IDA Pro plugins directory**:
The plugin file `ida_gaudit.py` is included in the repository root. Copy it to your IDA Pro plugins directory:
**Windows**:
```cmd
copy ida_gaudit.py "C:\Program Files\IDA Pro 8.5\plugins\"
```
**Linux/macOS**:
```bash
cp ida_gaudit.py /path/to/ida/plugins/
```
3. **Restart IDA Pro**
### Plugin Features
The GAudit IDA Pro plugin provides three main features:
#### 1. Document Database
Analyze your IDA database against GLIMPS's extensive library database and automatically apply function names from matched libraries. This feature:
- Builds an ELF from your IDA database
- Uploads it to GLIMPS for analysis
- Allows you to select which matched libraries to use
- Automatically renames functions based on matches
#### 2. Add to Dataset
Upload your analyzed binaries to custom GLIMPS datasets for building your own reference library:
- Export your IDA database as ELF
- Upload to GLIMPS server
- Add to your custom dataset with metadata
- Build your organization's binary knowledge base
#### 3. Build ELF
Export your IDA Pro database as an ELF file for external analysis or sharing:
- Converts IDA database to standard ELF format
- Preserves function information and symbols
- Enables analysis with other tools
### Plugin Usage
After installation, access the plugin through IDA Pro's menu:
**Edit Menu**:
- `Edit → GAudit → Document database` - Analyze and document functions
- `Edit → GAudit → Add database to a dataset` - Add to your dataset
- `Edit → GAudit → Build ELF` - Export as ELF file
**Plugin Menu**:
- `Edit → Plugins → GAudit settings` - Configure server and credentials
### Plugin Configuration
On first use:
1. Go to `Edit → Plugins → GAudit settings`
2. Enter server URL (default: `https://gaudit.glimps.re`)
3. Enter your email and password
4. Check "Ignore server certificate" only for self-signed certificates
5. Click OK to save
Configuration is stored and persists across IDA Pro sessions.
### IDC Script Integration
Generate IDC scripts from the command line for batch processing:
```bash
# Analyze a binary
gaudit audit create --group "ida" --file /path/to/binary.exe
# Get audit details to find library IDs
gaudit audit get <audit-id>
# Generate IDC script with selected libraries
gaudit audit idc <audit-id> \
--lib "lib_id_1" \
--lib "lib_id_2" \
--output documentation.idc
# In IDA Pro: File → Script file... → Select documentation.idc
```
## Quick Start
### First-Time Setup
1. **Obtain API Credentials**
- Sign up at [https://gaudit.glimps.re](https://gaudit.glimps.re)
- Navigate to your profile settings
- Note your email and password
2. **Configure Authentication**
Using CLI (Interactive):
```bash
gaudit login
# Enter email and password when prompted
```
Using Environment Variables:
```bash
export GLIMPS_AUDIT_URL="https://gaudit.glimps.re"
export GLIMPS_AUDIT_EMAIL="your-email@example.com"
export GLIMPS_AUDIT_PASSWORD="your-password"
```
3. **Verify Setup**
```bash
gaudit whoami
```
### Basic Workflow Example
```bash
# 1. Upload and analyze a binary
gaudit audit create --group security --file /path/to/application.exe --comment "Security scan"
# 2. Check analysis status
gaudit audit list --filter "application.exe"
# 3. Retrieve detailed results
gaudit audit get <audit-id>
# 4. Generate IDA Pro documentation
gaudit audit idc <audit-id> --lib <library-id> --output functions.idc
# 5. Export results as JSON
gaudit audit get <audit-id> --json > results.json
# 6. Build a custom dataset for future comparisons
gaudit dataset create my_libs --comment "Our validated libraries"
# 7. Add files to your dataset
gaudit dataset add-files my_libs \
--project "ValidatedLibs" \
--file "/path/to/safe_lib_v1.dll@1.0.0" \
--file "/path/to/safe_lib_v2.dll@2.0.0" \
--auto-upload \
--license "MIT"
```
## Configuration
### Configuration Hierarchy
The client uses the following priority order for configuration:
1. Command-line arguments (highest priority)
2. Environment variables
3. Configuration file
4. Default values (lowest priority)
### Configuration File Location
| Platform | Location |
|----------|----------|
| Linux | `~/.config/gaudit/config.json` |
| macOS | `~/Library/Application Support/gaudit/config.json` |
| Windows | `%APPDATA%\gaudit\config.json` |
### Configuration File Format
```json
{
"url": "https://gaudit.glimps.re",
"email": "user@example.com",
"token": "eyJ0eXAiOiJKV1QiLCJhbGc...",
"verify_ssl": true
}
```
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `GLIMPS_AUDIT_URL` | API server URL | `https://gaudit.glimps.re` |
| `GLIMPS_AUDIT_EMAIL` | User email for authentication | None |
| `GLIMPS_AUDIT_PASSWORD` | User password for authentication | None |
| `GLIMPS_AUDIT_TOKEN` | JWT authentication token | None |
| `GLIMPS_AUDIT_VERIFY_SSL` | SSL certificate verification | `true` |
## CLI Usage Guide
### Authentication Commands
```bash
# Interactive login
gaudit login
# Login with credentials
gaudit login --email user@example.com --password secret
# Check authentication status
gaudit whoami
# Change password
gaudit change-password
# Logout (clear stored credentials)
gaudit logout
```
### Audit Management
```bash
# Upload a file for analysis
gaudit audit upload /path/to/binary.exe
# Create audit with uploaded file
gaudit audit create \
--group "production" \
--file "6cbce50e71d810cd:binary.exe" \
--comment "Production release v1.2.3" \
--dataset "default,custom"
# Create audit with direct file upload
gaudit audit create \
--group "testing" \
--file /path/to/binary.exe \
--comment "Test build"
# List all audits
gaudit audit list
# Filter audits
gaudit audit list --filter "keyword" --size 50 --sort desc
# Get detailed audit results
gaudit audit get 51d6999a-d54d-4ac7-8af3-0425d24fa615
# Generate IDC script for IDA Pro
gaudit audit idc <audit-id> \
--lib "lib_id_1" \
--lib "lib_id_2" \
--output documentation.idc
# Download analyzed binary
gaudit audit download <audit-id> <file-id> --output recovered.exe
# Delete an audit
gaudit audit delete <audit-id>
# List available audit groups
gaudit audit groups
```
### Dataset Management
```bash
# List all datasets
gaudit dataset list
# Create a new dataset
gaudit dataset create my_reference --comment "Custom reference libraries"
# List dataset entries
gaudit dataset entries my_reference --filter "openssl"
# Upload a file for dataset use
gaudit dataset upload /path/to/library.dll
# Add files to dataset (multiple methods)
# Method 1: Using pre-uploaded file IDs
gaudit dataset add-files my_reference \
--project "MyProject" \
--file "6cbce50e71d810cd:library.dll@1.0.0" \
--license "MIT"
# Method 2: Auto-upload local files
gaudit dataset add-files my_reference \
--project "MyProject" \
--file "/path/to/library.dll@1.0.0" \
--file "/path/to/helper.dll@1.0.1" \
--auto-upload \
--license "MIT"
# Update dataset to latest version
gaudit dataset update my_reference
# Delete a dataset
gaudit dataset delete my_reference
```
### Library Search
```bash
# List all libraries
gaudit library list
# Search for specific libraries
gaudit library list --filter "openssl" --size 100
# Filter by architecture
gaudit library list --filter "amd64" --sort desc
```
### User Management
```bash
# View user statistics
gaudit user stats
# Delete all user analyses
gaudit user delete-analyses
```
## Python Library Usage
### Basic Example
```python
from gaudit import GlimpsAuditClient
# Initialize client
client = GlimpsAuditClient(
url="https://gaudit.glimps.re",
verify_ssl=True
)
# Authenticate
result = client.login("user@example.com", "password")
print(f"Logged in as: {result['name']}")
# Upload and analyze a file
upload = client.upload_file_for_audit("application.exe")
file_id = upload["id"]
# Create audit with analysis parameters
audit = client.create_audit(
group="security-scans",
files={file_id: "application.exe"},
comment="Automated security scan",
services={
"GlimpsLibCorrelate": {
"dataset": "default,custom",
"confidence": "1",
"valid": "true"
}
}
)
audit_id = audit["aids"][0]
print(f"Audit created: {audit_id}")
# Wait for completion and get results
import time
while True:
details = client.get_audit(audit_id)
if details["audit"].get("done_at"):
break
time.sleep(5)
# Process results
audit_data = details["audit"]
print(f"Analysis completed: {audit_data['done_at']}")
print(f"Libraries found: {len(audit_data.get('libraries', []))}")
for library in audit_data.get("libraries", []):
print(f"- {library['name']}")
for file, versions in library.get("files", {}).items():
for version in versions:
print(f" {file} v{version['version']} (score: {version['score']})")
```
### Advanced Usage with IDA Pro Integration
```python
#!/usr/bin/env python3
"""
Analyze binaries and generate IDA Pro documentation
"""
from gaudit import GlimpsAuditClient
from pathlib import Path
import time
class IDADocumentationGenerator:
def __init__(self, client: GlimpsAuditClient):
self.client = client
def analyze_and_document(self, binary_path: Path, output_dir: Path):
"""
Analyze a binary and generate IDA Pro documentation
Args:
binary_path: Path to the binary file
output_dir: Directory for output files
"""
# Upload and analyze
print(f"Uploading {binary_path.name}...")
upload = self.client.upload_file_for_audit(str(binary_path))
audit = self.client.create_audit(
group="ida-documentation",
files={upload["id"]: binary_path.name},
comment="IDA Pro documentation generation",
services={
"GlimpsLibCorrelate": {
"dataset": "default",
"confidence": "1",
"valid": "true"
}
}
)
audit_id = audit["aids"][0]
print(f"Analysis started: {audit_id}")
# Wait for completion
while True:
details = self.client.get_audit(audit_id)
if details["audit"].get("done_at"):
break
print(".", end="", flush=True)
time.sleep(2)
print("\nAnalysis complete!")
# Extract library IDs
libraries = details["audit"].get("libraries", [])
if not libraries:
print("No libraries matched")
return None
# Select top matches
library_ids = []
for lib in libraries[:10]: # Top 10 matches
for file, versions in lib.get("files", {}).items():
for version in versions:
if version["score"] > 80: # High confidence matches
library_ids.append(version["id"])
print(f"Including: {lib['name']} v{version['version']} (score: {version['score']:.1f})")
if not library_ids:
print("No high-confidence matches found")
return None
# Generate IDC script
idc_content = self.client.generate_idc(audit_id, library_ids)
# Save IDC file
output_dir.mkdir(parents=True, exist_ok=True)
idc_path = output_dir / f"{binary_path.stem}_documentation.idc"
with open(idc_path, "w") as f:
f.write(idc_content)
print(f"\nIDC script saved to: {idc_path}")
print(f"To use in IDA Pro:")
print(f" 1. Open {binary_path.name} in IDA Pro")
print(f" 2. File → Script file...")
print(f" 3. Select {idc_path}")
return idc_path
def batch_analyze(self, directory: Path, output_dir: Path):
"""
Analyze all executables in a directory
Args:
directory: Directory containing binaries
output_dir: Directory for IDC output files
"""
exe_patterns = ["*.exe", "*.dll", "*.so", "*.elf"]
binaries = []
for pattern in exe_patterns:
binaries.extend(directory.glob(pattern))
print(f"Found {len(binaries)} binaries to analyze")
results = []
for binary in binaries:
print(f"\n{'='*60}")
print(f"Processing: {binary.name}")
print(f"{'='*60}")
try:
idc_path = self.analyze_and_document(binary, output_dir)
results.append({
"binary": binary.name,
"idc_path": str(idc_path) if idc_path else None,
"status": "success" if idc_path else "no_matches"
})
except Exception as e:
print(f"Error processing {binary.name}: {e}")
results.append({
"binary": binary.name,
"error": str(e),
"status": "failed"
})
# Summary
print(f"\n{'='*60}")
print("BATCH ANALYSIS COMPLETE")
print(f"{'='*60}")
successful = sum(1 for r in results if r["status"] == "success")
no_matches = sum(1 for r in results if r["status"] == "no_matches")
failed = sum(1 for r in results if r["status"] == "failed")
print(f"Total processed: {len(results)}")
print(f" Successful: {successful}")
print(f" No matches: {no_matches}")
print(f" Failed: {failed}")
return results
# Usage example
if __name__ == "__main__":
# Initialize client
client = GlimpsAuditClient()
client.login("analyst@example.com", "password")
generator = IDADocumentationGenerator(client)
# Single file analysis
generator.analyze_and_document(
Path("/path/to/unknown_binary.exe"),
Path("/path/to/idc_output")
)
# Batch analysis
generator.batch_analyze(
Path("/path/to/binaries"),
Path("/path/to/idc_output")
)
```
## API Reference
### Client Initialization
```python
GlimpsAuditClient(url: str = "https://gaudit.glimps.re", verify_ssl: bool = True)
```
### Authentication Methods
| Method | Description | Returns |
|--------|-------------|---------|
| `login(email, password)` | Authenticate with credentials | Auth response with token |
| `refresh_token()` | Refresh authentication token | New auth response |
| `is_token_valid()` | Check token validity | Boolean |
| `ensure_authenticated()` | Ensure valid authentication | None or raises exception |
### Audit Methods
| Method | Description | Returns |
|--------|-------------|---------|
| `create_audit(group, files, comment, services)` | Create new audit | Audit creation response |
| `list_audits(filter, sort_order, page_number, page_size)` | List audits | Paginated audit list |
| `get_audit(audit_id)` | Get audit details | Full audit information |
| `delete_audit(audit_id)` | Delete an audit | Status response |
| `upload_file_for_audit(file_path)` | Upload file for analysis | File upload response |
| `download_audit_binary(audit_id, file_id, save_path)` | Download analyzed file | Binary content |
| `generate_idc(audit_id, library_ids)` | Generate IDA IDC script | IDC script content |
| `get_audit_groups()` | Get available audit groups | List of group names |
### Dataset Methods
| Method | Description | Parameters | Returns |
|--------|-------------|------------|---------|
| `list_datasets()` | List all datasets | None | Dataset list |
| `create_dataset(name, comment)` | Create new dataset | `name` (required), `comment` (optional) | Dataset creation response |
| `get_dataset_entries(dataset_name, size, from_index, filter)` | Get dataset entries | `dataset_name` (required), others optional | Paginated entry list |
| `add_dataset_entries(dataset_name, project_name, files, ...)` | Add entries to dataset | `dataset_name`, `project_name`, `files` (all required) | Addition status |
| `delete_dataset(dataset_name)` | Delete a dataset | `dataset_name` (required) | Status response |
| `update_dataset(dataset_name)` | Update dataset version | `dataset_name` (required) | None (202 Accepted) |
| `upload_file_for_dataset(file_path)` | Upload file for dataset | `file_path` (required) | Upload response with ID |
## Examples
### Example: Vulnerability Scanner
```python
#!/usr/bin/env python3
"""
Scan binaries for known vulnerable libraries
"""
from gaudit import GlimpsAuditClient
from pathlib import Path
import sys
def scan_for_vulnerabilities(client: GlimpsAuditClient, file_path: Path):
"""Scan a binary for vulnerable libraries"""
print(f"Scanning {file_path.name}...")
# Upload file
upload = client.upload_file_for_audit(str(file_path))
# Create audit with vulnerability detection
audit = client.create_audit(
group="vulnerability-scan",
files={upload["id"]: file_path.name},
comment="Vulnerability scan",
services={
"GlimpsLibCorrelate": {
"dataset": "vulnerable_libs",
"confidence": "1",
"valid": "true"
}
}
)
audit_id = audit["aids"][0]
# Wait for completion
import time
max_wait = 300 # 5 minutes
start_time = time.time()
while time.time() - start_time < max_wait:
details = client.get_audit(audit_id)
if details["audit"].get("done_at"):
break
time.sleep(5)
# Check for vulnerable libraries
vulnerabilities = []
for lib in details["audit"].get("libraries", []):
# Check against vulnerability database
if lib["name"] in ["log4j", "openssl-1.0.1"]:
vulnerabilities.append({
"library": lib["name"],
"severity": "HIGH",
"description": f"Known vulnerable library: {lib['name']}"
})
return vulnerabilities
# Usage
if __name__ == "__main__":
client = GlimpsAuditClient()
client.login("security@example.com", "password")
vulnerabilities = scan_for_vulnerabilities(
client,
Path(sys.argv[1])
)
if vulnerabilities:
print("VULNERABILITIES FOUND:")
for vuln in vulnerabilities:
print(f" - {vuln['library']}: {vuln['description']}")
sys.exit(1)
else:
print("No known vulnerabilities detected")
sys.exit(0)
```
### Example: Complete Dataset Management
```python
#!/usr/bin/env python3
"""
Complete example of dataset creation and management
"""
from gaudit import GlimpsAuditClient
from pathlib import Path
def create_and_populate_dataset(client: GlimpsAuditClient):
"""Example showing complete dataset workflow with required fields"""
# Step 1: Create a new dataset
dataset_name = "custom_libs_v1"
dataset = client.create_dataset(
name=dataset_name,
comment="Custom library dataset for internal projects"
)
print(f"Created dataset: {dataset['kind']}")
# Step 2: Upload files to be added to the dataset
library_files = [
"/path/to/mylib_v1.0.dll",
"/path/to/mylib_v1.1.dll",
"/path/to/helper.so"
]
uploaded_files = []
for file_path in library_files:
upload = client.upload_file_for_dataset(file_path)
uploaded_files.append({
"id": upload["id"],
"binary_name": Path(file_path).name,
"version": "1.0.0"
})
print(f"Uploaded: {Path(file_path).name} -> {upload['id']}")
# Step 3: Add files to dataset with REQUIRED fields
result = client.add_dataset_entries(
dataset_name=dataset_name,
project_name="InternalProject", # REQUIRED
files=uploaded_files, # REQUIRED
source_name="Internal Build System",
license="Proprietary",
home_page="https://internal.example.com/project",
project_description="Internal libraries for company projects"
)
if result["status"]:
print(f"Successfully added {len(uploaded_files)} files to dataset")
# Step 4: Verify dataset entries
entries = client.get_dataset_entries(dataset_name)
print(f"\nDataset now contains {entries['count']} entries:")
for entry in entries.get("entries", []):
print(f" - {entry['binary_name']} ({entry['architecture']})")
print(f" SHA256: {entry['sha256']}")
return dataset_name
# Usage
if __name__ == "__main__":
client = GlimpsAuditClient()
client.login("admin@example.com", "password")
dataset_name = create_and_populate_dataset(client)
print(f"\nDataset '{dataset_name}' created and populated successfully!")
```
### Example: Compliance Reporter
```python
#!/usr/bin/env python3
"""
Generate compliance reports for analyzed binaries
"""
from gaudit import GlimpsAuditClient
from datetime import datetime
import json
def generate_compliance_report(client: GlimpsAuditClient, audit_id: str):
"""Generate a compliance report from audit results"""
details = client.get_audit(audit_id)
audit = details["audit"]
report = {
"report_date": datetime.now().isoformat(),
"audit_id": audit_id,
"file": {
"name": audit["filename"],
"type": audit["filetype"],
"architecture": audit["arch"],
"size": audit["size"],
"hashes": {h["Name"]: h["Value"] for h in audit["hashes"]}
},
"compliance": {
"total_libraries": len(audit.get("libraries", [])),
"licensed_libraries": [],
"unlicensed_libraries": [],
"copyleft_licenses": [],
"permissive_licenses": []
}
}
# Analyze licenses
for lib in audit.get("libraries", []):
for file, versions in lib.get("files", {}).items():
for version in versions:
license_type = version.get("license", "Unknown")
lib_entry = {
"name": lib["name"],
"file": file,
"version": version["version"],
"license": license_type
}
if license_type == "Unknown":
report["compliance"]["unlicensed_libraries"].append(lib_entry)
else:
report["compliance"]["licensed_libraries"].append(lib_entry)
# Categorize by license type
if license_type in ["GPL", "LGPL", "AGPL"]:
report["compliance"]["copyleft_licenses"].append(lib_entry)
elif license_type in ["MIT", "BSD", "Apache"]:
report["compliance"]["permissive_licenses"].append(lib_entry)
# Add compliance summary
report["summary"] = {
"compliant": len(report["compliance"]["unlicensed_libraries"]) == 0,
"copyleft_risk": len(report["compliance"]["copyleft_licenses"]) > 0,
"license_coverage": (
len(report["compliance"]["licensed_libraries"]) /
report["compliance"]["total_libraries"] * 100
if report["compliance"]["total_libraries"] > 0 else 0
)
}
return report
# Usage
client = GlimpsAuditClient()
client.login("compliance@example.com", "password")
report = generate_compliance_report(client, "audit-id-123")
print(json.dumps(report, indent=2))
```
## Testing
### Running Tests
```bash
# Run all unit tests
pytest
# Run with coverage report
pytest --cov=gaudit --cov-report=html
# Run specific test module
pytest tests/test_client.py
# Run tests matching pattern
pytest -k "test_login"
# Run with verbose output
pytest -v
# Using the test runner script
python run_tests.py --coverage
```
### Integration Testing
Integration tests require a live API server:
```bash
# Set environment variables
export GLIMPS_TEST_API_URL="https://gaudit.glimps.re"
export GLIMPS_TEST_EMAIL="test@example.com"
export GLIMPS_TEST_PASSWORD="test-password"
# Run integration tests
pytest -m integration
# Or using the test runner
python run_tests.py --integration
```
### Test Coverage
Current test coverage targets:
- Minimum: 80%
- Target: 90%+
- Current: Check with `pytest --cov=gaudit`
## Development
### Project Structure
```
gaudit/
├── src/gaudit/ # Source code
│ ├── __init__.py # Package initialization
│ ├── client.py # API client implementation
│ ├── cli.py # CLI implementation
│ ├── config.py # Configuration management
│ ├── elfbuilder.py # ELF building utilities
│ └── ida/ # IDA Pro plugin modules
│ ├── __init__.py
│ ├── plugin.py # Main plugin code
│ ├── idbtoelf.py # IDA to ELF conversion
│ ├── ui.py # Plugin UI components
│ └── version.py # Plugin version
├── ida_gaudit.py # IDA Pro plugin entry point
├── tests/ # Test suite
│ ├── test_client.py # Client tests
│ ├── test_cli.py # CLI tests
│ └── utils.py # Test utilities
├── docs/ # Documentation
│ └── openapi.yml # API specification
├── examples/ # Example scripts
├── requirements.txt # Production dependencies
└── pyproject.toml # Project configuration
```
### Development Setup
```bash
# Clone repository
git clone https://github.com/GLIMPS/gaudit.git
cd gaudit
# Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install development dependencies
pip install -r requirements-dev.txt
pip install -e .
# Run linting
ruff check src tests
# Format code
ruff format src tests
# Run tests
pytest
```
### Code Style
This project follows:
- PEP 8 style guide
- Type hints for all public methods
- Docstrings for all modules, classes, and functions
- Maximum line length: 119 characters
### Pre-commit Hooks
```bash
# Install pre-commit
pip install pre-commit
# Install hooks
pre-commit install
# Run manually
pre-commit run --all-files
```
## Troubleshooting
### Common Issues
#### Authentication Failures
**Problem**: "API Error 401: Unauthorized"
**Solutions**:
1. Verify credentials are correct
2. Check if token has expired: `gaudit whoami`
3. Re-authenticate: `gaudit login`
4. Verify API URL is correct
#### SSL Certificate Errors
**Problem**: SSL certificate verification failed
**Solutions**:
1. Update certificates: `pip install --upgrade certifi`
2. For testing only: `gaudit --insecure login`
3. Set custom CA bundle: `export REQUESTS_CA_BUNDLE=/path/to/ca-bundle.crt`
#### IDA Pro Plugin Not Loading
**Problem**: Plugin doesn't appear in IDA Pro menus
**Solutions**:
1. Verify `ida_gaudit.py` is in the correct plugins directory
2. Check IDA Pro console for error messages
3. Ensure `gaudit` package is installed in IDA's Python environment
4. Verify Python version compatibility (3.8+)
#### File Upload Errors
**Problem**: "invalid file type" or "invalid file size"
**Solutions**:
1. Verify file is PE (Windows) or ELF (Linux) executable
2. Check file size is under 20MB limit
3. Ensure file is not corrupted: `file /path/to/binary`
#### Dataset Entry Errors
**Problem**: "Bad Request" when adding dataset entries
**Solutions**:
1. Ensure `project_name` is provided (REQUIRED field)
2. Ensure `files` array is provided (REQUIRED field)
3. Verify file IDs exist from prior upload
4. Check dataset name exists and you have permissions
**Example of correct usage**:
```python
client.add_dataset_entries(
dataset_name="my_dataset",
project_name="MyProject", # REQUIRED
files=[{ # REQUIRED
"id": "file_id_from_upload",
"binary_name": "library.dll",
"version": "1.0.0"
}]
)
```
#### Connection Timeouts
**Problem**: Requests timing out
**Solutions**:
1. Check network connectivity
2. Verify firewall settings
3. Try using a different DNS server
4. Contact your network administrator
### Debug Mode
Enable detailed logging for troubleshooting:
```python
import logging
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
# Now client will show detailed requests/responses
client = GlimpsAuditClient()
```
### Getting Help
1. Check the [FAQ](https://support.glimps.re/faq)
2. Search [existing issues](https://github.com/GLIMPS/gaudit/issues)
3. Contact support: support@glimps.re
4. Join our [community forum](https://community.glimps.re)
## Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
### Quick Contribution Guide
1. Fork the repository
2. Create a feature branch:
```bash
git checkout -b feature/amazing-feature
```
3. Make your changes and add tests
4. Ensure tests pass:
```bash
pytest
ruff check .
```
5. Commit with descriptive message:
```bash
git commit -m "Add amazing feature: description of changes"
```
6. Push to your fork:
```bash
git push origin feature/amazing-feature
```
7. Open a Pull Request
### Development Guidelines
- Write tests for new features
- Update documentation for API changes
- Follow existing code style
- Add type hints for new functions
- Keep commits atomic and descriptive
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
## Support
### Resources
- **Documentation**: [https://docs.glimps.re](https://docs.glimps.re)
- **API Reference**: [https://api.glimps.re/docs](https://api.glimps.re/docs)
- **Support Portal**: [https://support.glimps.re](https://support.glimps.re)
- **Email**: support@glimps.re
- **Company Website**: [https://www.glimps.re](https://www.glimps.re)
### Professional Support
For enterprise support, custom integrations, or training, contact contact@glimps.re
## Changelog
### Version 0.1.1 (Latest)
- **IDA Pro Plugin**: Full integration with IDA Pro for reverse engineering workflows
- **IDC Generation**: Generate IDA Pro documentation scripts from audit results
- **Enhanced Dataset Management**: Added `dataset upload` and `dataset add-files` commands
- **Auto-Upload Feature**: New `--auto-upload` flag for automatic file upload when adding to datasets
- **Improved Error Messages**: Better handling of required fields with helpful error messages
- **ELF Builder**: Added support for building ELF files from IDA databases
See [CHANGELOG.md](CHANGELOG.md) for complete version history and release notes.
## Acknowledgments
- GLIMPS development team for the API platform
- Open source community for invaluable tools and libraries
- All contributors and users providing feedback and improvements
---
**Copyright (c) 2025 GLIMPS** - Prevent tomorrow's threats today.
Raw data
{
"_id": null,
"home_page": null,
"name": "gaudit",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "python, glimps, reverse engineering, malware, binary, analysis, ida",
"author": null,
"author_email": "GLIMPS dev core team <contact@glimps.re>",
"download_url": "https://files.pythonhosted.org/packages/3a/c6/d41e688fea26a12b8705e41ea20afc5b60eab58df188fb903354c1d204d3/gaudit-0.3.1.tar.gz",
"platform": null,
"description": "# GLIMPS Audit Client Library and CLI\n\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/astral-sh/ruff)\n[](https://badge.fury.io/py/gaudit)\n\nA comprehensive Python client library, command-line interface, and IDA Pro plugin for interacting with the GLIMPS Audit API v2.0.4. This tool enables seamless integration with GLIMPS's binary analysis platform for software composition analysis and vulnerability detection.\n\n## Table of Contents\n\n- [Features](#features)\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [IDA Pro Plugin](#ida-pro-plugin)\n- [Quick Start](#quick-start)\n- [Configuration](#configuration)\n- [CLI Usage Guide](#cli-usage-guide)\n- [Python Library Usage](#python-library-usage)\n- [API Reference](#api-reference)\n- [Examples](#examples)\n- [Testing](#testing)\n- [Development](#development)\n- [Troubleshooting](#troubleshooting)\n- [Contributing](#contributing)\n- [License](#license)\n- [Support](#support)\n\n## Features\n\n### Core Capabilities\n- **Complete API Coverage**: Full implementation of GLIMPS Audit API v2.0.4\n- **Triple Interface**: Python library, command-line tool, and IDA Pro plugin\n- **Binary Analysis**: Submit executables (PE/ELF) for comprehensive analysis\n- **Library Detection**: Identify third-party libraries and their versions\n- **Dataset Management**: Create, populate, and manage custom reference datasets\n- **Vulnerability Correlation**: Match binaries against known vulnerable components\n- **IDA Pro Integration**: Direct integration with IDA Pro for reverse engineering workflows\n\n### Technical Features\n- **Cross-Platform Support**: Windows, macOS, and Linux compatibility\n- **Secure Authentication**: JWT-based authentication with automatic token refresh\n- **Comprehensive Testing**: 80%+ code coverage with unit and integration tests\n- **Type Hints**: Full type annotation support for better IDE integration\n- **Async Support**: Efficient handling of long-running operations\n- **Configurable**: Environment variables and configuration files support\n\n## Requirements\n\n- Python 3.8 or higher\n- pip package manager\n- Active GLIMPS Audit account with API access\n- For IDA Pro plugin: IDA Pro 8.5 or higher\n\n### System Dependencies\n- Operating System: Windows 10+, macOS 10.14+, or Linux (Ubuntu 18.04+)\n- Network: Internet connection for API access\n- Storage: ~50MB for installation\n\n## Installation\n\n### From PyPI (Recommended)\n\n```bash\npip install gaudit\n```\n\n### From Source (Latest Development)\n\n```bash\n# Clone the repository\ngit clone https://github.com/GLIMPS/gaudit.git\ncd gaudit\n\n# Install in production mode\npip install .\n\n# Or install in development mode (editable)\npip install -e .\n```\n\n### Development Installation\n\nFor contributors and developers:\n\n```bash\n# Clone and install with development dependencies\ngit clone https://github.com/GLIMPS/gaudit.git\ncd gaudit\npip install -r requirements-dev.txt\npip install -e .\n\n# Verify installation\ngaudit --version\n```\n\n### Docker Installation (Alternative)\n\n```dockerfile\nFROM python:3.11-slim\nRUN pip install gaudit\n```\n\n## IDA Pro Plugin\n\nThe GAudit package includes a powerful IDA Pro plugin for integrating GLIMPS Audit directly into your reverse engineering workflow.\n\n### Plugin Installation\n\n1. **Install the GAudit package**:\n ```bash\n pip install gaudit\n ```\n\n2. **Copy the plugin file to IDA Pro plugins directory**:\n \n The plugin file `ida_gaudit.py` is included in the repository root. Copy it to your IDA Pro plugins directory:\n \n **Windows**:\n ```cmd\n copy ida_gaudit.py \"C:\\Program Files\\IDA Pro 8.5\\plugins\\\"\n ```\n \n **Linux/macOS**:\n ```bash\n cp ida_gaudit.py /path/to/ida/plugins/\n ```\n\n3. **Restart IDA Pro**\n\n### Plugin Features\n\nThe GAudit IDA Pro plugin provides three main features:\n\n#### 1. Document Database\nAnalyze your IDA database against GLIMPS's extensive library database and automatically apply function names from matched libraries. This feature:\n- Builds an ELF from your IDA database\n- Uploads it to GLIMPS for analysis\n- Allows you to select which matched libraries to use\n- Automatically renames functions based on matches\n\n#### 2. Add to Dataset\nUpload your analyzed binaries to custom GLIMPS datasets for building your own reference library:\n- Export your IDA database as ELF\n- Upload to GLIMPS server\n- Add to your custom dataset with metadata\n- Build your organization's binary knowledge base\n\n#### 3. Build ELF\nExport your IDA Pro database as an ELF file for external analysis or sharing:\n- Converts IDA database to standard ELF format\n- Preserves function information and symbols\n- Enables analysis with other tools\n\n### Plugin Usage\n\nAfter installation, access the plugin through IDA Pro's menu:\n\n**Edit Menu**:\n- `Edit \u2192 GAudit \u2192 Document database` - Analyze and document functions\n- `Edit \u2192 GAudit \u2192 Add database to a dataset` - Add to your dataset\n- `Edit \u2192 GAudit \u2192 Build ELF` - Export as ELF file\n\n**Plugin Menu**:\n- `Edit \u2192 Plugins \u2192 GAudit settings` - Configure server and credentials\n\n### Plugin Configuration\n\nOn first use:\n1. Go to `Edit \u2192 Plugins \u2192 GAudit settings`\n2. Enter server URL (default: `https://gaudit.glimps.re`)\n3. Enter your email and password\n4. Check \"Ignore server certificate\" only for self-signed certificates\n5. Click OK to save\n\nConfiguration is stored and persists across IDA Pro sessions.\n\n### IDC Script Integration\n\nGenerate IDC scripts from the command line for batch processing:\n\n```bash\n# Analyze a binary\ngaudit audit create --group \"ida\" --file /path/to/binary.exe\n\n# Get audit details to find library IDs\ngaudit audit get <audit-id>\n\n# Generate IDC script with selected libraries\ngaudit audit idc <audit-id> \\\n --lib \"lib_id_1\" \\\n --lib \"lib_id_2\" \\\n --output documentation.idc\n\n# In IDA Pro: File \u2192 Script file... \u2192 Select documentation.idc\n```\n\n## Quick Start\n\n### First-Time Setup\n\n1. **Obtain API Credentials**\n - Sign up at [https://gaudit.glimps.re](https://gaudit.glimps.re)\n - Navigate to your profile settings\n - Note your email and password\n\n2. **Configure Authentication**\n\n Using CLI (Interactive):\n ```bash\n gaudit login\n # Enter email and password when prompted\n ```\n\n Using Environment Variables:\n ```bash\n export GLIMPS_AUDIT_URL=\"https://gaudit.glimps.re\"\n export GLIMPS_AUDIT_EMAIL=\"your-email@example.com\"\n export GLIMPS_AUDIT_PASSWORD=\"your-password\"\n ```\n\n3. **Verify Setup**\n ```bash\n gaudit whoami\n ```\n\n### Basic Workflow Example\n\n```bash\n# 1. Upload and analyze a binary\ngaudit audit create --group security --file /path/to/application.exe --comment \"Security scan\"\n\n# 2. Check analysis status\ngaudit audit list --filter \"application.exe\"\n\n# 3. Retrieve detailed results\ngaudit audit get <audit-id>\n\n# 4. Generate IDA Pro documentation\ngaudit audit idc <audit-id> --lib <library-id> --output functions.idc\n\n# 5. Export results as JSON\ngaudit audit get <audit-id> --json > results.json\n\n# 6. Build a custom dataset for future comparisons\ngaudit dataset create my_libs --comment \"Our validated libraries\"\n\n# 7. Add files to your dataset\ngaudit dataset add-files my_libs \\\n --project \"ValidatedLibs\" \\\n --file \"/path/to/safe_lib_v1.dll@1.0.0\" \\\n --file \"/path/to/safe_lib_v2.dll@2.0.0\" \\\n --auto-upload \\\n --license \"MIT\"\n```\n\n## Configuration\n\n### Configuration Hierarchy\n\nThe client uses the following priority order for configuration:\n1. Command-line arguments (highest priority)\n2. Environment variables\n3. Configuration file\n4. Default values (lowest priority)\n\n### Configuration File Location\n\n| Platform | Location |\n|----------|----------|\n| Linux | `~/.config/gaudit/config.json` |\n| macOS | `~/Library/Application Support/gaudit/config.json` |\n| Windows | `%APPDATA%\\gaudit\\config.json` |\n\n### Configuration File Format\n\n```json\n{\n \"url\": \"https://gaudit.glimps.re\",\n \"email\": \"user@example.com\",\n \"token\": \"eyJ0eXAiOiJKV1QiLCJhbGc...\",\n \"verify_ssl\": true\n}\n```\n\n### Environment Variables\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `GLIMPS_AUDIT_URL` | API server URL | `https://gaudit.glimps.re` |\n| `GLIMPS_AUDIT_EMAIL` | User email for authentication | None |\n| `GLIMPS_AUDIT_PASSWORD` | User password for authentication | None |\n| `GLIMPS_AUDIT_TOKEN` | JWT authentication token | None |\n| `GLIMPS_AUDIT_VERIFY_SSL` | SSL certificate verification | `true` |\n\n## CLI Usage Guide\n\n### Authentication Commands\n\n```bash\n# Interactive login\ngaudit login\n\n# Login with credentials\ngaudit login --email user@example.com --password secret\n\n# Check authentication status\ngaudit whoami\n\n# Change password\ngaudit change-password\n\n# Logout (clear stored credentials)\ngaudit logout\n```\n\n### Audit Management\n\n```bash\n# Upload a file for analysis\ngaudit audit upload /path/to/binary.exe\n\n# Create audit with uploaded file\ngaudit audit create \\\n --group \"production\" \\\n --file \"6cbce50e71d810cd:binary.exe\" \\\n --comment \"Production release v1.2.3\" \\\n --dataset \"default,custom\"\n\n# Create audit with direct file upload\ngaudit audit create \\\n --group \"testing\" \\\n --file /path/to/binary.exe \\\n --comment \"Test build\"\n\n# List all audits\ngaudit audit list\n\n# Filter audits\ngaudit audit list --filter \"keyword\" --size 50 --sort desc\n\n# Get detailed audit results\ngaudit audit get 51d6999a-d54d-4ac7-8af3-0425d24fa615\n\n# Generate IDC script for IDA Pro\ngaudit audit idc <audit-id> \\\n --lib \"lib_id_1\" \\\n --lib \"lib_id_2\" \\\n --output documentation.idc\n\n# Download analyzed binary\ngaudit audit download <audit-id> <file-id> --output recovered.exe\n\n# Delete an audit\ngaudit audit delete <audit-id>\n\n# List available audit groups\ngaudit audit groups\n```\n\n### Dataset Management\n\n```bash\n# List all datasets\ngaudit dataset list\n\n# Create a new dataset\ngaudit dataset create my_reference --comment \"Custom reference libraries\"\n\n# List dataset entries\ngaudit dataset entries my_reference --filter \"openssl\"\n\n# Upload a file for dataset use\ngaudit dataset upload /path/to/library.dll\n\n# Add files to dataset (multiple methods)\n\n# Method 1: Using pre-uploaded file IDs\ngaudit dataset add-files my_reference \\\n --project \"MyProject\" \\\n --file \"6cbce50e71d810cd:library.dll@1.0.0\" \\\n --license \"MIT\"\n\n# Method 2: Auto-upload local files\ngaudit dataset add-files my_reference \\\n --project \"MyProject\" \\\n --file \"/path/to/library.dll@1.0.0\" \\\n --file \"/path/to/helper.dll@1.0.1\" \\\n --auto-upload \\\n --license \"MIT\"\n\n# Update dataset to latest version\ngaudit dataset update my_reference\n\n# Delete a dataset\ngaudit dataset delete my_reference\n```\n\n### Library Search\n\n```bash\n# List all libraries\ngaudit library list\n\n# Search for specific libraries\ngaudit library list --filter \"openssl\" --size 100\n\n# Filter by architecture\ngaudit library list --filter \"amd64\" --sort desc\n```\n\n### User Management\n\n```bash\n# View user statistics\ngaudit user stats\n\n# Delete all user analyses\ngaudit user delete-analyses\n```\n\n## Python Library Usage\n\n### Basic Example\n\n```python\nfrom gaudit import GlimpsAuditClient\n\n# Initialize client\nclient = GlimpsAuditClient(\n url=\"https://gaudit.glimps.re\",\n verify_ssl=True\n)\n\n# Authenticate\nresult = client.login(\"user@example.com\", \"password\")\nprint(f\"Logged in as: {result['name']}\")\n\n# Upload and analyze a file\nupload = client.upload_file_for_audit(\"application.exe\")\nfile_id = upload[\"id\"]\n\n# Create audit with analysis parameters\naudit = client.create_audit(\n group=\"security-scans\",\n files={file_id: \"application.exe\"},\n comment=\"Automated security scan\",\n services={\n \"GlimpsLibCorrelate\": {\n \"dataset\": \"default,custom\",\n \"confidence\": \"1\",\n \"valid\": \"true\"\n }\n }\n)\n\naudit_id = audit[\"aids\"][0]\nprint(f\"Audit created: {audit_id}\")\n\n# Wait for completion and get results\nimport time\nwhile True:\n details = client.get_audit(audit_id)\n if details[\"audit\"].get(\"done_at\"):\n break\n time.sleep(5)\n\n# Process results\naudit_data = details[\"audit\"]\nprint(f\"Analysis completed: {audit_data['done_at']}\")\nprint(f\"Libraries found: {len(audit_data.get('libraries', []))}\")\n\nfor library in audit_data.get(\"libraries\", []):\n print(f\"- {library['name']}\")\n for file, versions in library.get(\"files\", {}).items():\n for version in versions:\n print(f\" {file} v{version['version']} (score: {version['score']})\")\n```\n\n### Advanced Usage with IDA Pro Integration\n\n```python\n#!/usr/bin/env python3\n\"\"\"\nAnalyze binaries and generate IDA Pro documentation\n\"\"\"\n\nfrom gaudit import GlimpsAuditClient\nfrom pathlib import Path\nimport time\n\nclass IDADocumentationGenerator:\n def __init__(self, client: GlimpsAuditClient):\n self.client = client\n \n def analyze_and_document(self, binary_path: Path, output_dir: Path):\n \"\"\"\n Analyze a binary and generate IDA Pro documentation\n \n Args:\n binary_path: Path to the binary file\n output_dir: Directory for output files\n \"\"\"\n \n # Upload and analyze\n print(f\"Uploading {binary_path.name}...\")\n upload = self.client.upload_file_for_audit(str(binary_path))\n \n audit = self.client.create_audit(\n group=\"ida-documentation\",\n files={upload[\"id\"]: binary_path.name},\n comment=\"IDA Pro documentation generation\",\n services={\n \"GlimpsLibCorrelate\": {\n \"dataset\": \"default\",\n \"confidence\": \"1\",\n \"valid\": \"true\"\n }\n }\n )\n \n audit_id = audit[\"aids\"][0]\n print(f\"Analysis started: {audit_id}\")\n \n # Wait for completion\n while True:\n details = self.client.get_audit(audit_id)\n if details[\"audit\"].get(\"done_at\"):\n break\n print(\".\", end=\"\", flush=True)\n time.sleep(2)\n \n print(\"\\nAnalysis complete!\")\n \n # Extract library IDs\n libraries = details[\"audit\"].get(\"libraries\", [])\n if not libraries:\n print(\"No libraries matched\")\n return None\n \n # Select top matches\n library_ids = []\n for lib in libraries[:10]: # Top 10 matches\n for file, versions in lib.get(\"files\", {}).items():\n for version in versions:\n if version[\"score\"] > 80: # High confidence matches\n library_ids.append(version[\"id\"])\n print(f\"Including: {lib['name']} v{version['version']} (score: {version['score']:.1f})\")\n \n if not library_ids:\n print(\"No high-confidence matches found\")\n return None\n \n # Generate IDC script\n idc_content = self.client.generate_idc(audit_id, library_ids)\n \n # Save IDC file\n output_dir.mkdir(parents=True, exist_ok=True)\n idc_path = output_dir / f\"{binary_path.stem}_documentation.idc\"\n \n with open(idc_path, \"w\") as f:\n f.write(idc_content)\n \n print(f\"\\nIDC script saved to: {idc_path}\")\n print(f\"To use in IDA Pro:\")\n print(f\" 1. Open {binary_path.name} in IDA Pro\")\n print(f\" 2. File \u2192 Script file...\")\n print(f\" 3. Select {idc_path}\")\n \n return idc_path\n \n def batch_analyze(self, directory: Path, output_dir: Path):\n \"\"\"\n Analyze all executables in a directory\n \n Args:\n directory: Directory containing binaries\n output_dir: Directory for IDC output files\n \"\"\"\n \n exe_patterns = [\"*.exe\", \"*.dll\", \"*.so\", \"*.elf\"]\n binaries = []\n \n for pattern in exe_patterns:\n binaries.extend(directory.glob(pattern))\n \n print(f\"Found {len(binaries)} binaries to analyze\")\n \n results = []\n for binary in binaries:\n print(f\"\\n{'='*60}\")\n print(f\"Processing: {binary.name}\")\n print(f\"{'='*60}\")\n \n try:\n idc_path = self.analyze_and_document(binary, output_dir)\n results.append({\n \"binary\": binary.name,\n \"idc_path\": str(idc_path) if idc_path else None,\n \"status\": \"success\" if idc_path else \"no_matches\"\n })\n except Exception as e:\n print(f\"Error processing {binary.name}: {e}\")\n results.append({\n \"binary\": binary.name,\n \"error\": str(e),\n \"status\": \"failed\"\n })\n \n # Summary\n print(f\"\\n{'='*60}\")\n print(\"BATCH ANALYSIS COMPLETE\")\n print(f\"{'='*60}\")\n successful = sum(1 for r in results if r[\"status\"] == \"success\")\n no_matches = sum(1 for r in results if r[\"status\"] == \"no_matches\")\n failed = sum(1 for r in results if r[\"status\"] == \"failed\")\n \n print(f\"Total processed: {len(results)}\")\n print(f\" Successful: {successful}\")\n print(f\" No matches: {no_matches}\")\n print(f\" Failed: {failed}\")\n \n return results\n\n# Usage example\nif __name__ == \"__main__\":\n # Initialize client\n client = GlimpsAuditClient()\n client.login(\"analyst@example.com\", \"password\")\n \n generator = IDADocumentationGenerator(client)\n \n # Single file analysis\n generator.analyze_and_document(\n Path(\"/path/to/unknown_binary.exe\"),\n Path(\"/path/to/idc_output\")\n )\n \n # Batch analysis\n generator.batch_analyze(\n Path(\"/path/to/binaries\"),\n Path(\"/path/to/idc_output\")\n )\n```\n\n## API Reference\n\n### Client Initialization\n\n```python\nGlimpsAuditClient(url: str = \"https://gaudit.glimps.re\", verify_ssl: bool = True)\n```\n\n### Authentication Methods\n\n| Method | Description | Returns |\n|--------|-------------|---------|\n| `login(email, password)` | Authenticate with credentials | Auth response with token |\n| `refresh_token()` | Refresh authentication token | New auth response |\n| `is_token_valid()` | Check token validity | Boolean |\n| `ensure_authenticated()` | Ensure valid authentication | None or raises exception |\n\n### Audit Methods\n\n| Method | Description | Returns |\n|--------|-------------|---------|\n| `create_audit(group, files, comment, services)` | Create new audit | Audit creation response |\n| `list_audits(filter, sort_order, page_number, page_size)` | List audits | Paginated audit list |\n| `get_audit(audit_id)` | Get audit details | Full audit information |\n| `delete_audit(audit_id)` | Delete an audit | Status response |\n| `upload_file_for_audit(file_path)` | Upload file for analysis | File upload response |\n| `download_audit_binary(audit_id, file_id, save_path)` | Download analyzed file | Binary content |\n| `generate_idc(audit_id, library_ids)` | Generate IDA IDC script | IDC script content |\n| `get_audit_groups()` | Get available audit groups | List of group names |\n\n### Dataset Methods\n\n| Method | Description | Parameters | Returns |\n|--------|-------------|------------|---------|\n| `list_datasets()` | List all datasets | None | Dataset list |\n| `create_dataset(name, comment)` | Create new dataset | `name` (required), `comment` (optional) | Dataset creation response |\n| `get_dataset_entries(dataset_name, size, from_index, filter)` | Get dataset entries | `dataset_name` (required), others optional | Paginated entry list |\n| `add_dataset_entries(dataset_name, project_name, files, ...)` | Add entries to dataset | `dataset_name`, `project_name`, `files` (all required) | Addition status |\n| `delete_dataset(dataset_name)` | Delete a dataset | `dataset_name` (required) | Status response |\n| `update_dataset(dataset_name)` | Update dataset version | `dataset_name` (required) | None (202 Accepted) |\n| `upload_file_for_dataset(file_path)` | Upload file for dataset | `file_path` (required) | Upload response with ID |\n\n## Examples\n\n### Example: Vulnerability Scanner\n\n```python\n#!/usr/bin/env python3\n\"\"\"\nScan binaries for known vulnerable libraries\n\"\"\"\n\nfrom gaudit import GlimpsAuditClient\nfrom pathlib import Path\nimport sys\n\ndef scan_for_vulnerabilities(client: GlimpsAuditClient, file_path: Path):\n \"\"\"Scan a binary for vulnerable libraries\"\"\"\n \n print(f\"Scanning {file_path.name}...\")\n \n # Upload file\n upload = client.upload_file_for_audit(str(file_path))\n \n # Create audit with vulnerability detection\n audit = client.create_audit(\n group=\"vulnerability-scan\",\n files={upload[\"id\"]: file_path.name},\n comment=\"Vulnerability scan\",\n services={\n \"GlimpsLibCorrelate\": {\n \"dataset\": \"vulnerable_libs\",\n \"confidence\": \"1\",\n \"valid\": \"true\"\n }\n }\n )\n \n audit_id = audit[\"aids\"][0]\n \n # Wait for completion\n import time\n max_wait = 300 # 5 minutes\n start_time = time.time()\n \n while time.time() - start_time < max_wait:\n details = client.get_audit(audit_id)\n if details[\"audit\"].get(\"done_at\"):\n break\n time.sleep(5)\n \n # Check for vulnerable libraries\n vulnerabilities = []\n for lib in details[\"audit\"].get(\"libraries\", []):\n # Check against vulnerability database\n if lib[\"name\"] in [\"log4j\", \"openssl-1.0.1\"]:\n vulnerabilities.append({\n \"library\": lib[\"name\"],\n \"severity\": \"HIGH\",\n \"description\": f\"Known vulnerable library: {lib['name']}\"\n })\n \n return vulnerabilities\n\n# Usage\nif __name__ == \"__main__\":\n client = GlimpsAuditClient()\n client.login(\"security@example.com\", \"password\")\n \n vulnerabilities = scan_for_vulnerabilities(\n client, \n Path(sys.argv[1])\n )\n \n if vulnerabilities:\n print(\"VULNERABILITIES FOUND:\")\n for vuln in vulnerabilities:\n print(f\" - {vuln['library']}: {vuln['description']}\")\n sys.exit(1)\n else:\n print(\"No known vulnerabilities detected\")\n sys.exit(0)\n```\n\n### Example: Complete Dataset Management\n\n```python\n#!/usr/bin/env python3\n\"\"\"\nComplete example of dataset creation and management\n\"\"\"\n\nfrom gaudit import GlimpsAuditClient\nfrom pathlib import Path\n\ndef create_and_populate_dataset(client: GlimpsAuditClient):\n \"\"\"Example showing complete dataset workflow with required fields\"\"\"\n \n # Step 1: Create a new dataset\n dataset_name = \"custom_libs_v1\"\n dataset = client.create_dataset(\n name=dataset_name,\n comment=\"Custom library dataset for internal projects\"\n )\n print(f\"Created dataset: {dataset['kind']}\")\n \n # Step 2: Upload files to be added to the dataset\n library_files = [\n \"/path/to/mylib_v1.0.dll\",\n \"/path/to/mylib_v1.1.dll\",\n \"/path/to/helper.so\"\n ]\n \n uploaded_files = []\n for file_path in library_files:\n upload = client.upload_file_for_dataset(file_path)\n uploaded_files.append({\n \"id\": upload[\"id\"],\n \"binary_name\": Path(file_path).name,\n \"version\": \"1.0.0\"\n })\n print(f\"Uploaded: {Path(file_path).name} -> {upload['id']}\")\n \n # Step 3: Add files to dataset with REQUIRED fields\n result = client.add_dataset_entries(\n dataset_name=dataset_name,\n project_name=\"InternalProject\", # REQUIRED\n files=uploaded_files, # REQUIRED\n source_name=\"Internal Build System\",\n license=\"Proprietary\",\n home_page=\"https://internal.example.com/project\",\n project_description=\"Internal libraries for company projects\"\n )\n \n if result[\"status\"]:\n print(f\"Successfully added {len(uploaded_files)} files to dataset\")\n \n # Step 4: Verify dataset entries\n entries = client.get_dataset_entries(dataset_name)\n print(f\"\\nDataset now contains {entries['count']} entries:\")\n for entry in entries.get(\"entries\", []):\n print(f\" - {entry['binary_name']} ({entry['architecture']})\")\n print(f\" SHA256: {entry['sha256']}\")\n \n return dataset_name\n\n# Usage\nif __name__ == \"__main__\":\n client = GlimpsAuditClient()\n client.login(\"admin@example.com\", \"password\")\n \n dataset_name = create_and_populate_dataset(client)\n print(f\"\\nDataset '{dataset_name}' created and populated successfully!\")\n```\n\n### Example: Compliance Reporter\n\n```python\n#!/usr/bin/env python3\n\"\"\"\nGenerate compliance reports for analyzed binaries\n\"\"\"\n\nfrom gaudit import GlimpsAuditClient\nfrom datetime import datetime\nimport json\n\ndef generate_compliance_report(client: GlimpsAuditClient, audit_id: str):\n \"\"\"Generate a compliance report from audit results\"\"\"\n \n details = client.get_audit(audit_id)\n audit = details[\"audit\"]\n \n report = {\n \"report_date\": datetime.now().isoformat(),\n \"audit_id\": audit_id,\n \"file\": {\n \"name\": audit[\"filename\"],\n \"type\": audit[\"filetype\"],\n \"architecture\": audit[\"arch\"],\n \"size\": audit[\"size\"],\n \"hashes\": {h[\"Name\"]: h[\"Value\"] for h in audit[\"hashes\"]}\n },\n \"compliance\": {\n \"total_libraries\": len(audit.get(\"libraries\", [])),\n \"licensed_libraries\": [],\n \"unlicensed_libraries\": [],\n \"copyleft_licenses\": [],\n \"permissive_licenses\": []\n }\n }\n \n # Analyze licenses\n for lib in audit.get(\"libraries\", []):\n for file, versions in lib.get(\"files\", {}).items():\n for version in versions:\n license_type = version.get(\"license\", \"Unknown\")\n lib_entry = {\n \"name\": lib[\"name\"],\n \"file\": file,\n \"version\": version[\"version\"],\n \"license\": license_type\n }\n \n if license_type == \"Unknown\":\n report[\"compliance\"][\"unlicensed_libraries\"].append(lib_entry)\n else:\n report[\"compliance\"][\"licensed_libraries\"].append(lib_entry)\n \n # Categorize by license type\n if license_type in [\"GPL\", \"LGPL\", \"AGPL\"]:\n report[\"compliance\"][\"copyleft_licenses\"].append(lib_entry)\n elif license_type in [\"MIT\", \"BSD\", \"Apache\"]:\n report[\"compliance\"][\"permissive_licenses\"].append(lib_entry)\n \n # Add compliance summary\n report[\"summary\"] = {\n \"compliant\": len(report[\"compliance\"][\"unlicensed_libraries\"]) == 0,\n \"copyleft_risk\": len(report[\"compliance\"][\"copyleft_licenses\"]) > 0,\n \"license_coverage\": (\n len(report[\"compliance\"][\"licensed_libraries\"]) / \n report[\"compliance\"][\"total_libraries\"] * 100\n if report[\"compliance\"][\"total_libraries\"] > 0 else 0\n )\n }\n \n return report\n\n# Usage\nclient = GlimpsAuditClient()\nclient.login(\"compliance@example.com\", \"password\")\n\nreport = generate_compliance_report(client, \"audit-id-123\")\nprint(json.dumps(report, indent=2))\n```\n\n## Testing\n\n### Running Tests\n\n```bash\n# Run all unit tests\npytest\n\n# Run with coverage report\npytest --cov=gaudit --cov-report=html\n\n# Run specific test module\npytest tests/test_client.py\n\n# Run tests matching pattern\npytest -k \"test_login\"\n\n# Run with verbose output\npytest -v\n\n# Using the test runner script\npython run_tests.py --coverage\n```\n\n### Integration Testing\n\nIntegration tests require a live API server:\n\n```bash\n# Set environment variables\nexport GLIMPS_TEST_API_URL=\"https://gaudit.glimps.re\"\nexport GLIMPS_TEST_EMAIL=\"test@example.com\"\nexport GLIMPS_TEST_PASSWORD=\"test-password\"\n\n# Run integration tests\npytest -m integration\n\n# Or using the test runner\npython run_tests.py --integration\n```\n\n### Test Coverage\n\nCurrent test coverage targets:\n- Minimum: 80%\n- Target: 90%+\n- Current: Check with `pytest --cov=gaudit`\n\n## Development\n\n### Project Structure\n\n```\ngaudit/\n\u251c\u2500\u2500 src/gaudit/ # Source code\n\u2502 \u251c\u2500\u2500 __init__.py # Package initialization\n\u2502 \u251c\u2500\u2500 client.py # API client implementation\n\u2502 \u251c\u2500\u2500 cli.py # CLI implementation\n\u2502 \u251c\u2500\u2500 config.py # Configuration management\n\u2502 \u251c\u2500\u2500 elfbuilder.py # ELF building utilities\n\u2502 \u2514\u2500\u2500 ida/ # IDA Pro plugin modules\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 plugin.py # Main plugin code\n\u2502 \u251c\u2500\u2500 idbtoelf.py # IDA to ELF conversion\n\u2502 \u251c\u2500\u2500 ui.py # Plugin UI components\n\u2502 \u2514\u2500\u2500 version.py # Plugin version\n\u251c\u2500\u2500 ida_gaudit.py # IDA Pro plugin entry point\n\u251c\u2500\u2500 tests/ # Test suite\n\u2502 \u251c\u2500\u2500 test_client.py # Client tests\n\u2502 \u251c\u2500\u2500 test_cli.py # CLI tests\n\u2502 \u2514\u2500\u2500 utils.py # Test utilities\n\u251c\u2500\u2500 docs/ # Documentation\n\u2502 \u2514\u2500\u2500 openapi.yml # API specification\n\u251c\u2500\u2500 examples/ # Example scripts\n\u251c\u2500\u2500 requirements.txt # Production dependencies\n\u2514\u2500\u2500 pyproject.toml # Project configuration\n```\n\n### Development Setup\n\n```bash\n# Clone repository\ngit clone https://github.com/GLIMPS/gaudit.git\ncd gaudit\n\n# Create virtual environment\npython -m venv venv\nsource venv/bin/activate # On Windows: venv\\Scripts\\activate\n\n# Install development dependencies\npip install -r requirements-dev.txt\npip install -e .\n\n# Run linting\nruff check src tests\n\n# Format code\nruff format src tests\n\n# Run tests\npytest\n```\n\n### Code Style\n\nThis project follows:\n- PEP 8 style guide\n- Type hints for all public methods\n- Docstrings for all modules, classes, and functions\n- Maximum line length: 119 characters\n\n### Pre-commit Hooks\n\n```bash\n# Install pre-commit\npip install pre-commit\n\n# Install hooks\npre-commit install\n\n# Run manually\npre-commit run --all-files\n```\n\n## Troubleshooting\n\n### Common Issues\n\n#### Authentication Failures\n\n**Problem**: \"API Error 401: Unauthorized\"\n\n**Solutions**:\n1. Verify credentials are correct\n2. Check if token has expired: `gaudit whoami`\n3. Re-authenticate: `gaudit login`\n4. Verify API URL is correct\n\n#### SSL Certificate Errors\n\n**Problem**: SSL certificate verification failed\n\n**Solutions**:\n1. Update certificates: `pip install --upgrade certifi`\n2. For testing only: `gaudit --insecure login`\n3. Set custom CA bundle: `export REQUESTS_CA_BUNDLE=/path/to/ca-bundle.crt`\n\n#### IDA Pro Plugin Not Loading\n\n**Problem**: Plugin doesn't appear in IDA Pro menus\n\n**Solutions**:\n1. Verify `ida_gaudit.py` is in the correct plugins directory\n2. Check IDA Pro console for error messages\n3. Ensure `gaudit` package is installed in IDA's Python environment\n4. Verify Python version compatibility (3.8+)\n\n#### File Upload Errors\n\n**Problem**: \"invalid file type\" or \"invalid file size\"\n\n**Solutions**:\n1. Verify file is PE (Windows) or ELF (Linux) executable\n2. Check file size is under 20MB limit\n3. Ensure file is not corrupted: `file /path/to/binary`\n\n#### Dataset Entry Errors\n\n**Problem**: \"Bad Request\" when adding dataset entries\n\n**Solutions**:\n1. Ensure `project_name` is provided (REQUIRED field)\n2. Ensure `files` array is provided (REQUIRED field)\n3. Verify file IDs exist from prior upload\n4. Check dataset name exists and you have permissions\n\n**Example of correct usage**:\n```python\nclient.add_dataset_entries(\n dataset_name=\"my_dataset\",\n project_name=\"MyProject\", # REQUIRED\n files=[{ # REQUIRED\n \"id\": \"file_id_from_upload\",\n \"binary_name\": \"library.dll\",\n \"version\": \"1.0.0\"\n }]\n)\n```\n\n#### Connection Timeouts\n\n**Problem**: Requests timing out\n\n**Solutions**:\n1. Check network connectivity\n2. Verify firewall settings\n3. Try using a different DNS server\n4. Contact your network administrator\n\n### Debug Mode\n\nEnable detailed logging for troubleshooting:\n\n```python\nimport logging\n\n# Enable debug logging\nlogging.basicConfig(level=logging.DEBUG)\n\n# Now client will show detailed requests/responses\nclient = GlimpsAuditClient()\n```\n\n### Getting Help\n\n1. Check the [FAQ](https://support.glimps.re/faq)\n2. Search [existing issues](https://github.com/GLIMPS/gaudit/issues)\n3. Contact support: support@glimps.re\n4. Join our [community forum](https://community.glimps.re)\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n### Quick Contribution Guide\n\n1. Fork the repository\n2. Create a feature branch:\n ```bash\n git checkout -b feature/amazing-feature\n ```\n3. Make your changes and add tests\n4. Ensure tests pass:\n ```bash\n pytest\n ruff check .\n ```\n5. Commit with descriptive message:\n ```bash\n git commit -m \"Add amazing feature: description of changes\"\n ```\n6. Push to your fork:\n ```bash\n git push origin feature/amazing-feature\n ```\n7. Open a Pull Request\n\n### Development Guidelines\n\n- Write tests for new features\n- Update documentation for API changes\n- Follow existing code style\n- Add type hints for new functions\n- Keep commits atomic and descriptive\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Support\n\n### Resources\n\n- **Documentation**: [https://docs.glimps.re](https://docs.glimps.re)\n- **API Reference**: [https://api.glimps.re/docs](https://api.glimps.re/docs)\n- **Support Portal**: [https://support.glimps.re](https://support.glimps.re)\n- **Email**: support@glimps.re\n- **Company Website**: [https://www.glimps.re](https://www.glimps.re)\n\n### Professional Support\n\nFor enterprise support, custom integrations, or training, contact contact@glimps.re\n\n## Changelog\n\n### Version 0.1.1 (Latest)\n- **IDA Pro Plugin**: Full integration with IDA Pro for reverse engineering workflows\n- **IDC Generation**: Generate IDA Pro documentation scripts from audit results\n- **Enhanced Dataset Management**: Added `dataset upload` and `dataset add-files` commands\n- **Auto-Upload Feature**: New `--auto-upload` flag for automatic file upload when adding to datasets\n- **Improved Error Messages**: Better handling of required fields with helpful error messages\n- **ELF Builder**: Added support for building ELF files from IDA databases\n\nSee [CHANGELOG.md](CHANGELOG.md) for complete version history and release notes.\n\n## Acknowledgments\n\n- GLIMPS development team for the API platform\n- Open source community for invaluable tools and libraries\n- All contributors and users providing feedback and improvements\n\n---\n\n**Copyright (c) 2025 GLIMPS** - Prevent tomorrow's threats today.\n",
"bugtrack_url": null,
"license": null,
"summary": "Library and CLI for GLIMPS Audit API",
"version": "0.3.1",
"project_urls": null,
"split_keywords": [
"python",
" glimps",
" reverse engineering",
" malware",
" binary",
" analysis",
" ida"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "d325b511b53ac18d93d1a1eadcac75f2fccb38d75a30f83ee71e9e43307d7956",
"md5": "bd1c902ee710630d57550f91c832b060",
"sha256": "7470e3954be320373c2bf6469e40a427c06379879250fe86e3530c16c3f78bba"
},
"downloads": -1,
"filename": "gaudit-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bd1c902ee710630d57550f91c832b060",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 40470,
"upload_time": "2025-08-11T16:11:45",
"upload_time_iso_8601": "2025-08-11T16:11:45.977325Z",
"url": "https://files.pythonhosted.org/packages/d3/25/b511b53ac18d93d1a1eadcac75f2fccb38d75a30f83ee71e9e43307d7956/gaudit-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3ac6d41e688fea26a12b8705e41ea20afc5b60eab58df188fb903354c1d204d3",
"md5": "a23ae3a9084624b671bbda9dce269e22",
"sha256": "e56a8d9c7066c0b921a2f38770d42fea1bd17b3ee17f26e1c4b2ae99d704e585"
},
"downloads": -1,
"filename": "gaudit-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "a23ae3a9084624b671bbda9dce269e22",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 91432,
"upload_time": "2025-08-11T16:11:47",
"upload_time_iso_8601": "2025-08-11T16:11:47.326300Z",
"url": "https://files.pythonhosted.org/packages/3a/c6/d41e688fea26a12b8705e41ea20afc5b60eab58df188fb903354c1d204d3/gaudit-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-11 16:11:47",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "gaudit"
}
JSON