Rule Optimization

SIEM Rule Performance Tracking, Auto-Tuning, FP-Rate Analysis, and Sigma-to-Native Transpilation

Overview

The Rule Optimization module addresses the most common operational pain in SIEM management: noisy detection rules that generate excessive false positives. It provides continuous performance tracking for every detection rule, generates prioritized optimization recommendations, can auto-disable the worst offenders, and includes a full Sigma rule transpiler for cross-SIEM rule deployment.

SOC analysts spend a significant portion of their time on false positive triage. Rules with high FP rates cause alert fatigue, slow response times, and increase the risk of missing true positives. This module quantifies rule performance, surfaces actionable recommendations, and automates the tuning lifecycle.

What Was Proposed

• Per-rule, per-tenant FP-rate tracking with multi-window analysis
• Automated detection and flagging of noisy rules
• Auto-tuning engine with dry-run mode and compliance audit trail
• Optimization recommendations with priority ranking
• Sigma rule transpilation to native SIEM query languages

What's Built Complete

RulePerformanceTracker

File: platform/api/app/services/rule_optimizer.py (~1020 lines)

• In-memory event store: Records rule trigger events with rule_id, tenant_id, alert_id, timestamp, and was_fp flag
• Multi-window analysis: Default windows at 7, 30, and 90 days for FP-rate calculation
• Performance snapshots: Aggregated metrics per rule including total_triggers, fp_count, tp_count, fp_rate, first_seen, last_seen
• Noisy rule detection: Configurable threshold (default 40% FP rate over 30 days, minimum 5 triggers)
• Recommendation engine: Generates prioritized actions based on FP severity:
  • ≥ 60% FP rate: Recommend disable (priority 1)
  • ≥ 50% FP rate: Recommend tune threshold (priority 2)
  • ≥ 40% FP rate: Recommend add exclusion (priority 3)
• Capacity management: Max 500,000 events in memory with oldest-10% eviction
• Periodic flush: Configurable flush interval (default 300s) for database persistence

RuleAutoTuner

• Auto-disable threshold: 60% FP rate with minimum 10 triggers over 30 days
• Dry-run mode: Preview actions without disabling any rules
• Audit log: Keeps last 100 tuning run records (run_id, timestamp, mode, rules reviewed/disabled/flagged)
• Notification: Sends real-time notifications when rules are auto-disabled via NotificationService
• Tenant scoping: Can run across all tenants or scoped to a specific tenant

SigmaTranspiler

• 3 Target SIEMs: KQL (Microsoft Sentinel), SPL (Splunk), EQL (Elastic)
• Field mapping: 16+ Sigma fields mapped to native equivalents per SIEM (CommandLine, Image, User, DestinationIp, etc.)
• Log source mapping: Process creation, file events, network connections, registry events, DNS, firewall, O365/M365 mapped to native tables/indices
• Modifier support: contains, endswith, startswith, re (regex)
• Condition parsing: Handles selection, not filter, selection and not filter, boolean combinations

Frontend UI

File: platform/frontend/src/app/rule-optimization/page.tsx (~520 lines)

• Summary cards: total rules tracked, noisy rules count, auto-disabled count, pending recommendations
• Noisy rules table with 7d/30d/90d FP rates (color-coded), trigger counts, and recommendation badges (Disable/Tune/Exclude)
• Auto-tune controls with Run button, dry-run toggle, and result display showing reviewed/disabled/tuned counts with detail lines
• Sigma transpiler with YAML input textarea, target SIEM checkboxes (KQL/SPL/EQL), transpile button, and tabbed output with syntax-highlighted native queries on dark background
• Auto-tune audit log table with timestamp, mode badge, rules reviewed, and rules disabled

Architecture

Frontend (Next.js)                       API (FastAPI)
+------------------------------+   +-----------------------------------+
| /rule-optimization           |   | /api/v1/rules/*                   |
| - Summary Cards              |-->|                                   |
| - Noisy Rules Table          |   | RulePerformanceTracker            |
| - Auto-Tune Controls         |   |   record_trigger()                |
| - Sigma Transpiler (KQL/     |   |   get_performance() / get_noisy() |
|   SPL/EQL)                   |   |   generate_recommendations()      |
| - Audit Log Table            |   |                                   |
+------------------------------+   | RuleAutoTuner                     |
                                   |   run(dry_run) / is_disabled()    |
      Alert Pipeline               |   audit_log / notifications       |
      +------------------+        |                                   |
      | Triage Service   |------->| SigmaTranspiler                   |
      | records triggers |        |   transpile(yaml, [kql,spl,eql])  |
      +------------------+        +-----------------------------------+
    

Routing

Type	Path	Description
Frontend	/rule-optimization	Rule optimization dashboard with FP analysis, auto-tune, and transpiler
API Prefix	/api/v1/rules	All rule optimization endpoints

API Endpoints

Method	Endpoint	Description
GET	/performance	FP rates and trigger counts for all tracked rules
GET	/performance/{rule_id}	Performance metrics for a specific rule (multi-window)
GET	/recommendations	Generate optimization recommendations for noisy rules
POST	/auto-tune	Trigger auto-tuning pass (dry_run param, admin)
POST	/transpile	Convert Sigma YAML to native queries (KQL/SPL/EQL)
GET	/noisy	List rules exceeding FP rate threshold
POST	/performance/record	Record a rule trigger event for FP tracking
GET	/auto-tune/audit-log	Auto-tuner audit log showing past tuning runs

Prerequisites

• FastAPI backend with the rule optimization router registered
• pyyaml package for Sigma YAML parsing
• Tenant context (via middleware) for per-tenant scoping
• Alert pipeline integration: the triage service calls /performance/record after each alert disposition to feed FP-rate data

Data Model

Enums

Enum	Values
OptimizationAction	disable, tune_threshold, add_exclusion, monitor

Key Data Structures

Structure	Fields
RuleTriggerEvent	rule_id, tenant_id, alert_id, timestamp, was_fp
RulePerformanceSnapshot	rule_id, tenant_id, window_days, total_triggers, fp_count, tp_count, fp_rate, first_seen, last_seen
OptimizationRecommendation	rule_id, tenant_id, action, reason, fp_rate, window_days, total_triggers, priority, created_at
AutoTuneResult	run_id, started_at, completed_at, dry_run, rules_reviewed, rules_disabled, rules_flagged, details[]

Sigma Transpiler Field Mappings

16+ Sigma fields mapped to native equivalents. Example mappings:

Sigma Field	KQL (Sentinel)	SPL (Splunk)	EQL (Elastic)
CommandLine	ProcessCommandLine	process	process.command_line
Image	InitiatingProcessFileName	process_name	process.executable
User	AccountName	user	user.name
DestinationIp	RemoteIP	dest_ip	destination.ip

UI Description

The Rule Optimization page is organized into five sections using the ThreatOps white-and-orange design system:

1. Summary cards (4-column grid): Total Rules Tracked, Noisy Rules (with red highlight for >40% FP), Auto-Disabled count, and Recommendations Pending
2. Noisy rules table: Sortable table with Rule ID, Rule Name, FP Rate columns for 7d/30d/90d (color-coded: red ≥ 70%, orange ≥ 40%, green below), trigger counts, and recommendation badge (Disable=red, Tune=yellow, Exclude=blue)
3. Auto-tune controls: Orange "Run Auto-Tune" button with spinner, dry-run toggle (ON by default), and result panel showing mode badge, rules reviewed/disabled/tuned counts, and detail lines with check icons
4. Sigma transpiler: Two-column layout with YAML input textarea on the left (pre-populated with a Mimikatz detection example), target SIEM checkboxes (KQL/SPL/EQL), transpile button, and tabbed native query output on the right in a dark terminal-style panel with green text
5. Audit log table: Timestamped log of past auto-tune runs with mode badge (dry-run=blue, live=green), rules reviewed, and rules disabled counts