Sigma and detection portability: the fourth foundational standard

Sigma is an open, engine-agnostic detection rule format. Detections are written in YAML (a structured description of log sources, field matches, conditions, and metadata) and then converted by a backend compiler into the query language of a target detection engine. Florian Roth and Thomas Patzke originated the format in 2017 out of the German SOC community, and it has been actively developed since. The Sigma project at SigmaHQ on GitHub maintains both the specification and a public rule repository with thousands of community-contributed detections.

The thing that matters architecturally is the intermediate representation. A Sigma rule is not directly executable. It compiles, via the pySigma toolchain, into Splunk SPL, Microsoft Sentinel KQL, Elastic ES|QL or EQL, Google SecOps YARA-L, lakehouse SQL, and around thirty other backends (about 31 pySigma plugins at the last count I checked) depending on which pySigma plugins you install. The single source of truth (the YAML rule) converts to whatever dialect the target engine speaks.

That is the structural argument. Detection content written once should not need to be re-authored every time a SOC swaps SIEMs or adds a second analytical engine. The Sigma specification itself is the only Tier A piece of evidence I cite without qualifier. The spec exists, is published, and is the authoritative description of the format. Everything else in this essay is Tier B or below.

The class of detection Sigma handles cleanly is the atomic detection, a single-event match against a known indicator pattern. Suspicious command-line invocations, known-bad process trees, specific registry modifications, anomalous Windows event IDs, exfiltration to specific TLDs, lateral-movement fingerprints that appear in one event record. If the detection answer is "this single event matches the following field conditions," Sigma is well-shaped for it.

The SigmaHQ public rule repository is the largest practical demonstration of this. It contains thousands of community-maintained rules organized by log source (Windows Sysmon, Linux auditd, AWS CloudTrail, Azure activity logs, Office 365, network IDS output) and tagged against MITRE ATT&CK techniques. For an organization that wants a baseline of community detections quickly, downloading the SigmaHQ repo and compiling it against the local engine is a sensible starting point, though it's a starting point rather than a finished detection program, giving you thousands of pre-written examples of the atomic-detection pattern to curate down from.

The MITRE ATT&CK coverage angle deserves its own note, because Sigma rules carry ATT&CK technique tags as metadata, which means a SOC can compile coverage maps mechanically and see which techniques have detections, which don't, and which have multiple detections from independent sources. That mechanical-mapping property is an underappreciated feature, since it supports detection-maturity work that previously required manual spreadsheet maintenance.

A point on how cross-SIEM coverage actually works in practice, because the mental model many people reach for is wrong, since the thing that delivers broad coverage today is aggregation rather than auto-translation. Michael Haag's Security Detections MCP aggregates 8,200-plus community detections and indexes them by MITRE ATT&CK technique, and those detections are written in each platform's native language and curated by the communities that authored them, rather than one canonical rule machine-translated across engines. That distinction matters for what you should expect from a Sigma pilot, because Sigma's compiler gives you per-backend conversion of the rules you author and not a magically translated universal corpus. Where coverage is broad, it's broad because a community wrote and curated the per-engine variants, and that curation is the work the tooling doesn't do for you.

For the SOC capability ladder I describe elsewhere on this site (see detection maturity for the long version), Sigma is the enabler that moves a team from ad-hoc rule authoring (level 1) to repeatable, portable, version-controlled detection content (level 2 and into level 3), though it doesn't by itself get you to the higher levels, and the correlation-maturity gap I describe next is why.

The four-pillar framing (Arrow, Iceberg, OCSF, Sigma) is a layered argument about which open standards remove which kinds of vendor lock-in: Arrow and Iceberg at the data layer, OCSF at the schema layer, and Sigma at the analysis layer, where it lets the detection content sitting on top of portable data move too. The correlation essay walks the full frame, so here I only want to make two points that matter for a reader new to Sigma.

One clarification, because the two standards get conflated in conversation. OCSF does not compete with Sigma as a detection language, and Sigma's authoring role is not under threat from it. OCSF's Detection Finding class (UID 2004) is an output schema. It describes the shape of a finding a detection emits, not the grammar you author the detection in. The pairing is complementary: Sigma is where you write the rule, OCSF is the normalized field set the rule reads from and the normalized finding it writes to. The Sigma project leans into exactly that pairing. SigmaHQ maintains pySigma-pipeline-ocsf, an official MIT-licensed processing pipeline that maps Sigma's field references onto OCSF-normalized fields so a single rule compiles against an OCSF-shaped lakehouse, which is the stack working as intended rather than the two standards contending.

There's a structural symmetry worth naming, because it runs through every pillar in this stack. Sigma's portability is read-strong and write-weak. The atomic rule grammar ports cleanly to around thirty backends; the things on the write side (value-typing, taxonomy and field mapping, the correlation semantics described above) don't port and stay anchored to whichever engine actually executes them. That's the same asymmetry I described in the lakehouse catalog work, where an open schema reads everywhere but the security-grade write controls (ABAC, provenance, audit-retention) don't travel with the data and stay bound to a specific catalog and engine, so read portability tends to be the easy half of an open standard in security while write portability is where the lock-in retreats to and survives. Sigma sits on the same fault line as Iceberg and OCSF, which is a reason to expect the correlation gap to close slowly rather than a reason to doubt the pillar.

I write about Sigma as a foundational pillar, which means I have a positive thesis on it. That doesn't excuse me from naming what I don't like, and there are four critiques I'd want any architect to internalize before adopting.

• Community rule quality varies wildly. The SigmaHQ repo has thousands of rules of widely different quality. Some are excellent. Some are stale. Some encode assumptions about a specific Sysmon configuration that doesn't match your environment. Loading the entire repo without curation is a false-positive disaster waiting to happen. The starting-baseline pattern requires curation as a discipline, not as a one-time setup task.
• Backend conversions need per-environment validation. A Sigma rule that compiles cleanly to SPL may produce a query that performs poorly on your specific Splunk indexer configuration, or matches differently because your field extractions differ from the rule author's. The conversion is not a contract; it's a starting point that needs validation per backend, per environment. Treat regression testing against historical telemetry as mandatory, not optional.
• Vendor incentives push against true portability. Splunk and Microsoft do not benefit commercially from detection content being trivially portable away from their platforms. Lock-in is a revenue model. Even when vendors offer Sigma import support, the path of least resistance (pricing, tooling, native authoring surfaces, partner integrations) favors staying on the vendor-native authoring path. I would not expect any SIEM vendor to make the Sigma path easier than their proprietary path. That's a structural reality, not a complaint about specific vendors.
• Sigma is community-led, which is both a feature and a risk. The format and the rule repo depend on volunteer maintainership. The project has been active and well-maintained for years, but it is not backed by a single foundation with a large engineering team in the way Arrow and Iceberg are. That's not a reason to avoid Sigma so much as a reason to track project health as part of your foundational-standards risk register.

None of these are reasons to dismiss Sigma, but they are reasons to size expectations and to plan the adoption work around them rather than discovering them in production.

Summed up, Sigma's atomic-detection portability is real, useful, and the foundation of a detection program that can survive a SIEM swap without rewriting hundreds of rules. The correlation extension is specified and compiles at full fidelity on some backends, but not on all of them, so it needs checking per engine. The rule repository is a starting point rather than a finished detection program, and vendor adoption claims should be tracked but not trusted at face value. For an architect making a decision this quarter, I'd pilot pySigma on a slice of your own backlog, treat Sigma as portable IR for the atomic-detection layer, and pair the adoption with the DetectFlow discipline, because even if correlation portability matures slower than I expect, you'll still have removed a real lock-in vector from your detection program.