Capability Matrix · Decision framework
Fourteen questions that turn a workload into an archetype, a shortlist, and a verdict.
Architecture decisions in security data fail in predictable places, and the ones that go wrong are usually the early ones, the foundational choices that cascade through every later choice. The framework below sequences those questions in four phases: sizing constraints that classify the workload into one of the scored archetypes, foundational commitments that hold for years, organizational reality, then the use-case mix that sets the weights. What comes out the other end is not a vendor beauty contest but the archetype's scored shortlist, the two gate verdicts, and the augment-vs-replace crossover, each of which you can audit on its own page.
Phase 1 — Sizing constraints (S1–S4)
Sizing questions that classify the workload before anything is scored.
Technical capability is irrelevant if a platform can't carry your volume, and the scored archetypes are defined in sizing terms, so these four questions decide which weight set applies before any cell is read.
S1 — Data volume per day.
Daily ingest is the first classifier, because the published archetypes carry volume bands: Archetype A, the Zeek-heavy SOC, is defined at 5–50 TB/day with 90 days hot and a year warm (MDR-0007); Archetype B, the multi-source federated lakehouse, at 10–100 TB/day (MDR-0017); and Archetype C assumes serverless-within-AWS, where capacity is bought per query rather than per node (MDR-0021). Volume also runs the one hard candidate filter that survives from this phase: DuckDB is single-node-only, so above roughly one host of ingest it cannot be the primary engine and is scored as an analyst overlay on top of a lakehouse instead. And volume is what makes the incumbent's economics bite, because per-GB pricing scales the bill with ingest: the one publicly citable Splunk rate card, the UK G-Cloud 14 EMEA distributor pricelist (2024-04-23), puts self-hosted Enterprise term licensing at $575–598 per GB/day per year in the terabyte-scale bands, which pencils to roughly $2.9M a year at 5 TB/day at list, before the Enterprise Security add-on and before whatever discount a buyer negotiates (published framework pricing — list rates, not what any given buyer pays).
S2 — Annual growth rate.
Growth decides whether pinned capacity is safe. The common drivers are real — cloud migration, M&A, compliance expansion, EDR or XDR rollouts that multiply per-endpoint telemetry — but I haven't found a defensible published basis for per-driver growth percentages, so the framework asks for the client's own trajectory rather than supplying one. High-growth profiles favor elastic and serverless shapes (Athena, the managed SQL warehouses) over pinned-capacity self-managed clusters, because guessing wrong costs you either capacity overruns or steady-state over-provisioning, and which of those you would rather own is a judgment the framework forces into the open rather than a default.
S3 — Source count.
Source count decides how much of the architecture the ingestion layer (Component 4) deserves. A handful of sources can be direct-ingested; once the count runs into the dozens, a normalization layer (Tenzir, Vector, Cribl) starts earning its place, because schema management by hand stops scaling before the pipeline does; at MSSP scale the pipeline becomes a product decision of its own. Those thresholds are practice judgment rather than measured cliffs, so the framework treats them as prompts for the Component-4 scoring rather than cut lines.
A caveat on OCSF normalization at-source, because it gets sold as a checkbox: it's scored on two levels in order, because the first binds before the second matters. Availability comes first — does the tool ship a usable mapping for the source at all, in the shape the client's actual shipper emits — and first-party pipeline-fidelity work (Tier B, single host, OCSF 1.8.0, version-bound; sdw-lab-benchmarks/pipeline-normalization-fidelity/FINDINGS-2026-06-14.md) found shipped-mapping availability, not per-field fidelity, to be the binding constraint across the open tools tested, with several common security sources having no usable shipped mapping at all, or one bound to a raw input shape (EVTX/XML) that pre-parsed EDR JSON doesn't match. Fidelity comes second, and the gap there is that a shipped mapping can get the OCSF class right and most field values right while still getting the activity classification wrong — "maps to OCSF" is a coverage claim, not a fidelity guarantee, and the part that fails is often the part detection content keys on.
Source count gates the ingestion-layer component scoring.
S4 — Retention horizon.
Hot retention (queryable, fast) versus cold retention (archival, slow) is a direct cost-versus-latency trade. The hot window is sized by detection lookback and hunting recency — days to a few weeks in the SOCs I've seen — while compliance sets the long tail, and the mandates that actually specify durations are narrower than vendor decks imply: PCI-DSS requires a year of audit-trail history with three months immediately available (Req 10.5.1), HIPAA's documentation-retention rule runs six years (45 CFR 164.316(b)(2)(i)), and SEC 17a-4 puts broker-dealer records at three to six years on storage that can enforce immutability, which is where the compliance gate's WORM lineage comes from (MDR-0030). GDPR is deliberately not on that list: it imposes storage limitation, keeping personal data no longer than necessary, so it pushes retention down rather than up. Two or more years of retention forces hot/warm/cold tiering as an architectural commitment, and what each tier costs should be re-based on the client's own log shapes rather than a borrowed headline ratio, because the measured compression band is wide: first-party storage measurement (Tier B, single host, DuckDB 1.5.3; sdw-lab-benchmarks/cost-to-serve-retention/) puts zstd-default reduction against raw at roughly 8.5× for flat Zeek conn, 7.9× for an EDR/Sysmon process-creation corpus, and only 2.6× for high-entropy payloads.
Phase 2 — Foundational architecture (F0–F4)
Commitments that hold for years and cascade through every later choice.
F0 — Isolation pattern.
The single most consequential architectural decision in the framework, and the one the scored component pages assume has already been made. Three patterns. Isolated-dedicated: security data lives in its own VPC or environment with no co-tenancy with corporate workloads, which means row-level security isn't needed at all and the $0-license OSS catalogs (Polaris, Nessie) stay viable; operators who have publicly described running security data isolated at scale include Huntress and Okta, and Netflix's engineering writing validates the isolation-at-scale pattern generally though not as a security-specific deployment (Tier B — named practitioner accounts, not audited deployments). Shared-corporate: security and PII colocated, so row-level security becomes mandatory and the catalog choice narrows to the ones that enforce it, which is why the formats-and-catalogs scoring weights RBAC hardest for that shape. Multi-tenant MSSP: every query is tenant-scoped, so RLS is the operating model rather than an overlay; Arctic Wolf, Expel, and Red Canary have publicly described running this pattern (Tier B). What RLS actually costs is measured rather than assumed: the lab's engine-side RLS bench (Tier B, DuckDB 1.5.3, 1M-row OCSF corpus;sdw-lab-benchmarks/ocsf-rls-overhead/) found the predicate cost is query-shape-dependent — count-shaped full scans paid +54% to +96% while scan-heavy aggregations came back faster than the unfiltered baseline because the tenant filter reads fewer rows — and that engine-side number is a lower bound, because catalog-layer enforcement (Polaris, Unity, Lake Formation) adds policy-evaluation and protocol round-trips the bench does not measure. The choice constrains every downstream component.
F1 — Table format.
Iceberg, Delta Lake, or Hudi, with the native SIEM formats (Splunk index, OpenSearch) typically filtered out on open-format grounds before scoring starts. The live matrix does not score formats alone, because a format is only deployable through a catalog and scoring them independently produces operationally undeployable pairs, so Components 1+2 are scored as format-plus-catalog pairs and this question decides which pairs are candidates at all. Iceberg keeps multi-engine optionality open and is the default MOAR pattern; Delta is the honest answer when the platform decision has already landed on Databricks; Hudi's niche is streaming-upsert-heavy pipelines.
F2 — Catalog.
Polaris, Nessie, Unity Catalog, AWS Glue, or Hive Metastore, scored as the catalog half of the pair. The catalog determines governance (row-level security, column masking), RBAC scope, and multi-engine access, and the scored pairs land differently per archetype: Iceberg+Polaris leads Archetype A at 4.25 and Archetype B at 4.10 with Delta+Unity close behind at 3.95 where the RBAC weight is heaviest, while Iceberg+Glue wins Archetype C at 4.20 because AWS-native depth is the design center there. Isolated Iceberg deployments work cleanly with Polaris (table-level RBAC, $0 license) or Nessie (Git-style versioning, $0 license); shared platforms effectively require Unity Catalog for fine-grained governance today. The full scored tables with per-cell evidence tiers are on theformats + catalogs page.
F3 — Transformation pattern.
dbt is SQL-based and analyst-friendly. Spark requires PySpark or Scala expertise, which is a data engineering skill set, not an analyst one. Vendor built-in (SPL for Splunk, KQL for Sentinel) creates query-language lock-in that survives migration discussions. The choice cascades into team hiring profile and the long tail of detection-engineering work.
F4 — Query engine selection.
Most architectures need more than one engine characteristic, and the lab evidence is now specific about what separates the candidates. At single-host SOC scale the raw-speed spread is genuinely compressed: the engine-join bench (2026-06-10, Tier B) had every open engine answering the SOC join suite in under 1.5 s, with StarRocks ahead on the multi-table joins and ClickHouse on the aggregation shapes. Concurrency separates them harder than raw speed does: the multi-user concurrency bench (2026-06-15, Tier B, single host) found the columnar Iceberg engines have almost no concurrency headroom (1.1–1.3× from one client to thirty-two, everything converging on the same ~6–7 QPS host ceiling), StarRocks the best-behaved under load with the highest QPS ceiling (6.79) and the tightest tail (p95/p50 ≈ 1.16 at N=16, against 1.30 for ClickHouse and 1.90 for Trino), and Trino carrying a low ceiling (~3 QPS) with large tail blow-ups. Every arm in that bench was the OSS/community edition, and several vendors sell concurrency isolation as a paid-edition feature, so the finding is scoped to OSS editions rather than claimed for the enterprise tiers. Serverless for compliance batch reporting points to Athena, which wins Archetype C at 4.29 on the scored engines page. Cost-optimized analyst work points to DuckDB as an overlay, never a primary. The framework handles multi-engine architectures explicitly, and most of the deployments I've seen end up running two engines.
A related caveat: a materialized view or MV-like acceleration feature makes a given query faster, but it buys no concurrency-knee headroom on its own — the interference knee sits at the same point with or without materialization (the StarRocks-MV upper bound hit the same ~64× interference knee as base StarRocks, 2026-06-14 workload-interference bench). Don't reach for materialized views to solve a concurrency problem; they solve a latency problem, and those are different problems here.
Phase 3 — Organizational constraints (Q1–Q4)
Team and budget realities that decide what is actually operable.
Q1 — Team size and capability.
Lean teams (1–2 engineers) need managed services. Standard teams (3–5) can carry a partly-managed, partly-self-hosted stack. Large teams (6+) can carry composable architectures with multiple OSS components. Those bands are practice judgment, not a measured threshold, but the match between operational complexity and team capacity is what decides whether a deployment survives its second year, and the scored matrices carry operational complexity as a weighted criterion so the judgment lands in a cell rather than a hallway conversation.
Q2 — Budget tier.
Budget eliminates whole categories before scoring starts, though the honest version of this question anchors on published list prices rather than invented tiers. The Splunk floor from S1 is the concrete case: roughly $2.9M a year at 5 TB/day at self-hosted G-Cloud 14 list before the ES add-on, with real buyers negotiating down from there, so a budget that cannot clear an enterprise SIEM's per-GB bill at the client's volume shifts the candidate set toward OSS plus managed services rather than merely re-ranking it. The matrix's criteria taxonomy carries a cost-to-serve criterion on every component (MATRIX.md), so the budget question maps to scored cells and the measured storage band from S4 rather than to a categorical guess.
Q3 — Cloud environment.
Single-cloud, multi-cloud (AWS + Azure or AWS + on-premises is common), or fully on-premises. On-premises removes the AWS-bound candidates outright — Athena, Glue, and AWS Security Lake are AWS-bound in the candidate catalog — and multi-cloud requires cloud-agnostic components (Trino, Iceberg, Polaris). Vendors marketed as "multi-cloud" sometimes mean "available in three regions" rather than "architecturally portable across clouds," and the difference shows up the first time you try to move the governance layer.
Q4 — Vendor tolerance.
OSS-first maximizes flexibility but requires engineering capacity. Commercial-only provides SLAs and 24/7 support but reduces optionality. Most environments end up mixed; the question forces explicit framing rather than a default-to-vendor pattern.
Phase 4 — Use cases (Q5)
Workload requirements set the weights that drive the final scoring.
The use cases (real-time SOC dashboards, ad-hoc threat hunting, compliance retention, streaming detection) are not filtered; they set the weights. This is MDR-0004, the most consequential methodology decision in the matrix: every component is scored separately per archetype, criterion definitions stay constant while weights vary, and an engagement is assigned to its closest archetype rather than negotiating a custom weight set — with a poor fit treated as a finding that either adds an archetype or documents an explicit deviation. Weights sum to 100 under an anchoring rule (no weight above 30, none below 5, multiples of 5, per MDR-0007), which exists so a weight set cannot be quietly tuned to predetermine a winner.
Two gate axes run before any weighted total is read. Detection-survivability (MDR-0029, the Component-5 criterion) scores what fraction of the client's detection corpus survives the move in three bands — survives, silently degrades, cannot port — and the silent-degradation band is scored worse than a clean failure because it runs wrong while the dashboard stays green, a posture anchored to the measured pySigma windowless-compilation evidence. The compliance/WORM axis (MDR-0030) is disqualify-not-discount: where the client's regime mandates a control, a candidate that cannot satisfy it is removed before totals are computed rather than low-scored. A path that fails either gate is ineligible to win on cost alone.
What comes out the other end is the archetype's scored shortlist: thirteen scoring files across engines, format-plus-catalog pairs, pipelines, detection-survivability, and the WORM axis, each cell carrying a 1–5 score, a weight, an A–D evidence tier, and citations, with null preserved as bounds when the evidence doesn't support a defensible score. On top of the shortlist sits the augment-vs-replace read: the crossover is computed ascrossover_months = migration_cost / (monthly_cost(Stay) − monthly_cost(path))rather than asserted, and every recommendation ships a four-part board-defensibility read. The scored tables are public; the weights tuned to your workload, the per-client crossover, and the gate verdicts against your actual detection corpus are theengagement.
Rebuilt 2026-07-19. The earlier version of this page described a ninety-vendor elimination funnel inherited from a retired tooling era; its elimination counts, compatibility counts, and price tiers had no maintained source, so the four-phase, fourteen-question shape was kept and every number on the page now cites its evidence or names itself as judgment.
The framework is the audit trail that makes the recommendation defensible.
The component criteria, the scored tables, and the decision path each carry the next layer of the same audit trail.