Writing · Pillar
Detection & hunting.
Detection-engineering maturity ladder, MLOps for hunters, latency tiers, feature stores, PEAK methodology on a modern data stack.
Depth tier for ch3 (Trustworthy) and ch6 (what good looks like) · 17 essays
Listed from the front door down to the measured evidence. Start here marks the entry essay. The reading-level tag says whether you are about to read the frame (Orientation), the argument (Depth), or the narrowest first-party finding (Evidence). The durability tag flags whether a number is durable architecture or an Evidence-pinned result tied to a version or price and meant to be re-run. The last tag is the provenance state, meaning who held the pen and who checked the work, which is explained in full on how these are made.
Data quality & verification
A rule can compile clean and silently never fire because a value was wrong three layers down, and the same silent-failure mode shows up one layer below the detection logic, in the engines themselves. Reading one byte-identical Parquet file through twelve query engines, eleven matched the generator's ground truth and one returned a confidently wrong row count with no error raised, which is the case for treating answer-equality across engines as something you verify with a standing check, not something an open format hands you for free.
- Start here OrientationDurableAI-drafted, observed
The ground you're already standing on.
Detection engineers are doing data modeling every day without naming it, and a rule can compile clean and quietly never fire because one of those mappings was wrong with nothing set to catch it. The vocabulary, the silent-failure mode, and a deductive check that caught 100% of injected type-crossings on a 925-row corpus.
Read →
- DepthDurableAI-drafted, observed
What your data means vs what shape it is.
Schema conformance checks the shape of a field and says nothing about its meaning, so a mapping can fit OCSF exactly and still mean the wrong thing. Eight flagged meaning-crossings from a six-schema crosswalk corpus (plausible coarse mappings worth a human review, not confirmed-wrong rows), made visible by a deductive check the syntax pass waves through.
Read →
- DepthDurableAI-drafted, observed
Catching the mistake that kills a detection.
The wrong field mapping that silently kills a detection is catchable with a reasoner and a few disjointness assertions D3FEND never shipped. How to run that check yourself against your own mappings, and the limits of what it catches.
Read →
- EvidenceEvidence-pinnedHuman revised
The query engine returned the wrong answer and didn't tell you.
Over an open format the engine is supposed to be interchangeable. In my lab one returned a filtered count tens of rows short of the others on the identical Parquet, no error raised, and a timing-only benchmark would have published it as a win. Why a cross-engine answer-equality gate is the only thing that caught it.
Read →
- EvidenceEvidence-pinnedHuman revised
The better the model, the quieter the wrong answer.
I answered one OCSF question battery six ways across three model tiers. On compute-over-population questions no LLM-authored arm was reliably correct, and the more capable model made the wrong answer quieter rather than rarer; an execution loop and self-consistency both failed the same way. Safety lived in the layer that refuses a query it can't answer.
Read →
- EvidenceEvidence-pinnedHuman revised
Parquet doesn't hash the way your chain-of-custody thinks it does.
Chain-of-custody, WORM, and dedup workflows key on a file hash, but Parquet isn't byte-reproducible by default: parallel row-group order makes the same logical data produce a different SHA-256 every write. A faithful re-export breaks the hash and looks like tampering. Force determinism (threads=1 or ORDER BY, ~20% smaller too), and hash the logical content so a faithful re-export still matches.
Read →
Detection & hunting practice
- DepthDurableAI-drafted, observed
Detecting the OT you can't parse.
Most industrial protocols will never get a deep parser, so the question that decides whether you can monitor an OT network is how much you can detect from behavior (timing, flow, who talks to whom) before a parser ever runs. Why behavioral-first gives immediate east-west coverage across the long tail, why it fits the NERC CIP-015-1 internal-monitoring mandate, where it stops, and the falsifier that would change my mind. It's a Tier B practitioner argument, without a benchmark behind it.
Read →
- DepthDurableAI-drafted, observed
Who actually does the hunting.
The analyst reverse-engineering a broken field mapping at 2am is doing applied ontology, and naming the work hands them open tooling and a community that's been building the same map from the other side.
Read →
- DepthEvidence-pinnedAI-drafted, observed
The open detection-grounding stack, piece by piece.
A practitioner's map of the open detection-grounding stack: OCSF, D3FEND, Sigma, the ROBOT/ELK reasoner toolchain, and the six-schema crosswalks, with what each piece is for and exactly where it's thin.
Read →
- DepthDurableAI-drafted, observed
The hunting maturity ladder: why HMM2 to HMM3 is the hard step.
David Bianco's Hunting Maturity Model read as an analyst-trust force multiplier: why most hunting programs sit at HMM2, why the step to HMM3 is the hardest and can't be bought, and why detection content authored at HMM3 survives a platform migration.
Read →
- DepthEvidence-pinnedAI-drafted, observed
PEAK and the lakehouse.
How modern data stacks enable threat hunting. The Splunk PEAK framework ported to lakehouse infrastructure, what works, and what doesn't.
Read →
- DepthEvidence-pinnedAI-drafted, observed
Three latency tiers: detection, hunting, analysis.
Detection in seconds, hunting in minutes, analysis in hours, separated by who is waiting on the result. The three-tier frame behind DetectFlow, why the 2026 breakout-time, time-to-exploit and Mythos data made detection latency a hard constraint, and the architecture that serves all three.
Read →
- DepthEvidence-pinnedAI-drafted, observed
MLOps tools for threat hunters.
Reproducibility for threat-hunt notebooks. What MLOps tooling brings to hunters that they don't already have, and where the friction lives.
Read →
- DepthEvidence-pinnedAI-drafted, observed
Where detection-as-code notebooks should live.
The notebook is how security analytics gets shared, and the live decision is the proprietary-but-production Databricks notebook versus the open, reproducible marimo .py. Why the lock-in moved up to the authoring layer, what marimo provably fixes, and why the detection you ship is text either way.
Read →
- DepthDurableAI-drafted, observed
Every security tool ships with a silent configuration gap.
Manageability as a force multiplier: the distance between how a control is configured today and how it should be run, made visible and rankable across the whole stack. Threat-informed reallocation and CIS-benchmark conformance scanning optimize different denominators, so the ranking weight I'd use is threat exposure against an independent baseline rather than raw configuration distance.
Read →
- EvidenceEvidence-pinnedHuman vetted
The assurance gap no single tool closes.
No single security tool knows your environment: on a planted 140,000-cell benchmark the best single tool recovers 47.7% of the truth and a freshness-scored cross-tool merge recovers 75.6%, while a 24.4% residual stays dark no matter how many tools you own. The transferable claim is the ordering, which holds across the whole parameter sweep, plus the entity-resolution tax a contested join key adds.
Read →
- EvidenceEvidence-pinnedHuman vetted
Measuring the detection coverage you actually have.
A coverage read from what actually fired rather than a vendor heatmap: eight ATT&CK techniques run through Sigma over an OCSF store on one APT29-style chain, scored against planted ground truth at 3 detected, 1 noisy, 4 missed, with the D3FEND join read narrowly, since countering a technique isn't detecting it.
Read →
