Writing · Pillar
Detection & hunting.
Detection-engineering maturity ladder, MLOps for hunters, latency tiers, feature stores, PEAK methodology on a modern data stack.
Depth tier for ch3 (Trustworthy) and ch6 (what good looks like) · 17 essays
Listed from the front door down to the measured evidence. Start here marks the entry essay. The reading-level tag says whether you are about to read the frame (Orientation), the argument (Depth), or the narrowest first-party finding (Evidence). The durability tag flags whether a number is durable architecture or an Evidence-pinned result tied to a version or price and meant to be re-run. The last tag is the provenance state — who held the pen and who checked the work — explained in full on how these are made.
- Start here OrientationDurableAI-drafted, observed
The ground you're already standing on.
Detection engineers are doing data modeling every day without naming it, and a rule can compile clean and quietly never fire because one of those mappings was wrong with nothing set to catch it. The vocabulary, the silent-failure mode, and a deductive check that caught 100% of injected type-crossings on a 925-row corpus.
Read →
- DepthDurableAI-drafted, observed
Detecting the OT you can't parse.
Most industrial protocols will never get a deep parser, so the question that decides whether you can monitor an OT network is how much you can detect from behavior — timing, flow, who talks to whom — before a parser ever runs. Why behavioral-first gives immediate east-west coverage across the long tail, why it fits the NERC CIP-015-1 internal-monitoring mandate, where it stops, and the falsifier that would change my mind. A practitioner argument, Tier B, not a benchmark.
Read →
- DepthDurableAI-drafted, observed
What your data means vs what shape it is.
Schema conformance checks the shape of a field, not its meaning, so a mapping can fit OCSF exactly and still mean the wrong thing. Eight real meaning-crossings from a six-schema crosswalk corpus, made visible by a deductive check the syntax pass waves through.
Read →
- DepthDurableAI-drafted, observed
Catching the mistake that kills a detection.
The wrong field mapping that silently kills a detection is catchable with a reasoner and a few disjointness assertions D3FEND never shipped. How to run that check yourself against your own mappings, and the honest limits of what it catches.
Read →
- DepthDurableAI-drafted, observed
Who actually does the hunting.
The analyst reverse-engineering a broken field mapping at 2am is doing applied ontology, and naming the work hands them open tooling and a community that's been building the same map from the other side.
Read →
- DepthEvidence-pinnedAI-drafted, observed
The tools you can use today.
A practitioner's map of the open detection-grounding stack: OCSF, D3FEND, Sigma, the ROBOT/ELK reasoner toolchain, and the six-schema crosswalks, with an honest read of what each piece is for and exactly where it's thin.
Read →
- DepthDurableAI-drafted, observed
The detection engineering maturity ladder.
From ad-hoc rules to coded detections to detection-as-code with CI. What each rung costs, what becomes possible, and where most security teams stall.
Read →
- DepthEvidence-pinnedAI-drafted, observed
PEAK and the lakehouse.
How modern data stacks enable threat hunting. The Splunk PEAK framework ported to lakehouse infrastructure, what works, and what doesn't.
Read →
- DepthEvidence-pinnedAI-drafted, observed
Three latency tiers: detection, hunting, analysis.
Sub-second for detection, sub-minute for hunting, sub-hour for analysis. The architecture that serves all three honestly, and the trade-offs at each tier.
Read →
- DepthEvidence-pinnedAI-drafted, observed
MLOps tools for threat hunters.
Reproducibility for threat-hunt notebooks. What MLOps tooling brings to hunters that they don't already have, and where the friction lives.
Read →
- DepthEvidence-pinnedAI-drafted, observed
Where detection-as-code notebooks should live.
The notebook is how security analytics gets shared, and the live decision is the proprietary-but-production Databricks notebook versus the open, reproducible marimo .py. Why the lock-in moved up to the authoring layer, what marimo provably fixes, and why the detection you ship is text either way.
Read →
- DepthDurableAI-drafted, observed
Every security tool ships with a silent configuration gap.
Manageability as a force multiplier: the distance between how a control is configured today and how it should be run, made visible and rankable across the whole stack. Threat-informed reallocation and CIS-benchmark conformance scanning optimize different denominators, and the honest ranking weight is threat exposure against an independent baseline rather than raw configuration distance.
Read →
- EvidenceEvidence-pinnedHuman vetted
The assurance gap no single tool closes.
No single security tool knows your environment: on a planted 140,000-cell benchmark the best single tool recovers 47.7% of the truth and a freshness-scored cross-tool merge recovers 75.6%, while a 24.4% residual stays dark no matter how many tools you own. The transferable claim is the ordering, which holds across the whole parameter sweep, plus the entity-resolution tax a contested join key adds.
Read →
- EvidenceEvidence-pinnedHuman revised
The query engine returned the wrong answer and didn't tell you.
Over an open format the engine is supposed to be interchangeable. In my lab one returned a filtered count tens of rows short of the others on the identical Parquet, no error raised, and a timing-only benchmark would have published it as a win. Why a cross-engine answer-equality gate is the only thing that caught it.
Read →
- EvidenceEvidence-pinnedHuman revised
The better the model, the quieter the wrong answer.
I answered one OCSF question battery six ways across three model tiers. On compute-over-population questions no LLM-authored arm was ever correct, and the more capable model made the wrong answer quieter rather than rarer; an execution loop and self-consistency both failed the same way. Safety lived in the layer that refuses a query it can't answer, not in the model.
Read →
- EvidenceEvidence-pinnedHuman revised
Parquet doesn't hash the way your security tools assume.
Chain-of-custody, WORM, and dedup workflows key on a file hash, but Parquet isn't byte-reproducible by default: parallel row-group order makes the same logical data produce a different SHA-256 every write. A faithful re-export breaks the hash and looks like tampering. Force determinism (threads=1 or ORDER BY, ~20% smaller too), and hash the logical content, not the bytes.
Read →
- EvidenceEvidence-pinnedHuman vetted
Measuring the detection coverage you actually have.
A measured-firing coverage read, not a vendor heatmap: eight ATT&CK techniques run through Sigma over an OCSF store on one APT29-style chain, scored against planted ground truth at 3 detected, 1 noisy, 4 missed, with the D3FEND join kept honest that counters is not detects.
Read →
