Security Data Works

Start here

How this site fits together.

There's a lot of research here, and it isn't obvious on a first visit where to begin or how the pieces connect, so this page is the reading order. The short version is that the site makes one argument, that security data is a data-engineering problem and mostly a solved one, backs it with evidence you can check, and then goes deep in whichever direction your question points. Every claim is labeled by how strong its evidence is, because I want you to be able to tell what I measured from what I'm only claiming.

If you only have ten minutes, read the thesis and skim one benchmark on the lab page. Everything else is depth on demand.

The map: claim, evidence, depth.

The core sections play one of three roles. The thesis states the claim; the matrix, the lab, and the research notebook carry the evidence for and against it; and the writing catalog plus the reference architectures are the depth, where the argument gets worked out topic by topic. Engagements sits apart, because it was the same method applied to a client's own environment, and it's paused while I work full-time. Everything else on the map is free to use.

How the site's sections connectThree columns. Left: the Thesis states the claim. Middle: three evidence surfaces test it — the Matrix scores platforms, the Lab holds first-party benchmarks, and Research tracks open hypotheses and contradictions. Right: two depth surfaces — the Writing catalog's ten pillars and the MOAR reference architecture. Arrows flow from claim to evidence to depth, and a band beneath links to Engagements, where the method was applied to a client's environment. Engagements are paused while I work full-time.THE CLAIMTHE EVIDENCETHE DEPTHThesisthe argument, three propertiesMatrixthe scoring instrumentLabfirst-party benchmarksResearchopen hypotheses + contradictionsWriting80+ essays, ten pillarsMOAR Architecturethe reference designEngagements (paused while I work full-time) · the same method, applied to one environment
The reading order runs left to right: the thesis states the claim, three evidence surfaces test it (and keep the contradictions on the record), and the depth surfaces work it out topic by topic.

A first visit in three moves.

  1. 1

    Read the thesis, the argument that security data is a data-engineering problem, and mostly a solved one, and the three properties it holds every platform to: trustworthy, well-connected, and performant. It's a short read, and the rest of the site hangs off it.

  2. 2

    Check the evidence: first-party benchmarks with methods and raw results posted, plus a cost model you can run against your own numbers. The Matrix is the scoring instrument behind the platform evaluations, and the research notebook keeps the open questions and the contradictions on the record, so you can check the argument instead of taking it on trust.

  3. 3

    Go deep where your question lives, in the essay catalog, which is organized as ten pillars in dependency order, from storage formats up through detection engineering and economics, and each pillar marks its own "start here" essay so you can enter mid-stack without reading everything above it.

If you came with a specific question.

Seven three-essay reading paths, each built to answer one question a security data team actually asks, so you can read three pieces instead of eighty.

Leaving Splunk without breaking detections

The cost case, the migration trap most teams walk into, and what the timeline actually costs.

  1. 1The cost math: schema-on-read vs schema-on-write
  2. 2The field-mapping anti-pattern
  3. 3Hidden costs and timeline reality

Whether you can trust your data

The quietest failures in security data, in the parsing layer and in the measurement problem underneath it.

  1. 1The parsing layer nobody owns
  2. 2Flattening away your detection logic
  3. 3How to run a benchmark that doesn't lie

Picking the query engine

Where each engine wins, from petabyte-scale detection down to an analyst's laptop.

  1. 1ClickHouse at petabyte scale
  2. 2DuckDB for analyst-driven hunting
  3. 3Push vs pull query engines

When hunting becomes data science

The path threat hunters are already on, made reproducible.

  1. 1PEAK and the lakehouse
  2. 2MLOps tools for threat hunters
  3. 3Where detection-as-code notebooks should live

Securing the AI you're deploying

One agent risk told three ways, the AI attack techniques the defensive ontology can't yet reach, and why a more capable model makes a wrong answer over security data quieter rather than rarer.

  1. 1The lethal trifecta, explained three ways
  2. 2The blind quadrant over AI systems
  3. 3The better the model, the quieter the wrong answer

Evidence: The ATLAS Wall: which AI controls to deploy first →

Whether your detections are telling the truth

The silent mapping failure that lets a rule compile clean and never fire, the gap between a field's shape and its meaning, and what no single tool can see about your environment.

  1. 1The ground you're already standing on
  2. 2What your data means vs what shape it is
  3. 3The assurance gap no single tool closes

Picking the table format

What the open stack is made of, the format decision on its production record, and the measured finding that how data enters decides more than which format holds it.

  1. 1MOAR, explained
  2. 2Iceberg vs Delta Lake for security data
  3. 3The write pattern is the architectural decision

Start where your question is.