Security Data Works

Field report

What two practitioners told me at RSA.

At RSA Conference 2025 I had two conversations, a few minutes apart, with people running security data at a scale most teams never see. One had normalized petabyte-scale telemetry into OCSF; the other had swapped a search engine for a columnar one and watched the bill drop. Between them they had built both halves of the open-architecture argument I'd been writing about, which was reassuring and clarifying in roughly equal measure, because it also showed me exactly how thin a single conference conversation is as evidence.

Reading time: about 11 minutes. Evidence: one unnamed verbal claim (treated as directional, not measured) and one set of figures from a vendor-co-authored blog (Tier C, conflicted, flagged inline). The point of the piece is what field conversations add to analyst reports and lab benchmarks, and where they stop.

Evidence scorecard

Claim 1: OCSF at scaleClaim 2: Huntress cost migration
SourceSingle unnamed practitioner, spoken account at RSA Conference 2025 (his preference, unverified against any artifact)A blog Huntress co-authored with ClickHouse; voiced at the booth by Chris Bisnett, Huntress's CTO
Evidence tierTier B for the existence of the work, no higher for the figuresTier C, conflicted: a vendor telling a success story about its own product
Headline number2 PB/day ingested, normalized into OCSF across 700+ source types~93% monthly cost drop (arithmetic on the dollar figures, not a stated number) migrating Elastic to ClickHouse
What is verifiedThat someone treated OCSF normalization at very large scale as a tractable engineering project and did most of the mapping himselfThe direction: a columnar engine cut cost meaningfully, a shape corroborated across other Elastic-to-columnar deployments
What is notThe 2 PB/day and 700+ figures against any artifact; not to be cited as established factThe precise percentage and dollar figures: a vendor's numbers about its own product, to verify before relying on them
Both claims surfaced in the same RSA Conference 2025 conversation, and neither number is measured or independently verified. The evidence-tier callouts below hold the full sourcing and the discount each figure earns.

The first conversation

Petabyte-scale OCSF, claimed on a sidewalk.

A friend mentioned, almost in passing, that his company was ingesting on the order of two petabytes a day of security logs and had normalized all of it into OCSF across more than seven hundred distinct log-source transformations. I asked him to repeat the throughput number because I thought I'd misheard, and then whether seven hundred meant fields or whole source types, and he said source types, and that he'd done most of the mapping himself. I have no reason to doubt him and every reason to be careful with the claim, because it is exactly that, a claim spoken at a conference, with no artifact I can point you to.

So I want to be precise about what kind of evidence this is. It is a single unnamed practitioner's verbal account, which I'd put at Tier B for the existence of the work and no higher for the specific figures. Two petabytes a day is roughly a hundred to a thousand times what a mid-sized enterprise generates, so if it holds it's a hyperscale outlier rather than a representative deployment, and the exact magnitude is the part I'd least want to repeat as fact. What I took from it was narrower and, I think, sturdier than the headline: someone had treated OCSF normalization as a tractable engineering project at very large scale rather than as a research problem, and had done the bulk of it himself.

What the claim is really about

Seven hundred mappings is the labor.

The number worth dwelling on is the seven hundred. People who haven't done OCSF mapping tend to picture a short list, where you map Windows Security events, map CloudTrail, map a firewall, call it thirty or forty source types and you're done. The reality at scale is that each vendor's output fragments into many event shapes that each need their own handling: Windows Security alone spans dozens of event IDs, CloudTrail covers hundreds of API actions across dozens of services, the Office 365 unified audit log has its own long list of record types, and every EDR splits into process, network, file, and behavioral streams. Seven hundred transformations isn't an exotic figure once you tally it; it's roughly what broad coverage costs.

That cost is the honest center of the OCSF story, and it's the part analyst reports and vendor whitepapers mostly skip. A whitepaper will tell you OCSF gives you vendor-neutral field names; it won't tell you what it takes to get seven hundred source types into those names and keep them there as the sources change. The encouraging thing I heard at RSA is that the work is finite and nameable rather than open-ended, a known list of sources where each one is a bounded mapping task, which is also why I think it's increasingly tractable with machine assistance. I've written up the per-source time math separately, with the numbers I actually tracked rather than the ones I heard at a booth, in the piece on LLM-assisted OCSF mapping, so I won't re-derive it here. The field report's contribution is just the existence proof: someone has carried this to seven hundred sources in production.

The second conversation

The storage-and-query half, at the Huntress booth.

A few minutes later we walked over to the Huntress booth and my friend asked for Chris. Chris Bisnett, Huntress's CTO, talked through the other half of the same problem: not getting data in and normalized, but storing and querying it cheaply once it's there. Huntress runs managed detection across a very large endpoint fleet, and the account he gave was that they had migrated their telemetry store from Elastic to ClickHouse because Elastic's cost and performance didn't hold at their scale, and that the columnar compression and cost picture improved substantially afterward.

I want to be careful with the specific figures here, more careful than the original conference write-up was. The numbers that get quoted for this migration (a roughly 93% monthly cost reduction, infrastructure spend falling from tens of thousands of dollars a month to single-digit thousands, and high events-per-second ingest on a small server footprint) trace back to a blog that Huntress co-authored with ClickHouse. That makes it Tier C and conflicted: it's a vendor telling a success story about its own product, with the customer's name attached. I find the direction of the result believable, because the Elastic-to-columnar cost story shows up across enough independent deployments that the shape isn't in doubt, but I wouldn't repeat the precise percentage as a neutral fact, and neither should you.

So I'll frame Chris's half the way the evidence supports: Huntress validated that a columnar OLAP engine can carry security telemetry that an inverted-index search store was straining under, and the cost moved in the expected direction. The headline percentage is a vendor number, while the architectural lesson, that the storage-and-query tier has a cheaper open option than the one most teams default to, is the part that survives the discount.

Why the pairing mattered

Two halves of one architecture, solved separately.

What struck me standing between those two conversations was that I'd just heard the two ends of the open-architecture argument described by two people who, as far as I know, had never compared notes. The first had solved ingest and transformation, which means getting heterogeneous security logs into an open, vendor-neutral schema at scale. The second had solved storage and query, keeping that data somewhere cheap and fast to search. Neither had built the whole stack, but between them they'd shown that each piece works in production rather than only on a slide, and the pieces are the ones the well-connected argument depends on: an open schema on one side, an open engine on the other, no single vendor owning both.

I'd make a modest claim from that and resist a larger one. The modest claim is that the components are real and field-proven, which means a team adopting this pattern is an early adopter following a path others have walked rather than a pioneer cutting a new one. The larger claim I'd resist is that because each half works separately, the integrated whole is therefore solved. It isn't, because the connective work of running normalized OCSF data through an open engine, with detection and retention and governance intact, is its own project, and a conference conversation tells you nothing about how cleanly those two halves would join. The parts exist, and the assembly is still where the engineering lives.

This pairing is also why open table formats matter to the story rather than just open schema. OCSF gives you vendor-neutral field names; an open table format like Apache Iceberg is what lets the normalized data sit under more than one engine without a migration, so the storage-and-query half stays as open as the ingest half. I work through that table-format choice in the Iceberg-versus-Delta piece, and the dual-tier storage pattern in schema-on-read versus schema-on-write.

What field reports are good for

A different kind of evidence than a benchmark.

You can read analyst reports all day, study vendor whitepapers, and benchmark engines in a lab, and each of those tells you something a conference conversation can't. But the reverse is also true, and it's why I keep having these conversations. An analyst report won't tell you that a real team carried OCSF to seven hundred source types in production; a whitepaper won't admit how many engineer-hours a single source mapping costs; a lab benchmark won't tell you whether anyone actually runs a columnar engine for security telemetry at fleet scale or just claims it on a slide. Practitioners at booths fill in the part where someone has done the thing and lived with the consequences.

The catch is that a field report tells you a thing was done rather than that it was done well, and it carries none of the disclosed methodology that lets you check. So I treat these conversations as leads rather than proof. The petabyte claim sent me to verify whether OCSF normalization is genuinely tractable at scale, which I could test in my own mapping work rather than take on faith. The Huntress figures sent me to the cross-engine cost question, which I'd want to run on disclosed methodology before quoting, and where two engines reading the same files can return different answers, so even the engine-swap half deserves its own correctness check, which I work through in the silent wrong answer.

That's the discipline I'd hand anyone using a conference conversation as input: take the existence proof, discount the numbers to their evidence tier, and go verify the part you intend to stand on. The research gets better when practitioners share what's actually working, and it stays honest when the rest of us remember what a hallway claim can and can't carry.