Public production architecture teardown
Huntress on ClickHouse
MDR/EDR business operating at fleet scale. Replaced Elasticsearch with ClickHouse Cloud on the same workload — driven by economics, not vendor advocacy. Ruby-on-Rails application stack on top, Vector.dev as the routing tier, columnar OLAP underneath.
Monthly bill on ClickHouse Cloud. Was $70K on Elasticsearch. Same retention envelope, more events ingested. Roughly 93% cost reduction at the analytics tier, while throughput grew to 200K records/sec.
The pipeline
-
Sources
Endpoints & identities
3M endpoints · 1M identities
-
Route
Vector.dev (HTTP)
Batched, templated; 200K rec/sec
-
Store
ClickHouse Cloud
MergeTree; columnar compression
-
Aggregate
MV + AggregatingMergeTree
Hourly + daily roll-ups
-
Serve
Huntress SOC tier
SIEM + analyst dashboards
What composes, what’s brittle
- 16B events/day. Ingested across the fleet.
- Compression. Terabytes → dozens of GB on sorted tag data.
- Why Vector.dev. Lightweight, templatable, mature HTTP insert path.
- Why ClickHouse. Columnar OLAP beats inverted-index store on this workload.
- What survives. SQL-based detection content portable to other engines.
- What's brittle. Vector batching tuning; egress at very high scale.
Sources: ClickHouse case study · Huntress engineering blog · ClickHouse video "Lessons Learned Building the Huntress SIEM with ClickHouse"