Public production architecture teardown
Standard Chartered — self-managed SIEM on Databricks
A global systemically-important bank replaced its traditional SIEM with a self-managed security lakehouse on Databricks — a distributed, multi-cloud Delta Lake architecture where detection content runs as code against the bank's own data plane rather than inside a per-GB SIEM. Presented by the Standard Chartered security team at Databricks Data + AI Summit 2025.
Reduction in time-to-detect after the SIEM-to-lakehouse move, alongside 92% faster threat investigation, 60% better detection accuracy, and ~35% cost reduction. These are vendor-published customer figures (Databricks DAIS 2025 / customer use-case set) — the bank is named and on the record, but the multipliers are the bank's and Databricks' own, not independently reproduced.
The pipeline
-
Sources
Multi-cloud security telemetry
Endpoint · identity · cloud · network across a distributed multi-cloud estate
-
Store
Delta Lake (distributed, multi-cloud)
Open table format; long retention without per-GB SIEM economics
-
Govern
Unity Catalog
Access control + lineage over regulated security data
-
Detect
Detection-as-code on Databricks
SQL / PySpark detections against the bank's own data plane
-
Serve
SOC + investigation
Faster triage; 92% faster investigation reported
What composes, what’s brittle
- 80% faster TTD. Time-to-detect cut after replacing the traditional SIEM.
- 92% / 60%. Faster investigation and better detection accuracy (bank-reported).
- ~35% cost. Reduction vs. the prior SIEM model — security telemetry off per-GB licensing.
- Why self-managed. Distributed multi-cloud lakehouse keeps detection on the bank's own data plane, not a vendor's query tier.
- Why it's credible. Presented by Standard Chartered's own team at DAIS 2025 — a named global bank, on the record.
- What's vendor-reported. Metrics are Databricks/SCB-published, not independently replicated; treat the architecture as durable, the multipliers as the bank's.
Sources: Databricks Data + AI Summit 2025 session "Revolutionizing Cybersecurity: SCB's Journey to a Self-Managed SIEM" · Databricks "Data Intelligence in Action: 100+ Data and AI Use Cases" (Standard Chartered security use case).