Public production architecture teardown
Atlassian — Project Banyan
Lakehouse-native security data platform on the Databricks medallion architecture: security telemetry normalized to OCSF on the Silver layer, Unity Catalog as the governance plane, more than 21 billion security events queryable in under a minute. Atlassian built and operates it; this is SDW's reading of the public architecture (the Databricks customer story and the Data + AI Summit talk), not work SDW delivered.
Security events queryable in under a minute. Photon cut query times from 17 s to 5 s, retention extended from 30 days to 12 months, and operational ingest cost dropped roughly 80%. Atlassian's Data + AI Summit talk puts the daily scale at many petabytes of security data. Winner of Databricks' 2026 Transformation Award.
The pipeline
Sources
Endpoint · network · identity · cloud · app
Full-breadth security telemetry
Bronze
Raw Delta retention
Source-fidelity, no lossy pre-processing
Silver
OCSF normalization
Every source mapped to OCSF; entity resolution
Gold
Analyst surface
Detections and investigation
Govern
Unity Catalog
Governance and access plane
What composes, what’s brittle
- 21B+ events, sub-minute. Queryable in under a minute on the lakehouse.
- 17 s → 5 s. Query times with Photon.
- 30 days → 12 months. Retention extended on cheaper storage.
- ~80% lower ingest cost. Operational ingest-cost reduction.
- OCSF on Silver. One normalized schema the detection surface depends on.
- What SDW did. Read and scored the public architecture; Atlassian built and runs it.
Sources: Databricks customer story "Atlassian Modernizes Threat Detection" (databricks.com/customers/atlassian/security-lakehouse, 2026-03-24; Niels Heijmans, Chief Security Architect; David Cross, CISO) · Atlassian Data + AI Summit 2026 Security Lakehouse talk