Writing · Pillar
Pipelines & streaming.
Cribl, Tenzir, Vector. Kafka, NATS, streaming-database decisions. Where pipeline lock-in moved after the SIEM lock-in eased.
Depth tier for ch4 / ch5 / ch7 · 12 essays
Listed from the front door down to the measured evidence. Start here marks the entry essay. The reading-level tag says whether you are about to read the frame (Orientation), the argument (Depth), or the narrowest first-party finding (Evidence). The durability tag flags whether a number is durable architecture or an Evidence-pinned result tied to a version or price and meant to be re-run.
- Start here OrientationEvidence-pinned
Cribl vs Tenzir vs alternatives.
Choosing your security data pipeline. The procurement-evidence-vs-OCSF-fidelity split, where Vector fits as the open-source third option, and what the v1 Capability Matrix puts at #1 across three archetypes.
Read →
- DepthEvidence-pinned
The pipe layer: what's missing from your AI security platform.
Tenzir as the OCSF-native pipe layer. What it ships today, what the production evidence floor actually is, and where it's the upgrade path from Cribl.
Read →
- DepthEvidence-pinned
Vector: the data router Datadog open-sourced.
Vector at the Archetype-C #1 spot for cost-and-lock-in-led shops. What VRL fluency costs, where Datadog stewardship raises trust questions, and the OCSF gap.
Read →
- DepthDurable
Pipeline lock-in.
Where switching costs moved next. The SIEM lock-in eased; the pipeline-tooling lock-in took its place. Which patterns reduce it; which vendors aggravate it.
Read →
- DepthDurable
The parsing layer nobody owns.
Security data quality breaks at the boundary between vendors, time most of all, and no one in the chain is paid to fix it. A first-hand case from a Palo Alto Splunk-app pull request, Zeek timestamps, and Tenable's nested data, and the argument for a fair broker.
Read →
- DepthDurable
Pipeline-based detection in stream processing.
Detection at the pipeline tier, before the lake. When it's appropriate, what it costs in operational complexity, and the trade-off against retroactive lake-side detection.
Read →
- DepthEvidence-pinned
Observability pipelines and the security overlap.
Datadog OP, Edge Delta, and the boundary question. Where observability pipeline tooling does the security job credibly, and where the security workload still needs purpose-built tools.
Read →
- DepthDurable
ETL vs ELT for security data.
Who owns the schema, and when. The shift from upstream-normalized ETL to downstream-normalized ELT, what it costs in storage, and what it earns in flexibility.
Read →
- DepthDurable
Kafka architecture deep-dive.
Kafka as the security-data stream bus. Partition design, retention, the broker-replication math, and where it leaks operational complexity at scale.
Read →
- DepthEvidence-pinned
Kafka to Iceberg: the integration hidden costs.
Streaming Kafka into Iceberg looks straightforward and isn't. Connector quality, schema evolution under load, exactly-once semantics, and the small-files problem.
Read →
- DepthEvidence-pinned
The streaming database decision.
Materialize, RisingWave, Flink SQL. Where streaming databases earn their keep for security workloads and where the batch lakehouse is still the right call.
Read →
- DepthEvidence-pinned
NATS JetStream: lightweight Kafka alternative, disqualified.
An honest disqualification. NATS JetStream looks like a Kafka simplification for security workloads; the durability and retention story disqualifies it. What it's actually good for instead.
Read →