V3 changed the boundary conditions. V4 is changing them again.

Puffin-based deletion vectors.

V3 introduces Puffin-based deletion vectors as the merge-on-read mechanism, replacing V2's position-delete and equality-delete files. A deletion vector is a compact bitmap stored in a Puffin file alongside the data file, marking which rows are logically deleted without rewriting the data file itself.

The security implications are direct. GDPR right-to-erasure becomes a metadata operation. In V2, a DELETE WHERE user_id = 'X' query wrote one or more position-delete files, and those files then needed to be compacted away on a maintenance schedule. The user's data physically remained in the data files until compaction; the auditor saw "deleted" in the table query, but the bytes were still on disk. V3 deletion vectors make the delete a metadata bitmap update. Compaction still cleans up eventually, but the operational urgency is much lower because there's no delete-file accumulation problem.

Streaming sinks become cheaper. Streaming pipelines that write to Iceberg with merge-on-read historically accumulated delete files at a rate proportional to the update rate, which forced aggressive compaction schedules to keep query performance acceptable. With deletion vectors, the per-row update cost drops materially and compaction becomes a less frequent maintenance operation.

And the "Iceberg MoR is less mature than Delta" critique closes. Delta Lake has used Puffin-based deletion vectors for longer and has more production mileage with them, but the capability gap between Iceberg and Delta on merge-on-read maturity is now closed. The remaining difference is production-track-record-years, not architectural mechanics.

What deletion vectors don't fix: the write amplification of large bulk updates. If you're rewriting half the rows in a partition, you're still rewriting half the data file — deletion vectors are only a win for sparse updates and deletes. They also don't change the latency floor for detection-tier workloads; even with V3, Iceberg is still a hunting/analysis-tier format, not a detection-tier surface. Sub-second detection still belongs in the streaming layer, not in the lakehouse query engine.

Variant type — semi-structured data without flattening.

V3 introduces a Variant type for storing semi-structured data — typically JSON — with native predicate pushdown and field extraction. The corresponding Parquet Variant type was ratified in October 2025 and Iceberg V3 is one of the first table formats to adopt it.

Security log ingestion is the canonical Variant use case. Almost every security log type that started life as JSON (CloudTrail, Kubernetes audit, Azure Activity, Okta system logs, GitHub audit, Slack audit) has the property that field presence carries information. The classic example:

{
  "userIdentity": {
    "sessionContext": {
      "attributes": {
        "mfaAuthenticated": "true"
      }
    }
  }
}

In CloudTrail, the mfaAuthenticated field is only present when MFA was used. If you flatten this to a wide column schema, you lose the distinction between "the field was absent" (no MFA) and "the field was NULL" (which never happens in CloudTrail but is what your flat schema produces). A detection rule checking WHERE mfaAuthenticated = false returns zero results and silently breaks. That's not a hypothetical — it's the failure mode the flattening anti-pattern essay documents.

V3 Variant fixes this by construction. You store the entire userIdentity object as a Variant column; the absence-vs-NULL distinction is preserved natively; predicate pushdown still works through variant_get() calls; and the same query works across any V3-aware engine without ETL rewrites.

V2 alternatives existed, all imperfect. ClickHouse Nested types worked inside ClickHouse but weren't portable across engines. Storing as a JSON STRING column with manual parsing worked but lost predicate pushdown and forced every query to parse the JSON. Flattening and accepting the absence-vs-NULL bug worked for some logs (where field presence doesn't carry information) but silently broke detections like the CloudTrail MFA case. V3 Variant is the cross-engine answer that none of the workarounds were.

Adoption caveat: Variant requires variant_get() and Variant-aware predicate pushdown in the query engine. As of April 2026, support is uneven: Spark 4.1 has it, DuckDB has partial support, Trino is rolling it out, Snowflake has its own native semi-structured type that maps to Variant on read but not on write. By Q4 2026 this should be a non-issue; for now, verify your engine version.

Row-level lineage — row IDs + last-updated tracking.

V3 introduces row IDs and last-updated tracking at the row level. Every row in a V3 Iceberg table has a stable identifier and a last-modified timestamp tracked through metadata, without requiring the data files themselves to carry the information. This is the V3 feature with the largest set of net-new use cases that didn't exist in V2 at all.

CDC-from-Iceberg without external metadata. In V2, building a change feed from an Iceberg table required either snapshot diffing (compute the difference between snapshot N and N+1) or external metadata tracking via Debezium-style log scraping. Both are operationally heavy. V3 row lineage lets you query "all rows that changed since timestamp T" directly from Iceberg metadata, with no external infrastructure.

dbt incremental without watermark columns. The standard dbt incremental pattern uses a watermark column (WHERE event_time > (SELECT MAX(event_time) FROM {{ this }})) to identify new rows. This pattern silently drops late-arriving events that arrive after the next dbt run. V3 row lineage lets you use the row's last-updated timestamp instead, which is robust to late arrivals because the lineage timestamp is the processing time, not the event time.

Audit trail integrity for compliance. For tables under regulatory audit (PCI-DSS, HIPAA, GDPR), being able to point at a specific row and say "this row was last modified at T by transaction X" without external infrastructure is a compliance simplification. V2 required either external audit logs or full table snapshots; V3 makes this a metadata query.

Incremental detection re-firing. If a detection rule changes (the SOC team adds a new IOC, refines a regex, fixes a bug), you typically want to re-evaluate it against historical data — but only against events that haven't already been evaluated by the new rule version. V3 row lineage lets you express this as "evaluate rule version 1.7 against rows whose last-modified timestamp is newer than the rule version 1.7 deployment time."

Adoption caveat: Row lineage is the V3 feature with the least mature engine support as of April 2026. Spark 4.1 has experimental support; Trino and DuckDB have it on the roadmap; Snowflake hasn't committed to a timeline. The use cases above are all real but most won't be production-deployable until 2027.

Default values for columns.

V3 allows you to add a column with a non-null default value as a metadata-only operation. In V2, adding a column with a default required either rewriting the entire table or accepting that the new column would read as NULL for old rows.

Schema evolution for security log tables is constant. New OCSF schema versions, new vendor field additions, new internal enrichment columns — most security tables see a schema evolution event every few weeks. In V2, adding a column with a default like WITH DEFAULT 'unknown' required reprocessing the entire table to backfill the default into existing rows. For petabyte-scale tables, that was a multi-day, multi-thousand-dollar operation.

V3 makes this a metadata-only operation: the default is recorded in the table schema and applied at read time for rows that don't have an explicit value. The data files are untouched. What it doesn't fix: default values still don't help if you need to backfill existing rows with computed values (e.g., "compute risk_score for all CloudTrail events from the last 90 days using the new model"). That's a write operation, not a schema evolution, and V3 doesn't change it.

Other V3 features (brief).

• Geospatial type — useful for IoT and physical security data with location attributes; not load-bearing for typical security-data use cases.
• Timestamp(nanos) — nanosecond-precision timestamps. Marginal for most security data (microsecond is fine), but matters for EDR and packet-capture ordering claims that some lakehouse architectures make loosely.
• Unknown type — forward-compatibility primitive that lets V3 readers handle columns of unknown future types gracefully. Mostly an ecosystem-stability feature, not directly user-facing.
• Encryption improvements — incremental tightening of the V2 encryption story. Useful for compliance-sensitive deployments but doesn't change the security architecture conversation meaningfully.

Variant type engine fragmentation.

V3 Variant is purpose-built for cross-engine portability, but the engine ecosystem isn't standardizing on the same surface API for it. Spark uses variant_get(). DuckDB has its own JSON-like syntax. Snowflake's existing semi-structured VARIANT type maps loosely to V3 Variant but not exactly. Trino is debating between several proposals.

The risk: by the time engine support is "complete" in late 2026, the query syntax for Variant fields will be different across engines, defeating the cross-engine portability that's the whole point. Teams will end up with "Variant in the storage layer, but per-engine query rewriting in the application layer" — most of the work without most of the benefit. Watch for an Apache Iceberg community spec that pins down a canonical Variant access syntax in the second half of 2026.

Row lineage and CDC production-readiness.

The CDC-from-Iceberg use case is the one most worth being excited about, but it's also the one where the engine support gap is the largest. As of April 2026, only Spark 4.1 has experimental row-lineage support. Trino and DuckDB are on the roadmap. Snowflake hasn't committed.

The risk: production CDC consumers (Debezium, Fivetran, Hightouch, RisingWave's Iceberg source) will need a year or more to integrate V3 row lineage into their existing CDC plumbing. The leading indicator to watch is a production reference deployment that uses row lineage to feed a downstream system without snapshot diffing or external metadata. As of April 2026, none has surfaced publicly. If that hasn't happened by Q4 2026, V3's CDC story will have shifted from "promising" to "spec-only."

Iceberg ships substantively every twelve to eighteen months. V2 in 2022, V3 in 2025, V4 under design now for likely 2026–2027 adoption. Each tranche closes a different set of boundary conditions. V3 closed deletion vectors and Variant and row lineage. V4 will close metadata I/O cost, nested-field statistics, and catalog authority. V5 will close something else — the projects already filing as v5 candidates include richer compression, time-series-native partitioning, and tighter Variant predicate semantics.

The pattern matters more than any single version. A security architecture that bets on Iceberg is betting on the cadence — on the project shipping substantive improvements at a predictable rhythm with backward-compatible read semantics. That bet has paid off through V1 → V2 → V3. The V3 → V4 transition is the third proof point.

The case against was always "what if it stalls?" — what if V3 is the high-water mark, what if the community fragments, what if a competing format wins. V4's design momentum is the strongest evidence against that case so far. The boundary conditions for the V2 case kept getting smaller. The boundary conditions for the V3 case are getting smaller too.