One engine in front: StarRocks over shared Iceberg, and what it costs.

A security operations team is not a data platform organization, though the multi-engine literature assumes it is. Four engines means four upgrade cadences, four authentication stories to wire into the same SSO, four JDBC drivers to keep current inside analyst tooling, four failure modes to page on, and four places the same query can return a subtly different answer, which in security work is the failure that matters most because nobody notices it. The constraint underneath all of this is the talent pool, because people experienced in both security operations and data infrastructure are scarce and expensive, and a team that has hired one of them has not hired four engines' worth of them. The same scarcity is why security teams are right to be conservative about unproven tools, since adopting something new that is hard to understand, build, and run is a bet placed with the team's scarcest resource, and the move only makes sense when the wins are large on both axes at once, technical and operational, which is the bar everything in this essay should be read against.

So the question I keep coming back to is not which engine wins each workload, since I already know the answer is "different ones," but how few engines you can run before what you lose on workloads costs more than what you save on operations. This essay works through the strongest one-engine answer I can currently defend: StarRocks alone, in front of the same Iceberg tables everything else already shares, covers three of the four workloads the multi-engine pattern exists for, from a single MySQL-wire endpoint. Dremio could mount the same one-engine argument through Reflections' algebraic substitution, and my own matrix ranks it ahead of StarRocks as a standalone primary at all three archetypes I've scored, but the case I can defend for a SOC mid-migration runs through the stock-MySQL front door, which Dremio doesn't have, so StarRocks is the version of the premise under test here rather than a verdict against the Dremio version.

I want to make that case honestly, which means leading with the complication: my own lab numbers show StarRocks winning nothing on single-table scans at 100M rows, so the case rests on transparent materialized-view rewrite, native joins, and the front door rather than raw speed. The join leg is no longer a prediction, though, because the benchmark I pre-registered the morning of 2026-06-10 scored the same day: StarRocks took four of the five TPC-H-derived joins, and the result that matters most for this essay is that the whole field compressed — every SOC-shaped join finished in under 1.5 seconds on every engine I ran (one host, 10M–60M-row tables), which is the number the manageability argument has been waiting for. The join bench and its code are in the lab.

Start with the front door, because for security teams it decides whether anything else matters. StarRocks speaks the MySQL wire protocol natively; the frontend's query_port defaults to 9030 and the deployment docs connect to it with a stock mysql client, with a floor of MySQL client 5.5.0 or later [B-vendor-doc]. No sentence in the official docs certifies a specific Connector/J version, but StarRocks' own Flink and Spark connector pages use the stock MySQL JDBC driver (com.mysql.cj.jdbc.Driver against jdbc:mysql://fe:9030), so this is MySQL-protocol compatibility demonstrated through the stock driver rather than a formal compatibility matrix [B-vendor-doc, indirect]. The reason I care about this particular protocol is Splunk DB Connect: during a migration the SPL-to-lakehouse bridge is a JDBC connection, and a stock-MySQL-driver endpoint should mean DB Connect can query the lakehouse through StarRocks without a custom driver. When this essay first published, that "should" was Tier-D inference, flagged loudly as the cheapest untested leg on the board, with a promise to update the page either way. I've now run it (2026-06-10): DB Connect 4.1.1's bundled stock driver (mysql-connector-j-8.2.0) connected to a StarRocks 4.1.1 all-in-one container on the FE's 9030 port and returned correct rows through dbxquery, with no handshake or auth-plugin failure — the classic break points for "speaks the MySQL protocol" claims. Two limits ride with that: it's a connectivity smoke test over a three-row table, so it proves the protocol leg and says nothing about performance at migration scale, and DB Connect 4.1.1's own jars require Java 17 while the Splunk container ships no JVM at all, so the Java runtime is on you either way. The front door covers the fourth workload too, with a transport tax I'll price in a moment, which is why I count three of four covered rather than two.

The hot tier is the cleverest part of the design. StarRocks has supported asynchronous materialized views built on external Iceberg catalogs since v2.5 [B-vendor-doc], and the optimizer performs transparent query rewrite against them: "without the need to modify the query statement, StarRocks can automatically rewrite queries against the base tables into queries against the corresponding materialized view," using a rewrite algorithm based on the SPJG (select-project-join-group-by) form [B-vendor-doc]. The covered shapes run wider than I expected, because beyond single-table SPJG there's join rewrite across inner, outer, semi, and anti joins, aggregation rollup, union rewrite, nested MV rewrite to a default depth of 3, and from v3.3.0 a text-based rewrite matching the query's abstract syntax tree against the MV definition [B-vendor-doc]. For Iceberg specifically, rewrite over external-catalog MVs is supported on both of the doc pages that discuss it, and the controlling property, force_external_table_query_rewrite, defaults to true since v3.3 [B-vendor-doc].

What makes this usable for dashboards is the freshness gate. The mv_rewrite_staleness_second property (v3.1+, citing the later of two pins the docs disagree on) works like this: if the MV's last refresh falls within the window, StarRocks uses it for rewrite regardless of whether base tables changed, and if the refresh is older than the window, it checks the base tables before substituting [B-vendor-doc]. That's a one-knob staleness contract per view, which maps cleanly onto how SOC dashboards tolerate staleness. The partition machinery underneath carries version texture: partitioned MVs over Iceberg partition transforms arrived in v3.2.3 and cover identity, year, month, day, and hour transforms (mapped to date_trunc), though bucket and truncate transforms are explicitly unsupported; partition-level refresh for Iceberg is pinned at v3.1.7/v3.2.3 in the feature-support matrix; partition-level change detection arrived at v3.1.4, and the same page caveats that the change-detection path currently supports Iceberg V1 tables only [B-vendor-doc]. Those last two are adjacent features carrying different version pins on two different official pages, which tells you how fast this corner of the product is moving and why I cite versions everywhere.

The fair counter-position on this tier is Dremio, because Reflections are the more ambitious version of the same acceleration idea and the comparison is worth a paragraph. A StarRocks async MV is a declared object: you decide what to materialize, the optimizer rewrites SPJG-shaped queries against it, and the materialization lives in StarRocks' internal storage. A Dremio Reflection is matched algebraically, so the planner can substitute it into queries that don't textually resemble it, including partial sub-tree matches, the newer Autonomous Reflections layer infers candidates from query patterns without anyone declaring them, and the materialization is written as Parquet into your own object store, which keeps the accelerated copy in an open format instead of inside the engine [B-vendor-doc]. That last property is an architectural advantage, and the cost is documented too, because algebraic matching is planner work and Dremio's own tuning guidance treats planning-time growth with reflection count and join depth as a thing to manage [B-vendor-doc]. The two then flip on the other legs: Dremio created Flight SQL and donated the Arrow Flight SQL JDBC driver, so its columnar transport is the most mature in the field, while its front door is its own driver rather than a wire protocol existing tools already speak; StarRocks holds the stock-MySQL front door while its Flight lane is the experimental one. The acceleration story there is the Reflections layer rather than the engine underneath, so the head-to-head that would settle this tier, declared MV rewrite against algebraic Reflection matching at matched freshness, has as far as I can tell never been benchmarked by anyone, including me, and it's first on my follow-up list.

The join tier is the engine's home turf. StarRocks runs a vectorized, pipeline-friendly hash-join core wired into four physical strategies the cost-based optimizer weighs at plan time (Colocate, Bucket Shuffle, Broadcast, and Shuffle, each hintable if you disagree with the planner), and the consumed side of a join generates runtime filters, Bloom or IN, that get pushed back into upstream scans to cut probe input early [B-vendor-doc]. Whether those runtime filters reach Iceberg scans matters for this essay's premise, and no doc page states it in versioned prose, but the fix stream presupposes it: PR #57651 fixes "the bug of the runtime filter cannot be pushed down for iceberg table with equality delete," which can only be a bug if the pushdown path exists [B-community], and the 4.1.1 release notes enable TopN runtime-filter pushdown for Iceberg aggregations alongside a datetime min/max scan optimization for Iceberg [B-vendor-doc]. I read that stream as plan-level behavior being actively extended rather than a marketing claim, and that's the direction of evidence I prefer to bet on.

The missing workload of the four is federation, and it stays missing. StarRocks' external catalog list runs to roughly ten types — Hive, Iceberg, Hudi, Delta Lake, JDBC, Elasticsearch (v3.1+), Paimon (v3.1+), a Unified catalog (v3.2+), plus MaxCompute and Kudu — and every one of them is lake- or database-shaped [B-vendor-doc]. Trino 481 ships 37 connectors, including Kafka, Cassandra, MongoDB, OpenSearch, Pinot, Druid, ClickHouse, Snowflake, BigQuery, Prometheus, Loki, and Redis [B-vendor-doc, comparator]. So the comparison comes out this way: StarRocks federates lakes and JDBC databases, while Trino federates the operational, streaming, and observability systems StarRocks cannot reach, and no amount of MV machinery changes that. I'll weigh what that costs below, because I think it's the giveaway whose weight varies most by team.

I'm not the first to make the fewer-engines call either. SmartNews publicly replaced a Trino-plus-ClickHouse pair with StarRocks citing the dual-engine operational burden, though that account travels through the CelerData channel, so weight it as vendor-adjacent [C-vendor-marketing], and Uber built and then retreated from Neutrino, their Presto-fronting-Pinot translation layer, on maintenance cost (Tier B). The academic record points the same way, because a decade of polystores (BigDAWG, Apache Wayang, CloudMdsQL, Polypheny, all Tier-A papers) tried optimizer-grade cross-engine placement and none survived to production, while what ships everywhere is thin, deterministic rule routing keyed mostly on the time predicate.

Before weighing costs I should reconcile this essay with my own graduated hypothesis, because "no single engine wins every workload" sits at 0.97 confidence in my tracker and I'm not walking it back. Specialization is real and I've measured it. The single-front move is the manageability counterweight rather than a refutation, and the question it poses is whether the spread between the per-workload winner and StarRocks, at the single-node scale a SOC runs, is large enough to pay for the second, third, and fourth endpoints. That's a quantitative question, and the join half of it now has a scored answer below; here are the giveaways as I'd weight them today, heaviest first.

The first giveaway is measured single-table performance, and I have to be blunt about my own numbers. On the 100M-row OCSF network_activity workload matrix (median of four trials, single host), StarRocks won no shape: ClickHouse took the full-scan count at 10.5 ms against StarRocks's 48.2; DuckDB took the dst_port=3389 needle at 77.7 ms with StarRocks a close second at 95.0, took the group-by at 103.1 against 194.4, and took the high-cardinality distinct at 4,091 ms against 5,180, with Trino erroring on that shape entirely [B-first-party]. The concurrency sweep tells the same story from a different angle, because StarRocks scales from 16 to 42 queries per second and plateaus around eight concurrent clients, while ClickHouse scales from 20 to 58, overtakes DuckDB by sixteen clients, and shows the gentlest p95 growth in the field at 59 ms to 355 [B-first-party]. Both runs are one host, so they measure scheduling under contention rather than cluster concurrency, but the direction is clear enough that I won't argue with it: if your dominant workload is hot scheduled scans at high concurrency, ClickHouse is the better-measured choice, and the MV rewrite story has to earn the difference back. The single-front case rests on the rewrite, the joins, and the endpoint, and I'd rather say that plainly than let the premise imply a scan-speed story my own benches contradict. The concurrency sweep is published with the rest of the benches, and the worked scorecard shows how this concurrency reweighting flips the ClickHouse-versus-StarRocks ranking from one archetype to the next.

The second giveaway is materialized-view freshness on streaming security data, where the constraint stack is taller than the headline feature suggests. External-catalog MVs refresh on a fixed interval or manually, with no event-driven option [B-vendor-doc]. Partition-level change detection only works against Iceberg V1 tables [B-vendor-doc], an unpartitioned base table forces a whole-MV refresh every cycle [B-vendor-doc], and Anton Borisov's account of running these MVs at Fresha is the best practitioner writeup I've found of what that does in production: partition change tracking where "if 10 records touched 10 partitions, you'd recalculate 10 partitions. The granularity was too coarse, the cost unpredictable, sometimes dangerous," refreshes that can OOM on many group-by keys over large data, and incremental view maintenance that in its first phase covers append-only Iceberg only, with joins still reading full snapshots from one side and retractable changes waiting on Iceberg V3 CDC [B-practitioner, no benchmarks in the piece]. The vendor's own release notes corroborate the in-progress state, because 4.1.0 extended incremental MV refresh to Iceberg append-only tables and then 4.1.1 disabled query rewrite over INCREMENTAL/AUTO MVs entirely while rejecting FORCE and partition refresh on them [B-vendor-doc], so the incremental path exists but isn't yet usable for transparent acceleration. The practical reading: a dashboard tier that tolerates minutes of staleness sits comfortably inside what mv_rewrite_staleness_second can contract for, while sub-minute alerting does not and shouldn't live in this layer anyway; it belongs to the streaming pipeline upstream of the lake.

The third giveaway is Iceberg write and maintenance maturity, which matters because a front engine still sits in an architecture where something has to write. StarRocks can INSERT INTO and INSERT OVERWRITE Parquet-formatted Iceberg tables from v3.1 (Parquet-only on the sink, no ORC writes), can CREATE TABLE and CTAS from v3.1, and only gained DELETE, via position delete files, in 4.1.0 [B-vendor-doc]. MERGE INTO hasn't shipped in any release as of 4.1.1 (a parser-and-analyzer PR landed on main the day I drafted this, so the gap is being worked), and the open feature issue states the consequence better than I would: the workaround chains "DELETE + INSERT or UPDATE + INSERT, which (a) splits the operation across statements so readers can observe the partial state, (b) duplicates the join logic each side, and (c) silently does the wrong thing when a source row matches more than one target row" [B-community, #73684]. That same issue asserts UPDATE works on Iceberg, while the official 4.1 release notes document only DELETE, so I'm not claiming UPDATE in print until I can verify it. Trino's Iceberg connector, for comparison, supports INSERT, UPDATE, DELETE, TRUNCATE, and MERGE [B-vendor-doc, comparator], and StarRocks' Iceberg maintenance tooling was still filling in at 4.1, which added the rewrite_manifests procedure and extended expire_snapshots and remove_orphan_files with finer-grained arguments [B-vendor-doc].

I also owe you a tension from my own paid work here, because my private Capability Matrix scores StarRocks 3 out of 5 on the "Iceberg native vs connector" criterion, Tier C, flagged "less mature," last in the field behind Trino at 5 and ClickHouse and DuckDB at 4, while my public offering page's one-word identity for StarRocks is "Iceberg-native." Both are mine and they pull against each other. The reconciliation I believe is that the read path has filled in fast — position deletes from v3.1.0 on Parquet, equality deletes from roughly v3.2.5 per the catalog page's phrasing, time travel at v3.4.0, Iceberg metadata tables at v3.4.1, hidden partitions on CREATE at v4.0, and Iceberg V3 default values plus row lineage in 4.1 [B-vendor-doc] — while the write path trails Trino and Spark by a clear margin. So the strongest version of this essay's claim is StarRocks as the read-and-serve front, with writes delegated to whatever already owns ingestion, which in the estates I've seen is usually Spark. That framing is also how I square this essay with my own matrix's cross-archetype verdict on StarRocks ("No archetype where it dominates; right home is dual-engine pattern"), because one engine in front was never one engine in the building: Spark keeps writing, the streaming pipeline keeps doing sub-minute detection, and the question this essay argues is the narrower one of how many engines need to sit between an analyst and the tables.

The fourth giveaway is the federation breadth I set aside earlier, and its weight depends almost entirely on your estate. If investigation queries routinely need to reach a Kafka topic, the MongoDB behind an internal app, or a Prometheus instance, then Trino's 37 connectors are not optional and the single-front pattern needs Trino beside it, at which point you're running two engines and should read my dual-engine material instead. But in the security estates I've worked in, federated sources tend either to get ingested into the lake eventually, because retention and correlation demand it, or to stay out of SQL reach entirely, so federation reads to me as a transition-period workload more often than a steady-state one, and giving it up is cheaper at year two than at month two. That's an opinion from operational pattern, not a measurement, and I'd weight it accordingly.