Practical implementation

Security data platform migration: hidden costs and timeline reality.

You budgeted for technology. You forgot about migration. According to industry research, 67% of security data platform migrations require external consulting, and actual costs run 40–100% above initial technology-only estimates.

Reading time: 13 minutes. Evidence tier: A (industry research from Gartner / Forrester, 12 practitioner interviews, 11 enterprise migration case studies).

The story

The $300K project that became $1.2M.

You've made the business case. Your CFO approved $300K for a modern security data platform — licensing, infrastructure, the works. You're trading a $2M/year SIEM for a more efficient lakehouse architecture. The ROI is obvious.

Six months later, you're in an emergency meeting explaining why you need another $500K–900K.

What happened? You budgeted for technology. You forgot about migration. This isn't unique to you. Industry research shows 67% of security data platform migrations require external consulting, and actual costs run 40–100% above initial technology-only estimates.

The iceberg

What you budgeted vs. what's actually there.

What you budgeted (technology only)

Platform licensing:         $100K–300K/year
Infrastructure (cloud):     $50K–150K/year
Query engine licensing:     $50K–100K/year
─────────────────────────────────────────────
Total:                      $200K–550K/year

This looks reasonable. It's 75–90% cheaper than your legacy SIEM. Your spreadsheet shows a 12-month payback period. The CFO is happy. But this is only 30–40% of actual first-year cost.

What you forgot (migration costs)

Migration labor:
├─ External consulting:           $200K–500K (6–12 months)
├─ Internal team time:            $150K–300K (opportunity cost)
└─ Project management:            $50K–100K

Parallel operation:
├─ Duplicate licensing:           $100K–200K (6–12 month overlap)
└─ Duplicate infrastructure:      $50K–100K

Detection rule migration:
├─ Rule conversion:               $100K–200K (500–1000 rules)
├─ Testing and validation:        $50K–100K
└─ Runbook updates:               $30K–50K

Training:
├─ Formal training:               $20K–50K
└─ Productivity ramp:             $50K–100K (reduced output during learning)
─────────────────────────────────────────────────────────────────
Total Hidden Costs:               $750K–1.7M

Realistic Total Project Budget:
  First year:  $950K–2.25M (technology + migration)
  Ongoing:     $200K–550K/year (technology only)

Your $300K estimate was off by 3–7×.

Why this gets missed

Four root causes.

1. Technology-only budgeting

The problem: focus on licensing costs, ignore labor and services. Vendor quotes show platform pricing. You fill out a spreadsheet with infrastructure costs. It feels comprehensive.

What you miss: who's building the data pipelines? Who's converting 800 Splunk detection rules to SQL? Who's migrating 5 years of dashboards? Who's running the project?

Reality check: if you're migrating from Splunk to a lakehouse architecture, plan for external consulting ($200K–500K for 6–12 months), internal team allocation (1–3 FTE fully dedicated, not "20% of their time"), and project management (an experienced PM at $150K–200K fully loaded, or a $50K–100K consultant PM).

2. Vendor quote reliance

Vendors provide platform costs, not total migration costs. They sell software. Migration is "professional services" (separate quote, often deferred).

Example quote:
  ClickHouse Enterprise:   $100K/year
  Polaris Catalog:         $50K/year
  Tenzir Professional:     $80K/year
  ─────────────────────────────────
  Total:                   $230K/year

Looks great. But this doesn't include who configures ClickHouse for security workloads, who sets up Iceberg tables and compaction jobs, who builds Tenzir OCSF transformation pipelines, or who migrates your 500 detection rules from SPL to SQL.

Vendor response: "We offer professional services at $250–400/hour. Typical engagement: 500–1,000 hours." Translation: add $125–400K to your budget, and that's just one vendor's services.

3. Internal labor assumption

Expecting 100% internal delivery without external expertise. You have a smart security engineering team. "We'll figure it out." Reality: your team knows Splunk SPL, not Spark, Iceberg, and ClickHouse. Learning curve is 3–6 months of reduced productivity. Tracking 11 enterprise migrations (2022–2024): internal-only approach finished in 21 months on average (2.3 FTE, $575K total) with a 40% abandonment rate. Consulting-led: 8 months, 0.7 FTE, $675K, 100% success. Hybrid: 12 months, 1.5 FTE, $625K, 100% success with skill transfer. The "expensive" consulting option often costs less total because faster delivery means less internal opportunity cost.

4. Optimistic timeline

Underestimating duration leads to underestimating costs. "How hard can it be? We'll migrate 100 GB/day, run queries, and cut over."

The reality of security data migration breaks into four phases.

Phase 1 (months 1–3): lakehouse foundation. Provision S3/ADLS/GCS. Deploy Iceberg catalog (Polaris, Glue, Unity Catalog). Ingest one log source (Windows events or CloudTrail). Validate compression (should see 10–20×). Query via SQL (Trino, Dremio, Spark). Outcome: 30 days of logs stored and queryable.

Phase 2 (months 4–6): real-time engine. Deploy query engine (StarRocks, ClickHouse, Trino). Migrate 5 SIEM dashboards to Grafana or Superset. Set up alerting (PagerDuty / Slack). Create materialized views for top queries. Run parallel with SIEM for validation. Outcome: dashboards live, analysts using new system read-only.

Phase 3 (months 7–9): detection migration. Deploy data pipeline (Cribl, Tenzir, or Kafka+Flink). Migrate 500–1,000 detection rules from SPL to SQL. Test each rule against historical data (6–12 months validation). Update runbooks. Outcome: detection rules ported, tested, validated.

Phase 4 (months 10–12): cutover. Analysts use new platform as primary. SIEM relegated to historical data only. Gradual decommission of legacy platform. Final optimization and tuning. Outcome: legacy SIEM off, new platform primary.

Total timeline: 12–15 months. Not the 3–6 months you hoped for.

Worked example

10,000-employee enterprise, 500 GB/day, migrating off Splunk.

Year 1 technology costs

Storage (S3, 10× compression):       $420/month × 12   = $5,040
Polaris Catalog (managed):           $50/month × 12    = $600
Cribl Stream (2× m5.xlarge):         $280/month × 12   = $3,360
StarRocks (4× r5.2xlarge):           $1,520/month × 12 = $18,240
Trino (2× r5.4xlarge, 50% avg):      $760/month × 12   = $9,120
Spark (EMR, 40 hrs/month):           $80/month × 12    = $960
Grafana Cloud (Pro, 10 users):       $300/month × 12   = $3,600
Data transfer (optional):            $150/month × 12   = $1,800
──────────────────────────────────────────────────────────────────
Total Technology:                                       $42,720/year

Year 1 migration costs

External consulting (9 months):
├─ Platform architecture:        200 hrs × $300/hr  = $60,000
├─ Iceberg setup & tuning:       150 hrs × $300/hr  = $45,000
├─ Detection rule migration:     300 hrs × $300/hr  = $90,000
├─ Performance optimization:     100 hrs × $300/hr  = $30,000
└─ Total consulting:                                 $225,000

Internal team allocation (1.5 FTE × 12 months):
├─ Senior security engineer:     1 FTE × $150K     = $150,000
├─ Security analyst (testing):   0.5 FTE × $100K   = $50,000
└─ Total internal:                                   $200,000

Project management (9 months):
├─ PM consultant:                500 hrs × $200/hr = $100,000

Parallel operation (9 months):
├─ Splunk licensing:             $2M/yr ÷ 12 × 9   = $1,500,000

Detection rule migration:
├─ Rule conversion:              800 rules × 2 hrs = $48,000
├─ Testing & validation:         400 hrs × $200/hr = $80,000
├─ Runbook updates:              200 hrs × $150/hr = $30,000
└─ Total rule migration:                             $158,000

Training:
├─ Formal training:                                  $30,000
├─ Productivity ramp:                                $75,000
└─ Total training:                                   $105,000
──────────────────────────────────────────────────────────────────
Total Migration Costs:                               $2,288,000

Year 1 total: $2,330,720

Wait — that's more expensive than Splunk ($2M/year). Yes. Year 1 is always more expensive. You're paying for legacy SIEM (9 months parallel operation: $1.5M), new platform (12 months: $43K), and migration labor ($788K).

Year 2 ongoing: $42,720/year (just technology, no migration costs).

Net savings year 2+: $2M − $43K = $1.96M/year.

Payback period: $788K migration labor ÷ $1.96M annual savings = 4.8 months into Year 2.

CFO framing

How to justify the budget.

Year 1 total cost:        $2.33M (technology + migration + parallel)
Year 2+ ongoing:          $43K/year (technology only)
Legacy SIEM annual cost:  $2M/year
Net savings year 2+:      $1.96M/year
Payback period:           4.8 months (Year 2)
5-year TCO:               $2.33M + ($43K × 4) = $2.5M
Compare to 5-year Splunk: $10M
Total 5-year savings:     $7.5M

CFO reaction: "Okay, Year 1 is a wash, but we save $2M/year after that. Approved."

The template

A comprehensive budget worksheet.

TECHNOLOGY PLATFORM
├─ Platform licensing:                       $_______
├─ Infrastructure (cloud/on-prem):           $_______
├─ Query engine licensing:                   $_______
└─ Subtotal Technology:                      $_______

MIGRATION SERVICES
├─ External consulting (6–12 months):        $_______
├─ Internal team allocation (%FTE × salary): $_______
├─ Project management:                       $_______
└─ Subtotal Migration:                       $_______

PARALLEL OPERATION
├─ Legacy platform licensing (6–12 months):  $_______
├─ Duplicate infrastructure:                 $_______
└─ Subtotal Parallel:                        $_______

CONTENT MIGRATION
├─ Detection rule conversion:                $_______
├─ Dashboard rebuilding:                     $_______
├─ Runbook updates:                          $_______
└─ Subtotal Content:                         $_______

TRAINING & RAMP
├─ Formal training:                          $_______
├─ Productivity reduction (learning curve):  $_______
└─ Subtotal Training:                        $_______

CONTINGENCY (15–20%):                        $_______

TOTAL PROJECT BUDGET:                        $_______

Phased funding

Don't ask for $2.3M upfront.

Instead of one scary number, request phased funding. Each phase delivers something concrete before the next phase funds.

Phase 1 (months 1–3): platform setup plus consulting. Technology setup $50K, consulting kickoff $100K, internal team $50K. Request $200K. Outcome: lakehouse operational, one log source ingested.

Phase 2 (months 4–6): parallel operation plus rule migration. Technology $50K, consulting (rule migration) $100K, internal team $50K, parallel Splunk licensing $500K. Request $700K. Outcome: dashboards live, 50% rules migrated.

Phase 3 (months 7–9): optimization plus cutover prep. Technology $50K, consulting (optimization) $75K, internal team $50K, parallel Splunk $500K. Request $675K. Outcome: all rules migrated, analysts using new platform.

Phase 4 (months 10–12): legacy decommission. Technology $50K, consulting (final tuning) $50K, internal team $50K, parallel Splunk $500K. Request $650K. Outcome: Splunk decommissioned, new platform primary.

Total: $2.225M. The CFO approves $200K initially. You prove value in Phase 1 before requesting Phase 2. Much easier than asking for $2.3M upfront.

Checklist

Before your next budget approval.

Budget includes external consulting (not just technology).
Parallel operation costs accounted for (6–12 months duplicate).
Detection rule migration labor estimated (100–200 hours per 100 rules).
Training costs included (formal plus productivity ramp).
Internal team opportunity cost calculated (not assumed "free").
15–20% contingency reserve allocated.
Phased funding approach established (not single budget approval).

Takeaways

Seven things to remember.

Year 1 is always more expensive. You're paying for migration plus parallel operation plus new platform. Savings start Year 2.
Budget for consulting. 67% of migrations require external expertise. Plan for $200K–500K consulting over 6–12 months.
Detection rule migration is labor-intensive. 800 rules = 1,600 hours = $240K–320K at consultant rates, or 9 months of internal effort.
Parallel operation doubles costs. You'll run legacy SIEM and new platform for 6–12 months. Budget duplicate licensing.
Training isn't optional. Formal training ($20K–50K) plus productivity ramp (3–6 months at 50% efficiency) = $70K–150K.
15–20% contingency. Migrations always hit unexpected complexity. Reserve for scope changes.
Phased funding is easier. Instead of $2M upfront, request $200K–700K per quarter. Prove value before next phase.