Project 3 · Research territory
MLOps-enabled model threat hunting.
Models surface anomalies and prioritize hunts; MLOps manages the model lifecycle (training, drift detection, retraining, evaluation) with the same discipline DetectFlow applies to detection content. The point isn't "AI in the SOC." It's treating models as production artifacts with the discipline that detection content has earned, in an industry that has earned a healthy skepticism about both. This page names what isn't yet known more directly than the other two.
The framing
Why this is research territory rather than a productized service.
The current AI-in-the-SOC vendor wave overwhelmingly fails the empirical-skepticism test the thesis runs every other claim through. "Autonomous SOC analyst." "AI-driven triage." "Self-healing detection." The marketing pattern is the AI-augmented-versus-AI-native distinction captured on the research surface: chatbots layered on top of the same BI-era infrastructure, marketed as if the chatbot changed the data platform underneath. The infrastructure doesn't carry agent-scale workloads cleanly yet. The model accuracy claims don't survive production validation. The "AI talent reduction" promises, the ones advertising 60–80% headcount cuts, shrink materially once the human-oversight workload is counted back in — a field read rather than a measured claim, since post-implementation staffing audits are rare and the precise figures are almost all vendor-sponsored. The category isn't ready, and pretending otherwise would compromise every other recommendation I make.
At the same time, the underlying argument has merit. Models do surface anomalies that rules don't catch. Models do prioritize hunts in ways that human analysts can't scale to. The hunting program at a mature SOC has analyst time as the binding constraint, and the model-prioritized hunt is a productivity lever when it works. The "when it works" is carrying a lot of weight in that sentence.
My position: this is the right direction; the field hasn't yet produced the evidence to operationalize it; I track the territory as research and publish what holds up rather than selling a service, and since I'm not taking engagements while I work full-time, that's where it stays for now. Any later move toward a service would wait on an evidence threshold rather than a date.
The discipline
Models as production artifacts. The lifecycle is the work.
The foundation of "MLOps for hunting" isn't novel. MLOps as a discipline is mature in adjacent fields. Reproducible training pipelines. Lineage-traceable features. Versioned model artifacts. Drift detection against production traffic. Retraining on a defined cadence with documented evaluation. Rollback through version control. The disciplines exist; the application to security-data hunting is what's underspecified.
Training data lineage.
Models are no better than the training data they're built on. Hunting models trained on historical telemetry need to know exactly which telemetry, in which schema version, with which OCSF normalization applied. The Iceberg lakehouse gives this for free. Time-travel queries reconstruct the exact data state a model was trained on, and lineage tracking back through the pipeline shows which sources fed it. Without that view, model retraining drifts silently across schema changes, and the model's behavior in production diverges from its training-time evaluation in ways nobody can debug.
Drift detection.
Production telemetry doesn't stay shaped like the training set. New cloud services land. New attack patterns surface. EDR vendors update their event schemas. Without active drift detection, the model's accuracy degrades across all of these without anyone noticing. The MLOps discipline runs continuous statistical comparison between the production telemetry distribution and the training distribution, with alerting when the divergence crosses defined thresholds. The alert says "this model is now operating outside its training envelope; treat its output with caution until retraining lands."
Evaluation discipline.
Every model carries a defined evaluation suite. Known-bad samples it should flag, known-good samples it should not flag, and a measurable false-positive rate against unlabeled production data. Promotion to production requires the evaluation suite to pass; demotion is automatic if production performance drifts below threshold. The same discipline DetectFlow applies to detection content, applied to model output. The mechanism that prevents "the model worked in the notebook; we have no idea what it's doing in production."
Why most current attempts fail
Three patterns that recur across the AI-in-the-SOC failure cases.
Notebook research, not production engineering.
A data scientist trains a model in a Jupyter notebook against a curated subset of historical telemetry. The model performs well on the curated subset. It gets shipped to production with no MLOps infrastructure underneath. No drift detection, no retraining cadence, no evaluation suite. Six months later the model's output is noisy in ways nobody can characterize, the analysts have learned to ignore it, and the program admits the experiment failed. The model wasn't bad; the operationalization wasn't there.
Models that surface anomalies the analysts can't act on.
Anomaly detection without context is alert fatigue dressed up. If the model flags a behavioral pattern that's statistically rare but the analyst can't determine within reason whether it's malicious, the model has produced workload rather than insight. The mature pattern: models that flag anomalies with the supporting context graph attached, so the analyst can triage in seconds rather than start a five-tab investigation. The supporting context comes from the data-health foundation: entity resolution, asset ownership, recent activity. Without that foundation, the model output is noise.
Treating model output as authoritative.
The "autonomous SOC" framing assumes model output can drive action without human review. The empirical evidence, including the AI-asymmetry hypothesis tracked on the research page, does not support that framing for security-critical decisions in 2026. Models hallucinate. In controlled simulations, models have taken harmful actions when their goals were put under pressure (Anthropic's 2025 agentic-misalignment study, in a lab setting, not a production incident). Models encode biases from training data that are invisible until they fire on production data and produce false positives at category-level rates. So I treat human oversight of security-critical decisions as architectural rather than transitional, though that position has a serious counter-argument I should name. Phil Venables argues in "Autonomic Defense" (September 2026) that containment shouldn't wait on analyst approval and that the right goal is human-on-the-loop rather than human-in-the-loop oversight. It's an argument I take seriously and haven't resolved, and because it rests on reasoning and not data on either side, I carry it as a live contradiction and not a settled point. Programs that skip oversight entirely produce expensive incidents that get retroactively rebadged as "model tuning issues" rather than recognized as the architecture decision they actually were.
Securing the models themselves
A hunting model trains on data the attacker helps write.
Most MLOps practice assumes the training data is honest, and security telemetry isn't, because an attacker inside the environment produces some of it. A model retrained on recent telemetry can be taught, slowly, that the attacker's activity is normal, which is training-data poisoning delivered through the logs rather than through a dataset someone downloaded, and a model that scores behaviour can be probed and evaded by an attacker who shapes activity to sit just under its threshold. MITRE ATLAS catalogues both, as Training Data Poisoning and Evade AI Model, and the ATLAS Wall shows how thinly the framework's mitigations cover the attacks people are actually seeing.
So the lifecycle discipline above needs a security half. Training windows should be pinned to snapshots that can be audited, which the lakehouse's time travel already allows, drift alerts should be read as possible manipulation as well as natural change, and the evaluation suite should include planted attacker behaviour that the model must keep flagging after every retrain. None of this is measured yet in the lab, so treat it as the design requirement I would hold a hunting model to, not a tested result.
When the work becomes scoped
The conditions that have to be in place first.
I track MLOps for hunting against one prerequisite and four research triggers. The prerequisite is non-negotiable, and the four triggers are the evidence threshold the field has to clear before I would recommend model-driven hunting as a production practice, which is a research bar and not a commercial plan, since I'm not taking engagements.
The prerequisite, non-negotiable.
The data-health foundation has to be in place. Models trained on telemetry of unknown completeness, freshness, and schema conformance produce output of unknown reliability. The data-health gate on the thesis page is the precondition; without it, the model output inherits the foundation's gaps without anyone seeing them. Without the foundation, every other trigger below is moot.
The four research triggers, all of which have to be true.
- Two reference architectures in the SDW catalog demonstrate ML-driven detection or hunting at production scale. The RisingWave + Palo Alto Networks (Cortex XSIAM) reference is half of this already; streaming SQL with sub-100ms model inference is exactly the production foundation MLOps-hunting needs. The second half is missing: an MLOps reference for model serving and lifecycle at security-data scale, with measured drift, retraining cadence, and false-positive rate over time. Until the catalog carries both, "MLOps-hunting" is a thesis without a working precedent.
- At least three documented programs running production hunts at 50+ per quarter on a verified data foundation."Hunt" here is scoped. A named investigation that goes through the DetectFlow pipeline (defined hypothesis, telemetry, evaluation criteria, dispositioning decision), not a one-off notebook query. Programs below 50/quarter don't need model-driven prioritization; the analysts already know what to chase. The leverage shows up only at the volume where analyst time is the binding constraint and the next-investigation decision is meaningful triage.
- MLOps tooling has stabilized to a small candidate set, with at least one ranked in the matrix at Tier A or B evidence. Today's candidate set (MLflow, Kubeflow, Metaflow, Seldon, Modal, Replicate) is too broad to recommend against. The matrix-scoring discipline applied to query engines and catalogs has to extend cleanly to the MLOps tier before a recommendation on tool choices can claim methodology rigor.
- The AI-native-versus-augmented research moves from Tier C to Tier B with at least one production reference. Currently the research surface carries Tier C evidence: vendor announcements without production validation. When that body of work gets backed by a documented production deployment at security scale, the empirical-skepticism threshold the thesis runs every claim through has been cleared. Until then, recommending model-driven hunting would mean "trust the model on data we haven't validated", exactly the failure mode the thesis is meant to block.
For shops where Foundation isn't yet in place, the right work is the foundation first, then DetectFlow on the data platform. MLOps-hunting waits until all four triggers land, however long that takes. The timeline is the consequence of the evidence threshold, not the other way around, and recommending this stage on a calendar would compromise every other recommendation the practice makes.
The current track
Research now, and a recommendation when the evidence catches up.
The current posture is to track the territory as research and publish what's defensible, with no commercial service attached. The thought-leadership surface lives across three artifacts: the AI-native-versus-augmented research tracking the infrastructure rebuild signals, the reinvented-Kimball essay tracking the AI-generated-infrastructure pattern, and the long-form treatment in the forthcoming MOAR book where one chapter covers the model-as-production-artifact discipline applied to security data.
The lab roadmap covers the testable parts. OCSF mapping accuracy now has first-party numbers, a mapping-oracle bench where putting the schema in a small model's context cut invented field paths from 0.99 to 0.69 and a silent-error bench on natural-language queries over OCSF, both summarized on the AI-native research page. Testing Tenzir's MCP claims for AI-generated parsers and measuring production query patterns under agent workloads hadn't been run as of November 2026, and each lands here when it is ready. When the lab work produces reproducible numbers, the thesis updates.
If your program is running into the constraints this project addresses (analyst time as the limit, a hunting backlog dominated by prioritization decisions, a willingness to invest in model lifecycle discipline), the research and essays linked above are the place to start, and I'm glad to compare notes by email. I'm not taking engagements while I work full-time, and selling this as a finished service before the evidence exists is the failure mode the empirical-skepticism method is meant to prevent.
The direction is right, and the evidence isn't in yet.
The foundation gates everything; MOAR is the data platform; DetectFlow is the discipline that makes production model deployment operationally tractable. MLOps-hunting is the research track that follows them.