Technology deep-dive
MLOps tools for threat hunters.
A mature threat hunt is a small data-science experiment in disguise. The hunter forms a hypothesis, queries logs, measures precision and recall, documents what worked, and shares it with the team. That is the scientific method applied to security telemetry. The tooling the rest of the data world uses for this work (notebooks, experiment trackers, data versioning, quality tests, feature stores) has names that sound foreign to most SOC analysts, even though the work underneath the names is work those analysts already do.
Reading time: about 28 minutes. Evidence tier: B overall (SANS Hunting Maturity Model framework, Splunk PEAK framework, vendor documentation for MLflow, Kubeflow, Feast, Kedro, ClearML, and Weights & Biases), dropping to C–D for the security-specific feature-store patterns near the end, flagged explicitly where it matters. The HMM-to-tool mapping is my synthesis, not a published framework. I have not independently benchmarked any of these tools on production OCSF data, and I could not find a single publicly documented production deployment of a feature store inside a SOC in early 2026.

The reframing
Threat hunting is data science with a different vocabulary.
A typical Tuesday for a hunter at the HMM2-to-HMM3 transition looks like this. You open the SIEM and form a hypothesis, say that "adversaries are using compromised cloud credentials to disable CloudTrail logging," and then you write a query that scans for API calls modifying logging configuration and filter out the noise, so that forty-seven raw results walk down to twelve true positives. You write the finding up, share it with the team, and maybe codify the best variant as a detection rule.
Every step of that workflow has a data-science name, because the hypothesis is a hypothesis, the query is exploratory data analysis, the forty-seven-to-twelve precision number is model validation, the write-up is reproducible research, and the detection rule is a deployed model. The reason most hunters don't call it that is partly cultural ("data scientist" reads as a different role with different credentials) and partly because the tools the data-science world uses for this work haven't been packaged for SOC teams.
The Splunk PEAK framework (Prepare, Execute, Act, Knowledge) and the SANS Hunting Maturity Model both describe this same workflow shape. PEAK is the per-hunt cycle. HMM is the organizational maturity ladder: HMM0 (no hunting) through HMM4 (leading with automation). I cover the PEAK mechanics in PEAK plus lakehouse hunting, and the HMM-to-detection-engineering progression in detection maturity. This piece is about the tools that sit underneath both.
One thing I want to set up front: none of what follows requires a PhD, a neural network, or years of Python. If you can write a SQL query and read a Markdown document, you have the prerequisites. The tools amplify the hunting skills you already have. They don't replace them, and they don't turn a hunt into a research paper unless you want it to.
I came to this reframing the slow way, by being the person hunters came to. For years, threat hunters kept landing at my desk for query help. They could hunt; what they couldn't always do was the data part, the work of expressing the question against the logs they had, making it run across the depth and duration the hunt actually needed, and shaping the results into something they could reason over. The good ones had taught themselves that craft because the job demanded it, and what they were doing underneath was querying at scale, building features, and hunting for patterns across breadth and time, which is data science whether or not anyone in the room would have used the words. That is the reframing from the inside, and it cuts two ways. Hire actual data scientists into hunting, and give the hunters you already have data-science-grade instruments instead of leaving them to improvise against a SIEM search bar.
The five categories
What MLOps tooling is, in hunter terms.
MLOps (machine learning operations) is the discipline of running data-science workflows as production systems rather than one-off experiments. The category was built for teams shipping ML models, but the underlying tools solve four problems that any mature hunter recognizes: reproducibility, collaboration, versioning, and quality validation.
Five tool categories cover most of what a HMM2-to-HMM4 program needs:
- Interactive notebooks. Jupyter and JupyterLab. The work surface where the hunt lives.
- Experiment tracking. MLflow, ClearML, Weights & Biases. The system of record for which hunt variant beat which.
- Data versioning. DVC (Data Version Control). The way you reproduce a hunt result six months later against the same data.
- Data quality testing. Great Expectations. The early-warning system for schema drift in the log sources you depend on.
- Feature stores. Feast, with Kedro for pipeline orchestration alongside it. Centralized, reusable computations across detection models. An HMM4 concern, not an HMM2 one, and the one piece of this list with essentially no public security track record yet.
A sixth category, orchestration platforms like Kubeflow, sits adjacent to all of these. Kubeflow runs ML pipelines on Kubernetes; it's the production-deployment piece, not the hunt-development piece. I mention it because security teams already running Kubernetes platforms sometimes ask about it. For HMM2-to-HMM3 work, it's overkill. For HMM4 production detection pipelines, it earns its keep, though the operational complexity is real.
Notebooks
Jupyter as the hunt surface.
A Jupyter notebook is a document that mixes code cells (SQL, Python, shell), Markdown narrative, and visualizations in one file. You run the cells top to bottom and the document carries the working state: query, dataframe, chart, paragraph explaining what you saw. Save the file, commit it to Git, and the hunt becomes a reproducible artifact rather than a transient SIEM session.
The contrast with a SIEM console is most of the argument, and the cleanest way to feel it is to live through the failure. It's Tuesday, I open the SIEM console, paste a hunting query I spent an hour crafting yesterday, and run it against two weeks of CloudTrail logs. The query completes. Zero results. I stare at the screen. Was the query wrong, or is the environment actually clean? I cannot remember the exact parameters from yesterday: was it fourteen days or seven, and did I filter out service accounts? A week later my manager asks me to re-run that hunt to validate the results, and by then the data has changed (new logs arrived, old logs aged out), the exact query logic is gone, and my notes are scattered across three Slack messages and a Word doc. That is the reproducibility problem in threat hunting, and it is the same problem data scientists faced ten years ago, so the data-science community already built the tools to solve it. They are free, open source, and well-documented, but most threat hunters haven't picked them up yet.
A Jupyter notebook checked into Git closes that gap, because it has the query, the filter, the chart code, and the narrative pinned together, so re-running it tells you whether the result still holds. What a notebook captures that a SIEM console doesn't, line by line:
- Query logic. SQL saved as a code cell, not lost when the session expires.
- Analysis code. The pandas filter that removed the compliance-scanner false positives, with a comment explaining why.
- Visualizations. Charts rendered inline at the point in the workflow where they matter, not screenshotted into a separate slide deck.
- Narrative. Markdown cells explain why each step happened, not just what it produced, which is the difference between an artifact a teammate can use and one they have to reverse-engineer.
- Reproducibility. Six months from now, another analyst opens the notebook, runs every cell, and gets the same answer, assuming the underlying data is preserved (which is what the dataset-versioning section below is for).
This pattern is well established outside the security world and increasingly inside it: Microsoft Sentinel ships Jupyter integration, Secureworks Taegis uses notebooks for investigation, and SpecterOps publishes notebook-based hunt content. The notebook doesn't replace the detection rule that eventually runs in production; it's the document that explains how the rule was derived, so when the rule fires six months later and someone asks "why does this exist?", the notebook is where they find the answer.
Time to first productive notebook is in the 2-to-8-hour range for an analyst who already writes SQL and reads Python comfortably. That estimate is drawn from training-workshop reports rather than a controlled study, so treat it as directional. Analysts who haven't worked with the pandas DataFrame model (the table-like object that holds query results in memory) tend to land at the upper end of that range; the shift from "SIEM result grid" to "DataFrame you can slice programmatically" is the one conceptual hurdle. Jupyter is free, open source, and runs on a laptop or a shared server. JupyterHub adds multi-tenant hosting for a team. For most HMM2-to-HMM3 teams, Jupyter is the only tool on this list they should adopt in the first six months.
A worked example
What a CloudTrail hunt notebook looks like end to end.
The abstract description only goes so far, so here is the structure I actually use for a hunt campaign that starts with the hypothesis "adversaries are using compromised cloud credentials to disable logging and evade detection." Cells alternate between Markdown for narrative and code for execution, and the first Markdown cell sets the frame:
# Hunt Campaign: Suspicious Cloud API Activity
## Hypothesis
Adversaries are using compromised cloud credentials to disable
logging and evade detection.
## Dataset
- Source: CloudTrail logs (S3 / Iceberg)
- Timeframe: 2026-01-01 to 2026-01-14 (14 days)
- Volume: ~2.3 TB, 847M events
- Dataset version: cloudtrail_baseline_v2.3 (frozen for reproducibility)The first query asks the broad question. Across the data lake, who has been calling the APIs that suppress or remove logging?
SELECT
eventTime,
userIdentity.principalId,
eventName,
sourceIPAddress,
userAgent
FROM iceberg_scan('s3://security-lake/cloudtrail/')
WHERE
eventName IN ('StopLogging', 'DeleteTrail', 'PutEventSelectors')
AND eventTime BETWEEN '2026-01-01' AND '2026-01-14'
ORDER BY eventTime DESC;Forty-seven results come back, which is too many to triage without filtering, so the next code cell is pandas narrowing the candidate set against two known patterns: an automated compliance scanner and routine business-hours activity. The cell is intentionally a separate step from the SQL, because the filters encode operational knowledge that belongs in narrative rather than buried in a WHERE clause.
import pandas as pd
df = pd.read_sql(query_1, engine)
# Compliance scanner — known automated false positive
df = df[~df['userAgent'].str.contains('ComplianceScanner')]
# Business hours (9 AM to 5 PM EST) — also dominated by legitimate use
df = df[(df['eventTime'].dt.hour < 9) | (df['eventTime'].dt.hour >= 17)]
print(f"Remaining candidates: {len(df)}")Twelve candidates remain. A Markdown cell records what the analyst learned: the true-positive pattern is off-hours activity plus an unfamiliar geographic source plus the specific logging-disable APIs, and the false-positive pattern is the compliance scanner during scheduled maintenance windows. A second cell prototypes the detection rule that follows from those findings, the same logic, now expressed as production SQL with the filters inline. The structure is what matters, running from hypothesis to dataset to broad query to filter to finding to prototype detection, because six months later a different analyst can open this file and see what I did and why, which makes the notebook the audit trail for the eventual detection rule. The same workflow maps cleanly onto the five-cell shape I use for any cloud-control-plane hunt: SQL against the Iceberg table returning a dataframe, a Python filter stripping known service-role principals, a Matplotlib chart of API-call frequency over the investigation window, a Markdown paragraph describing the surviving false-positive pattern, and the candidate detection rule written out as SQL ready to lift into the SIEM or detection-as-code repo.
Connecting to the lake
DuckDB is the cheapest path from a notebook to Iceberg.
A notebook is only useful if it can reach the data, and the lowest-friction option for a lakehouse-backed SOC is DuckDB, which you install as a Python package, point at S3, and use to query Iceberg tables directly from a notebook cell, with no server, no driver-installation argument with the SRE team, and no JDBC connection string. I cover DuckDB for threat hunting in a separate post, including its limits.
import duckdb
con = duckdb.connect()
df = con.sql("""
SELECT * FROM iceberg_scan('s3://security-lake/cloudtrail/')
WHERE event_time > '2026-01-01'
""").df()
print(df.head())For larger workloads, the notebook can connect to a shared engine instead, whether that's StarRocks or ClickHouse via SQLAlchemy, Trino via its Python client, or PySpark for very large jobs, and the notebook code stays structurally the same because only the connection target changes. That portability is one of the reasons I push analysts toward notebooks early, since the same code runs against a local DuckDB cache for development and a production engine for the real hunt. The integration surface is wide: DuckDB for local Iceberg queries, SQLAlchemy for ClickHouse or StarRocks, REST calls for catalogs, and embed cells for Grafana.
Getting started takes about fifteen minutes if you already have Python, since you install JupyterLab and DuckDB with pip, launch jupyter lab, open a new notebook, and run a SELECT, and if you know SQL and basic Python you'll be productive in a working day. Practitioner training data suggests two to four hours for the first useful hunt, with longer for analysts new to pandas, and the interface is intentionally simple, so that Shift+Enter executes a cell, Markdown cells render inline, and charts appear where the code runs. The Arrow plus ADBC path is the columnar-native version of this connection, and removes the row-to-column serialization tax on result sets.
Experiment tracking
MLflow, ClearML, and Weights & Biases.
Experiment tracking is the system of record for "I tried ten variants of this detection rule. Which one wins on precision and recall, and what exactly was variant seven?" Spreadsheets are the default tool, and they work for two or three variants but break by variant ten, because no one remembers to update the spreadsheet consistently and the artifacts (the query text, the chart, the dataset snapshot) live somewhere else. Jupyter and MLflow solve adjacent problems and they're easy to confuse: Jupyter is for the single-hunt artifact, while MLflow is for the comparative record, so when I run the same hunt with ten different filter combinations MLflow gives me a sortable table to see which variant produced the best precision-recall balance. The order matters, because MLflow assumes you already have a script or notebook that runs end-to-end, so if the hunt itself is still living in the SIEM console there's nothing for MLflow to track. Start with Jupyter and add MLflow once you find yourself running the same hunt with three or four variants and losing track of which one performed best.
An experiment tracker records three things per run: parameters (what you fed the experiment: dataset, timeframe, threshold, query text), metrics (what came out: precision, recall, true-positive count, false-positive count), and artifacts (the files: the detection rule, the notebook, the chart, the report). The conceptual model carries over cleanly to hunting, because a run (one execution logging parameters, metrics, and artifacts) maps to a hunt variant, and an experiment maps to a hunt campaign, while the UI lets you compare runs side by side and click through to the exact configuration that produced any given result.
The three credible options in early 2026:
- MLflow. Open source, originally from Databricks. The most widely adopted option. Self-hostable on a small VM with a PostgreSQL backend and S3-compatible artifact storage. The UI is functional rather than polished. This is what I would default to for a security team standing up experiment tracking for the first time, and what most public documentation in the security-data-science space assumes.
- ClearML. Open source, with a hosted SaaS tier. Stronger out-of-the-box experience than MLflow, including a more polished UI and built-in pipeline orchestration. Smaller community, which matters if you're going to lean on Stack Overflow and blog posts to unblock yourself.
- Weights & Biases. Commercial SaaS, with a generous free tier for individual use and paid plans for teams. The UI is the best of the three by a useful margin, and the collaboration features (commenting on runs, sharing reports) are designed for team workflows. The trade-off is that you're sending experiment metadata to a vendor cloud, which matters more in security than in most domains.
Instrumenting a run is a few extra lines per notebook: start a run, log parameters, log metrics, log artifacts, end the run. The case becomes obvious the third or fourth time you find yourself tracking ten variants of a PowerShell detection rule in a spreadsheet, where variant one alerts on any Invoke-Expression, variant two adds a signed-script filter, variant three excludes known admin users, variant four adds an entropy score for obfuscated commands, and variant five adds parent-process checks. Each variant has a true-positive count, a false-positive count, a precision number, and an estimated recall. Without MLflow you track that in a Google Sheet and lose it the moment someone closes the tab; with MLflow the notebook itself records the run as a side effect of executing:
import mlflow
with mlflow.start_run(run_name="powershell_hunt_v3"):
# Inputs — what configuration was this run?
mlflow.log_param("dataset", "windows_events_v2.1")
mlflow.log_param("timeframe_days", 14)
mlflow.log_param("detection_logic",
"invoke_expression + signed_filter + admin_filter")
# Execute the hunt
results = run_detection_query()
# Outputs — how effective was it?
mlflow.log_metric("true_positives", 12)
mlflow.log_metric("false_positives", 47)
mlflow.log_metric("precision", 0.20)
mlflow.log_metric("recall_estimated", 0.85)
# Files — what did this run produce?
mlflow.log_artifact("detection_rule.sql")
mlflow.log_artifact("results_chart.png")After ten variants, you open the MLflow UI and sort by precision, and the winning approach is the row at the top, so you click it, download the attached detection_rule.sql, and you have the production artifact. The UI surfaces four things worth knowing about: the run list (every variant with parameters and metrics inline), the comparison view (select two or more runs and see them stacked), the charts panel (metric trajectories across runs, so you can see if precision is rising with each iteration), and the artifacts panel (download any file attached to any run). The comparison view is the part that's hard to replicate in a spreadsheet, because it lets you see two runs that differ only in the admin filter with their precision deltas plotted on the same chart. A typical comparison table from a PowerShell-detection campaign:
| Run | Detection logic | TP | FP | Precision |
|---|---|---|---|---|
| v1 | invoke_expression | 85 | 412 | 17% |
| v2 | + signed_filter | 73 | 189 | 28% |
| v7 | + signed_filter + entropy_scoring | 68 | 6 | 92% |
The jump from v1 (17%) to v7 (92%) is the story this table tells. Without MLflow, that story tends to get reconstructed in retrospect from chat-log fragments, after someone asks "why does the production rule include the entropy check?" With MLflow, the answer is two clicks away. A few honest hedges belong here: the precision and recall numbers above are illustrative, and real recall is hard to compute in hunting because you rarely know the full denominator (every true positive in the environment), so what gets logged is usually an "estimated recall" against a labelled subset, and that estimate may carry a wider error bar than the precision figure. MLflow doesn't fix that, but it does ensure that whatever number you computed, the inputs and the computation are recoverable.
The honest caveat on adoption timing: experiment tracking pays off when you have ten or more variants to compare. Below that, a Markdown table in the notebook is sufficient, so I would not adopt MLflow on day one but rather after the first ten-to-twenty Jupyter hunts, when the cost of forgetting which variant you ran starts to bite.
Deploying MLflow
Local first, shared backend later, and treated as production.
For a first run, MLflow needs almost nothing. Install with pip, start the UI on localhost, point your notebook at it, and log a test run.
pip install mlflow
mlflow ui # serves http://localhost:5000
# In a notebook
import mlflow
mlflow.set_tracking_uri("http://localhost:5000")
with mlflow.start_run(run_name="test_hunt"):
mlflow.log_param("test_param", "hello")
mlflow.log_metric("test_metric", 42)That's enough to learn the concepts. For a team, you'll want a shared deployment so multiple analysts log runs to the same place, and the standard pattern is a PostgreSQL backend for the run metadata plus an S3 bucket for artifacts, both of which a security team usually already has provisioned. The PostgreSQL backend can co-locate with your catalog's database to reduce operational surface, and the artifact root can point at the same object storage that backs your Iceberg tables.
mlflow server \
--backend-store-uri postgresql://mlflow:password@localhost/mlflow \
--default-artifact-root s3://security-lake/mlflow/artifacts/Production hosting cost is modest. A small PostgreSQL instance plus modest artifact storage runs in the range of fifty to two hundred dollars a month at typical SOC scale, based on AWS list pricing for a db.t3.medium RDS plus a few hundred gigabytes of S3. Managed offerings exist (Databricks MLflow, AWS SageMaker Experiments) if you'd rather not run it yourself, and they cost two to five times the self-hosted figure. I lean self-hosted for security teams because the artifacts are the detection rules themselves, and keeping them inside your existing S3 footprint avoids a separate data-residency conversation.
There is a security catch worth stating plainly, since the whole premise is that you will run this yourself. The MLflow tracking server has a documented remote-attack-surface record, and the things it holds (your detection rules, your dataset pointers, sometimes the warehouse credentials a run needs) are exactly what an attacker would want. The 2025–2026 advisories are not theoretical. CVE-2025-11201 is an unauthenticated directory traversal that reaches remote code execution through an unvalidated source path, and CVE-2026-2614 lets an unauthenticated request read arbitrary files by smuggling a file:// source past a validation bypass. CVE-2026-2635 shipped exploitable default credentials in basic_auth.ini that hand over admin to whoever finds the port before you rotate them (fixed ahead of 3.8.0rc0), while CVE-2026-2734 lets any authenticated low-privilege tenant read model metadata across tenant boundaries it should not cross. A service that stores the logic your SOC runs on should not be the easiest box on the network.
None of that is a reason to avoid MLflow; it is a reason to run it like the production service it is. The discipline that answers this has a name, SecMLOps: keep the tracking server off the public internet, bound to localhost or behind the same VPN the rest of your SOC tooling already sits behind, and rotate the default credentials on day one, ideally replacing basic auth with your existing SSO. Constrain artifact source paths to an allowlist and reject file:// and absolute filesystem paths, so a traversal has nowhere to escape to. And because a model artifact is executable code, prefer pickle-free serialization like ONNX or Safetensors over raw pickle, joblib, or torch weights, which run arbitrary code on load and turn a poisoned model file into an entry point. The principle is the one this site keeps returning to: the tooling that holds your security logic is security-relevant in its own right, and it earns the same scanning, patching, and network hygiene you would give anything else in production.
Patterns that fail
Four ways notebook adoption tends to go sideways.
I've watched a number of SOCs try to roll out notebook-based hunting and stall. The failure modes cluster into a handful of patterns that are worth naming, because they're avoidable once you know to look for them.
The shared-notebook-server trap. Some teams stand up a multi-user JupyterHub deployment as the first step, because IT culture defaults to "shared infrastructure." This usually slows adoption rather than accelerating it. The friction of arguing with the SRE team about authentication, the SSO integration, the storage quota, and the kernel-environment management consumes the window where analysts would otherwise be writing their first hunt notebook. Start with analysts running JupyterLab locally on their laptops against a development data slice. Move to shared infrastructure only once people are demanding it because the workloads outgrew a laptop.
Notebooks as throwaway scratch. The opposite failure mode is treating notebooks as disposable. An analyst opens a fresh notebook, writes a hunt, finds something, closes the laptop. The notebook never enters version control. Six months later it's gone, the detection rule it produced is in production with no audit trail, and the next analyst has to re-derive the same logic from scratch. The fix is procedural: every hunt notebook gets committed to a Git repository before the hunt is considered finished. The repository can be the same one that holds detection rules. Treat the notebook as a first-class deliverable, not a side artifact.
Hidden state. Notebooks let cells execute out of order, which means the on-screen state can diverge from the state you'd get by running the cells top to bottom. Six months later, a teammate opens the file, runs every cell sequentially, and gets a different result. The discipline is boring but effective: before declaring a notebook "done," restart the kernel and run all cells in order. If it doesn't reproduce, fix it before checking it in. JupyterLab has a single menu command for this, "Restart Kernel and Run All Cells."
MLflow as another spreadsheet. MLflow's value compounds when teams agree on metric names and parameter names. If one analyst logs precision and another logs prec and a third logs positive_predictive_value, the sortable table the UI is supposed to produce becomes three sortable tables that can't be compared. A short style guide (roughly a one-pager listing the canonical names for true positives, false positives, precision, estimated recall, dataset version, timeframe) pays for itself within the first month. It's a small investment that prevents a slow drift into MLflow-as-spreadsheet.
None of these failures kill the adoption outright, but they produce a slow-motion stall where the tools are nominally in use while the team isn't realizing the benefits. The signal that a team has crossed into actual HMM3 work is concrete, because it shows up when a new analyst joins and can answer "why does this detection exist?" by reading the notebook that produced it rather than asking the analyst who wrote it, and until you can pass that test the work isn't done.
Data versioning
DVC and the "same data six months later" problem.
MLflow tracks the run, but it does not freeze the dataset the run ran against, so if the underlying CloudTrail data changes between Tuesday and the following Tuesday (new events arrive, old events age out of hot storage), re-executing the run will produce different numbers, and MLflow will dutifully log the new numbers without telling you the data drifted. Closing that gap is what data versioning is for. DVC is the simplest tool on this list to explain and the hardest to convince a team they need. The pitch: it is Git for data. You version multi-terabyte datasets using small metadata files in Git, with the actual data living in object storage (S3, Azure Blob, GCS). A dvc checkout against a tag pulls the exact bytes present when that tag was created, which gives you time travel without copying petabytes around.
Why a hunter cares: detection-rule validation is fragile. You tune a rule against ninety days of CloudTrail data, measure 86% precision, and ship it. Three months later, you tune it again and want to know whether the change moved precision from 86% to 92% or whether it stayed flat and the apparent gain came from log volume drift. Without a frozen baseline dataset, you can't tell. With DVC, you check out the same ninety-day window you used originally and re-run the validation against bit-identical data.
The Iceberg-specific wrinkle: Iceberg already has snapshot IDs and time-travel queries. For teams already on Iceberg, the DVC layer mostly captures the snapshot ID as metadata rather than copying the data. Iceberg vs Delta Lake covers the table-format mechanics. The honest position is that for an Iceberg-native shop, DVC may be redundant; Iceberg snapshots plus a tagging convention can do the job. DVC earns its keep when you're moving filtered subsets out of Iceberg into training datasets for ML-based detection, or when you need to track non-Iceberg artifacts (a CSV of known-malicious IPs, a parquet file of labeled training data) with the same workflow as the code.
Learning curve is four-to-six hours for someone who already understands Git. The harder part is operational: provisioning the remote storage, agreeing on a tagging convention, and getting the team to actually create tags before running validation experiments. That last item is the one that determines whether DVC pays off or sits unused.
Data quality
Great Expectations and the silent-detection-failure problem.
Great Expectations is a Python framework for writing data-quality tests ("expectations") that run automatically against your tables and alert when they fail. An expectation is a declarative assertion: this column must exist, this column must not be null more than 5% of the time, this column's values must fall in this range, this table must have rows from the last hour.
For a hunter, the failure mode this addresses is the silent miss. Your detection rule queries a CloudTrail field called userIdentity.principalId. AWS renames the field, or your pipeline normalizes it differently after an OCSF mapping update, or the source schema rolls over. The query keeps running and returns zero results, and zero results looks identical to "no adversary activity," so the adversary operates undetected for as long as it takes someone to notice the change.
An expectation suite for CloudTrail might assert:
- Required columns are present, including the OCSF-normalized identity fields.
- The
event_timecolumn has rows within the last 60 minutes. - The
regioncolumn's distinct values stay inside the expected set. - The
eventNamecolumn's null rate stays under 1%.
Great Expectations runs the suite on a schedule. When something fails, the failure routes to Slack or PagerDuty before the detection rule misses anything. The framework also generates a data-quality report you can hand to the source owner ("your CloudTrail feed broke at 03:14, and here's the specific assertion that caught it") which tends to shorten the time-to-fix.
The cost is real: writing expectations is its own skill, and a bad expectation suite generates alert noise no more useful than a noisy SIEM. The Great Expectations CLI has a profiler that generates a starting suite from a sample of your data, which is a reasonable way to start, and from there you prune the assertions that don't matter for your actual hunts. Six-to-ten hours to a working first suite is typical.
Feature stores
When you actually need a feature store.
I want to put the recommendation first, because the rest of this section covers tooling most of you don't need yet. SANS research puts roughly 8% of organizations at HMM4, "Leading" maturity, where systematic automation, ML pipelines, and reproducible workflows are the working pattern, and the other 92% are at HMM2 (procedural hunting on community playbooks) or HMM3 (innovative analytics on home-grown queries). At HMM2 and HMM3 the right toolchain is the one above, Jupyter for hunt prototyping and MLflow for experiment tracking, because a feature store at that stage is mostly overhead you pay for before there's enough model count to give it something to do. Most security teams I talk to are running zero supervised ML models in production, not the dozens that would make the feature-store problem bite.
A feature store is a system that computes derived attributes ("features") from raw data once, stores them, and serves them consistently to every model that needs them. Three pieces of vocabulary do most of the work. A feature is a derived numeric or categorical signal computed from raw data: failed_logins_24h is a feature, login_geo_diversity_score is a feature, rare_process_count is a feature; the raw authentication log line is not a feature, but the count of failures in a 24-hour window for a given user is. A feature store centralizes the definition, computation, and serving of those features, so there's one definition served consistently to every model. The store has two faces: the offline store holds historical features for training (typically Iceberg tables, Snowflake, BigQuery), and the online store holds the latest precomputed values for real-time inference at low latency (typically Redis or DynamoDB), because production detection cannot wait several seconds for a feature-extraction query to run against raw logs.
The third piece is point-in-time correctness, which is subtle and matters more than people realize. When I train a model to predict whether a user session was compromised at 09:00 on Tuesday, I can only use features whose values would have been computable at 09:00 on Tuesday. If I accidentally let the training set see failed_logins values aggregated as of 17:00 the same day, the model "cheats" by accessing future data that won't exist at inference time, and the production model then performs far worse than the training metrics promised. A feature store handles this correctly by joining historical entity-and-timestamp rows against the feature value as of that timestamp, and hand-rolled feature pipelines almost always get this subtly wrong on the first few iterations. The reason isn't that the SQL is impossible to write but that it's easy to write incorrectly, and the failure mode (overfit training metrics, underperforming production model) is hard to debug after the fact, which makes point-in-time correctness one of the strongest standalone arguments for using a feature store rather than rolling your own.
The clearest signal you need one is operational rather than architectural. If your detection team maintains more than ten or fifteen supervised ML models and the same derived features (failed_logins_24h, geo_diversity_score, rare_process_count, parent-process-chain-depth) are being extracted independently by each model with slightly different windowing logic, that's when the duplication and inconsistency cost starts to outweigh the platform cost. Below that threshold a shared SQL view or a dbt model probably does the job. The second signal is that an analyst investigating an alert from Model A pulls their own ad-hoc query and gets a different number from what the alert reports, because Model A counts a rolling 24-hour window and the analyst's ad-hoc query counts the calendar day, so the analyst starts to stop trusting the alerts. That's the feature-definition-drift problem, and a feature store is one solution to it, though it isn't the only one.
The duplication problem
Eighty models, eighty definitions of "failed logins in 24 hours."
The motivating scenario is useful to picture. Imagine a detection-engineering team maintaining eighty supervised ML models (credential stuffing, account compromise, brute force, impossible travel, suspicious privilege escalation, rare process execution, beaconing, exfiltration, a long tail of variants). Each model needs a handful of derived features, and many of those features overlap across the eighty.
One detection counts failed logins in a rolling 24-hour window from the current time. Another counts failed logins since midnight today (calendar day). A third counts failed logins in the last 1,440 minutes, which is functionally equivalent to the rolling window but written by a different engineer in different code. All three call the result failed_logins_24h. The first time an analyst investigates an alert and finds the model's reported count disagrees with their ad-hoc query, the trust erosion starts.
There's also a compute cost. Eighty models each running their own feature extraction against the same raw authentication logs is roughly eighty times the cluster work for one feature, repeated for every shared feature. I've seen architecture reviews where this duplication ran to a sustained 60–90× over what a single shared extraction would cost, on EMR or Databricks clusters that were running 24/7 specifically to keep these features fresh, which is the strongest dollar argument for centralizing.
I want to flag the size of the leap, though, because the "eighty supervised ML models" scenario shows up in fraud detection and at the largest, most ML-mature security teams and is uncommon elsewhere, since most of the SOC teams I see have zero models in production. So the cost of duplication only matters once you've crossed into the territory where you have enough models for it to bite.
Feast and Kedro
The open-source feature store, and the pipeline framework beside it.
Feast began at Google and Gojek, was open-sourced in 2019, and now lives under Linux Foundation AI & Data governance with an Apache 2.0 license. It supports more than fifteen offline stores (BigQuery, Snowflake, Redshift, Iceberg, DuckDB, and others) and around ten online stores (Redis, DynamoDB, Cassandra, PostgreSQL). The maturity story on the platform itself is solid; the maturity story on security-specific deployments is the gap I keep coming back to. Feast has four moving parts: a feature registry in S3 or Postgres holds the definitions (the YAML and Python that say what features exist and how they're computed), the offline store (Iceberg in the lakehouse architectures I work with) holds historical feature values for training, the online store (Redis is the typical choice) holds the latest values for low-latency inference, and an optional feature server is a REST API for non-Python clients to read features.
A minimum-viable feature definition in Feast looks like this:
from feast import Entity, FeatureView, Field
from feast.types import Float32, Int64
from datetime import timedelta
user = Entity(name="user", join_keys=["user_id"])
user_behavior_fv = FeatureView(
name="user_behavior_features",
entities=[user],
ttl=timedelta(days=7),
schema=[
Field(name="failed_logins_24h", dtype=Int64),
Field(name="successful_logins_7d", dtype=Int64),
Field(name="login_geo_diversity_score", dtype=Float32),
Field(name="privileged_action_count", dtype=Int64),
],
online=True,
source=iceberg_source,
)Two things to notice. The schema definition is the single source of truth, so every model that reads user_behavior_features:failed_logins_24h reads the same value computed by the same definition. And online=True tells Feast to materialize the values into the online store, which is what makes the feature available for sub-50ms inference rather than only for batch training. The training-side API is where point-in-time correctness shows up: you pass an entity dataframe of (user_id, event_timestamp) rows and the feature names you want, and Feast joins back against historical feature values as of each timestamp:
from feast import FeatureStore
store = FeatureStore(repo_path=".")
training_df = store.get_historical_features(
entity_df=entity_df,
features=[
"user_behavior_features:failed_logins_24h",
"user_behavior_features:login_geo_diversity_score",
"user_behavior_features:privileged_action_count",
],
).to_df()The corresponding online call returns the latest precomputed values for a given entity in a few milliseconds from Redis, which gives you the basic shape: you define a feature once, train against the historical features and infer against the online ones, and you stop copy-pasting the same SQL into eighty different model repos.
Kedro is the pipeline framework that tends to travel with Feast. It's an open-source framework for organizing data-science and ML pipeline code into modular nodes and pipelines, developed at QuantumBlack (McKinsey's analytics arm) and open-sourced under Apache 2.0, and the honest one-liner is that it structures workflows to enforce modularity, configuration management, and reproducibility on code that would otherwise be a pile of notebooks. A node is a Python function with declared inputs and outputs, a pipeline is a directed acyclic graph of nodes wired together by name, and a data catalog defines where each named input or output lives (an Iceberg table, a parquet file, a feature view, a Redis key). Running kedro run materializes the pipeline; kedro viz opens an interactive DAG view in the browser. A condensed detection pipeline looks like:
from kedro.pipeline import Pipeline, node
from .nodes import (
ingest_cloudtrail, normalize_ocsf, extract_features,
load_feature_store, run_detection_models,
filter_high_confidence, send_alerts,
)
def create_detection_pipeline() -> Pipeline:
return Pipeline([
node(ingest_cloudtrail, "params:cloudtrail_path", "raw_cloudtrail"),
node(normalize_ocsf, "raw_cloudtrail", "ocsf_events"),
node(extract_features, "ocsf_events", "user_features"),
node(load_feature_store,"user_features", None),
node(run_detection_models, "ocsf_events", "raw_alerts"),
node(filter_high_confidence, "raw_alerts", "validated_alerts"),
node(send_alerts, "validated_alerts", None),
])What Kedro buys you, specifically: the OCSF normalization step can be changed without re-testing the feature-extraction step, the data catalog tracks lineage automatically so when an alert fires you can walk backwards through the pipeline and see exactly which input dataset, feature materialization, and model version produced it, and each node is unit-testable in isolation. That last point is the one that matters most for security, because detection-engineering code that's been through a Kedro refactor tends to have meaningfully higher test coverage than the equivalent notebook-and-cron sprawl, and the production deployment story improves with it. The natural question is "why not use Airflow," and the answer is that Airflow is the dominant orchestrator for batch ETL and deserves to be, but the place it doesn't fit cleanly is the ML-development inner loop where a data scientist is changing feature logic, retraining, evaluating, and redeploying multiple times a day. Kedro is built around that inner loop, so the typical mature pattern is to develop the pipeline in Kedro and deploy it to Airflow (or AWS Step Functions, or Kubeflow) for scheduled production execution, the two complementing each other rather than competing.
Kedro adoption in security specifically is rare but not nonexistent. Two security data-engineering teams I've talked to (financial services and technology, both anonymized) migrated detection-pipeline code from manual Python scripts to Kedro and reported faster deployment, higher test coverage, and better lineage for incident-response forensics. Both also flagged the same thing: the learning curve is real and the data-catalog concept is the part that bounces people, so a team that commits without engineering buy-in tends to abandon the Kedro work partway through.
Build versus buy
Feast, Tecton, and the vendor landscape.
Feast is not the only feature store in the market. Tecton is the commercial managed offering founded by the team that built Uber's Michelangelo platform. Databricks ships a feature store as part of Unity Catalog. Hopsworks is an independent open-source option with a strong on-premises story. The build-versus-buy decision matters more in security than in fraud-detection ML, because security teams almost never have data-engineering staffing as their core skillset.
My read: if you're already a Databricks shop and you have one team that owns both the lakehouse and the detection-engineering ML work, the Unity Catalog feature store is the path of least resistance, because you pay for the integration rather than for re-implementing point-in-time correctness. If you're not on Databricks and you have engineering capacity to operate Redis and a scheduled materialization pipeline, Feast is the open-source default; the cost of ownership is real but the lock-in is minimal. Tecton is the right call when you need a managed offering with strong point-in-time correctness guarantees and you have budget that won't survive a build-it-yourself delay.
I have not seen a security-specific deployment of Tecton or Databricks Feature Store either. The vendor neutrality call here is essentially "pick the one that fits the surrounding stack and the team's skill profile," because there is no security-domain validation that would tip the scales for any of them yet.
OCSF and the feature schema
Normalize once, define features once.
One argument for feature stores in security that I do find genuinely persuasive is the interaction with OCSF (Open Cybersecurity Schema Framework). If your authentication, network, and process telemetry are normalized to OCSF at ingestion, your feature definitions stop being vendor-specific. A failed_logins_24h feature is "count where class_uid = 3002 (Authentication) and status_id = 2 (Failure) in the trailing 24-hour window," which is one definition that works across CloudTrail, Azure AD, Okta, Windows Event Logs, and any other authentication source you normalize.
Without OCSF, the same feature needs ten different per-source implementations because each vendor names the outcome field differently and encodes "failure" with a different enum value. The practitioners I've talked to who are running OCSF-normalized ML pipelines report something like 50–70% reduction in feature-engineering effort once the OCSF mapping is paid down. I want to flag the sample size on that number: two organizations, both with mature OCSF adoption and a self-selection bias toward teams who would talk to me about it. The directional claim, that OCSF collapses per-source feature work, is sound. The exact percentage is more uncertain than the number suggests. (I cover the upstream mapping work in LLM-assisted OCSF mapping.)
The complementary observation is that without OCSF, a feature store is partially fighting the wrong fight. If your feature definitions are still vendor-specific underneath the store, you've centralized the symptom (duplicate code in eighty model repos) without centralizing the cause (vendor-specific schemas at ingestion). The combination that pays off is OCSF normalization in the route layer together with feature-store centralization in the ML layer, because either one on its own only gets you partway.
Cost shape and the evidence gap
What a deployment costs, and the case studies that don't exist.
The cost line is one of the places I have to be careful. I have modeled cost estimates from AWS list pricing for Redis ElastiCache, Lambda or ECS for materialization, and S3 for the offline store. I do not have measured cost data from a production security deployment, because I have not been able to find one to measure. With that caveat, the modeled envelope:
| Component | Monthly (list price) | Cost driver / sizing |
|---|---|---|
| Online store (Redis ElastiCache) | roughly $50–500/month | for r6g.large to r6g.2xlarge instances, sized to hold 10K to 100K precomputed feature rows. Scale up linearly with entity count. |
| Materialization compute | roughly $100–500/month | for Lambda or ECS jobs running on an hourly or daily cadence over 1M to 100M source rows. The shape of the cost is dominated by materialization frequency more than data volume. |
| Offline store and feature registry on S3 | roughly $50–200/month | for 100GB to 1TB of Parquet feature data and registry metadata. |
| Total envelope | $200–1,200/month for typical deployments, $500–2,000/month for high-scale (sub-minute refresh, hundreds of millions of features) | I would size at the high end for any production security deployment until measured data argues otherwise. |
These numbers are list-price, single-region, and reserved-instance discounts plus committed-use agreements compress them noticeably. The reason I quote a wide range is that I would rather over-communicate uncertainty than under-communicate it on a number this central to budgeting. Run the spreadsheet for your specific scale before quoting these to a CFO.
The harder caveat is the evidence gap, and this is the part I want to be most direct about. As of early 2026, I could not find a single publicly documented production deployment of Feast inside a SOC or detection-engineering team. I searched the Feast GitHub issues for the "security" tag (eighteen results, all fraud detection, no SOC use cases), I searched the Feast Slack community (around 15,000 members, three threads mentioning security, all exploratory), and I searched LinkedIn for profiles listing both Feast and security experience (eight results, six in fraud detection, two in cybersecurity roles both marked as proof-of-concept or exploration). My best estimate is that fewer than 5% of HMM4 teams are using a feature store in production, and HMM4 itself is only 8% of organizations, so the aggregate population of "security teams running a production feature store" is small enough that the absence of public case studies isn't surprising. It is also a constraint on how strongly I can recommend this pattern.
The adjacent validation I do trust: Feast is widely deployed in fraud detection at fintech and e-commerce, and the workload shape (real-time risk scoring on user-and-event entities, behavior analytics, point-in-time correctness for training) is similar to threat detection, so the technical fit is plausible. What's missing is knowledge transfer from fintech deployment experience into security teams who have ML literacy concentrated in a few headcount and a different operational rhythm. The code examples above are illustrative patterns adapted from fraud-detection use cases, so I would call them informed hypotheses rather than validated practice and treat them accordingly, because if you adopt this pattern you are part of the first generation of security teams to do it in public. That is a defensible position, with upside (you'll have novel operational learnings worth talking about) and cost (you will not have a peer reference to call when something breaks). If you do it, please write about it; the next generation of security teams trying to make this decision will thank you.
Verification flags
What I'd ask a vendor before committing to a feature store.
Because the evidence gap means a buyer has to do more of the validation work than usual, here is the short list of questions I'd put to anyone selling or recommending the feature-store pattern in security:
- Show me one named production deployment of a feature store in a SOC or detection-engineering team. If the answer is fraud detection, that's adjacent but not the deployment. If the answer is a vendor case study without a customer name, that's marketing.
- Show me the point-in-time correctness implementation. This is the place hand-rolled feature pipelines most often go subtly wrong. A vendor or team that can't walk you through how they handle it is selling you a feature store that hasn't earned the name.
- Where's the OCSF normalization layer? If features are still vendor-specific underneath, the feature store is centralizing the symptom and not the cause. The combination is what pays off.
- What's the operational cost of materialization? Modeled cost from list pricing is a starting point. Measured cost from a real deployment at your scale is the answer that matters.
- What's the failure mode when the online store goes down? Redis is reliable but not infallible. The detection pipeline's behavior when feature reads fail (fail-open, fail-closed, fail-stale) needs to be a deliberate design decision, not something that comes up during an incident.
A vendor who answers all five well has earned a closer look, and one who can't isn't disqualified so much as asking you to do the validation work yourself, because Feast and Kedro do production work in adjacent domains and the security application is a sound architectural hypothesis that stays a hypothesis until more SOC teams put it into production and publish what they learn.
HMM mapping
Which tools, which maturity level.
The SANS Hunting Maturity Model defines five levels. HMM0 is no hunting. HMM1 is alert-driven triage. HMM2 is procedural hunting (applying community playbooks from MITRE ATT&CK and threat reports). HMM3 is innovative hunting (developing novel hunts, applying data-analysis methods, measuring effectiveness). HMM4 is leading with automation: codifying mature hunts into ML-assisted production detection.
The mapping below is my synthesis of where each tool earns its keep against the HMM ladder. It is not a published framework. Treat it as a working hypothesis to test against your own program. The general shape (start light, add tools as the cost of not having them shows up) seems to hold across the teams I've worked with.
HMM2 to HMM3 transition
Goal: turn procedural hunts into a library of documented, reproducible, peer-reviewed playbooks. Add Jupyter first, and document one hunt end-to-end in a notebook, commit it to Git, have a teammate review it, and repeat ten times, because that single workflow change pays back more than anything else on the list.
Great Expectations follows in the same phase. After your second or third hunt fails because a log source schema changed without anyone noticing, the data-quality framework starts paying for itself.
What to defer at this stage: MLflow, DVC, Feast. You don't have enough variants yet to need experiment tracking, you're not validating against frozen datasets yet, and you have no ML models. Adopting these too early creates infrastructure overhead without proportional benefit.
Timeline I'd plan against: six-to-twelve months at this phase. Success looks like 20-or-more documented hunts in Jupyter, five-or-more reproducible playbooks with precision over 80%, data quality tests on the critical log sources, peer-review as part of the hunt workflow.
HMM3 innovative hunting
Goal: build a library of validated hunts and start identifying automation candidates. Add MLflow (or ClearML, or Weights & Biases; pick one and commit) once you're routinely comparing ten-plus variants of a detection rule. The experiment tracker becomes the system of record for which variant ships.
Add DVC when you start validating detection rules against frozen historical windows. If you're Iceberg-native, the lighter version of this is a tagging convention against Iceberg snapshot IDs and a notebook that pins the snapshot it ran against. Either approach works; the discipline matters more than the tool.
Continue Jupyter and Great Expectations from the previous phase, and keep deferring Feast, Kedro, and Kubeflow.
Timeline: 12-to-18 months. Success looks like 50-or-more hunt campaigns tracked in the experiment tracker, 10-or-more versioned baseline datasets, and five-or-more validated detection rules ready for automation with precision over 90%. The five rules at over 90% precision are the input to the HMM4 transition.
HMM4 leading with automation
Goal: automate the routine investigation work that the validated playbooks now describe. The readiness signal is concrete: ten or more supervised ML models in production with measurably overlapping feature definitions, ML-literate detection-engineering staffing (not one data scientist on loan, but a real team), and an OCSF normalization layer in the ingestion pipeline. When all three are true, a feature store starts to pay off. The realistic ceiling for HMM4 automation in a human-in-the-loop SOC is roughly 30-to-40% of routine investigations. I work through why it sits there, and why agent-native claims in the high nineties don't yet hold up, in the agentic-SOC reality piece.
At this phase, the full toolchain comes into play, with Jupyter for prototyping new techniques, MLflow (or equivalent) for tracking model versions in production, DVC for training datasets, Great Expectations for production data-quality gates, Feast for the feature layer once you have multiple ML detectors, Kedro for pipeline modularity and lineage, and Kubeflow (or an equivalent orchestrator: Airflow, Prefect, Dagster) for running the production detection pipelines on Kubernetes. Combined learning curve for the Feast-plus-Kedro pair is roughly 40–60 hours across the team on top of the development-and-quality stack you already run.
What distinguishes HMM4 from earlier phases is the workflow move from "human runs the hunt, tool documents it" to "tool runs the playbook, human reviews exceptions." The toolchain is the same; the operating model changes. That shift takes 12-to-24 months from the start of HMM3 maturity, and the limits on it are organizational (analyst workflows, on-call structure, escalation paths) more than technical.
I want to flag one debate that lives at this maturity level. The Feast-plus-Kedro pattern is a human-centric MLOps architecture, where automation augments analysts, models score events, and analysts review and approve actions, and the proven ceiling for that shape is the roughly 30-to-40% of analyst time Expel reports for its Ruxie pipeline. The alternative some agent-architecture vendors claim is agent-native investigation, where AI agents orchestrate end-to-end and the cited numbers run in the high nineties (98% or more), though those figures come from controlled or marketing settings rather than production SOC deployments. I don't have independent evidence that the high-nineties claim generalizes, and the published 30-to-40% MLOps automation number from Expel is the floor I trust, so treat the higher agent-architecture claims as Tier C-to-D until production validation lands. My recommendation is to plan for MLOps as the proven path and treat agent-native architectures as an option to monitor rather than commit to.
Lakehouse integration
How the toolchain connects to the data platform.
The MLOps tools assume a data platform underneath them. For a security team running an Iceberg lakehouse with a query engine (StarRocks, ClickHouse, Trino, or DuckDB) and a catalog (Polaris, Unity, Glue), the integration points are straightforward:
- Jupyter to Iceberg. DuckDB's
iceberg_scan()reads Iceberg tables directly from notebooks. PySpark or Trino's Python client work for larger windows. SQLAlchemy connectors give SQL access to ClickHouse and StarRocks for engines that don't speak Iceberg natively. The Arrow plus ADBC path is the columnar-native version of this, and removes the row-to-column serialization tax on result sets. - MLflow artifact storage. Point MLflow at the same object storage that backs your Iceberg tables. The PostgreSQL backend for the tracking database can co-locate with your catalog's database to reduce operational surface.
- DVC remote storage. Same object storage, different prefix. When you need to version an Iceberg-derived training dataset, the DVC metadata file captures the Iceberg snapshot ID alongside the file hash so the lineage is recoverable.
- Great Expectations against OCSF tables. The framework's SQL connector runs assertions against any engine that speaks SQL. The most valuable assertions are at the OCSF-normalized layer: required fields present, expected categories present, time windows current. Schema-drift detection at this layer catches a useful share of the silent-detection-failure problem.
- Feast against Iceberg. Feast can use Parquet on object storage as the offline store and a separate online store (Redis, DynamoDB) for serving. The feature definitions point at OCSF-normalized tables, which means the feature layer composes with the schema standard rather than fighting it.
The composability that matters: Iceberg as the table format, OCSF as the schema, Arrow as the in-memory format, and a query engine of your choice, none of which require you to commit to a particular MLOps tool. You can swap MLflow for ClearML without re-architecting the data layer. That's the same argument I make for Iceberg over Delta in the table-format piece, applied one layer up.
Honest limits
What this approach does not solve.
Six honest limits, none of which I'd hide from a stakeholder evaluating this stack:
- Tooling does not produce hunters. Jupyter does not generate hypotheses, and MLflow does not invent novel detection techniques, because the tools amplify the analytical discipline of a team that already has it, so a team that hasn't formed hunting hypotheses before will not start because they installed JupyterLab.
- The HMM-to-tool mapping is my synthesis. SANS publishes the maturity model; the tool placements against the maturity ladder are an opinion that fits the teams I've worked with. Other practitioners will draw the lines differently and may be right for their context. Treat it as a starting point, not a prescription.
- I have not independently benchmarked these tools on production OCSF data. The descriptions of where each tool earns its keep are drawn from documentation, training workshops, and conversations with practitioners, not from a controlled study. A security-specific benchmark of MLflow tracking overhead against high-volume hunt campaigns is on the lab roadmap and would be Tier B evidence at best when it lands.
- The feature-store patterns are C-to-D evidence for security. Feast and Kedro are Tier B as tools (Linux Foundation governance, mature ecosystem, fraud-detection production deployments), but the security-specific application drops to C-to-D because I could not find a single publicly documented production deployment of a feature store inside a SOC in early 2026. The code examples are informed hypotheses adapted from fraud-detection use cases, not validated security practice.
- The 30-to-40% HMM4 automation ceiling is Tier B, not Tier A. Expel's published number is a single vendor's reported outcome on a particular managed-detection workload. The structural argument for why it sits at roughly that ceiling (human-centric SOC workflow, MLOps tools designed for human-in-the-loop) is plausible. The number may shift as agent architectures mature.
- The toolchain is itself an attack surface. Every tool here is a service that holds your detection logic, your dataset pointers, and often the credentials a run needs, which makes the MLOps layer a target in its own right rather than neutral plumbing. MLflow's tracking server has a documented record of unauthenticated path-traversal and remote-code-execution flaws (CVE-2025-11201, CVE-2026-2614, CVE-2026-2635, CVE-2026-2734), and model artifacts serialized as pickle, joblib, or torch weights run arbitrary code on load, so a poisoned model file is an entry point and not just bad data. The discipline that answers this has a name, SecMLOps: keep these services off the public internet, rotate default credentials, constrain artifact paths, and prefer pickle-free formats like ONNX or Safetensors. The point to carry is that standing up data-science tooling expands the surface you have to defend, and a service that stores your security logic earns the same patching and network hygiene as anything else in production.
None of these are reasons to dismiss the approach, but they are reasons to size expectations and sequence adoption correctly.
Practical guidance
What to do in 2026.
Four moves I'd actually recommend, ordered by how cheap they are to execute:
- Adopt Jupyter for one hunt this week. Pick a hunt campaign you already know the answer to. Document it in a notebook end-to-end: query, filter, chart, narrative. Commit it to a Git repo. Have a teammate review it. That single exercise produces the template for every subsequent hunt and exposes the rough edges (data access, library installs, kernel choice) on a low-stakes problem rather than a live investigation. Do that three or four times before reaching for MLflow, because you won't feel the need to compare variants until you've already started generating them.
- Treat Great Expectations as the second adoption, not the fifth. Pick the three log sources your detection rules depend on most. Write a minimal expectation suite for each: required fields present, freshness within the expected window, distinct values inside the expected set. Route failures to the channel your team actually monitors. The cost of building this is hours; the cost of a silent detection failure can run into weeks of adversary dwell time.
- Defer experiment tracking, data versioning, and feature stores until they earn their keep. The trap with MLOps tooling is adopting it all at once because the ecosystem is well marketed. The marketing is mostly for ML platform teams shipping models, not for hunters documenting hypothesis-driven work. Adopt MLflow when you have ten variants to compare and a spreadsheet that has fallen out of date. Adopt DVC when you need to revalidate against bit-identical data. Adopt Feast and Kedro when you have more than ten supervised ML models reading the same features. Not before.
- Measure precision and recall on the hunts you ship. This is independent of any tool on this list, because a hunt that ships a detection rule without a precision number tends to produce alert noise, while a hunt that ships with measured precision and recall against a labeled or red-team-validated dataset has the property that makes it promotable to automation later. The MLOps tooling exists to make this discipline easier, so it is worth adopting, but it does not substitute for measuring in the first place.
The reframing I started with is the one I want to land on, which is that a mature threat hunter is doing data-science work whether they call it that or not, so the MLOps tools are for the role you already play, with better instruments, rather than for some different role you'd have to become. Start with the instrument that costs the least and pays back the fastest (Jupyter for hunt documentation) and let the rest of the toolchain earn its place as the discipline matures, with the feature store at the far end of that ladder where only a small fraction of security teams have any business standing today.