Why the old maturity models stopped working
The AI/MLOps maturity models that circulated in 2020-2022 – Google's three-level model, Microsoft's five-level model, the various consultancy riffs – were correct for the workloads of their time. They assumed training-serving split, model registries, feature stores, monitoring for drift, and pipelines that looked like CI/CD for classifiers. For that world, they still mostly hold.
For the world enterprise ML teams actually inhabit in 2026 – a mix of classical ML, LLM-backed workflows, agentic systems, and edge inference, with a regulatory environment that insists on explainability and a FinOps environment that insists on cost attribution – those models leave too much unsaid. The maturity model below is our current read, informed by what separates the teams that ship reliably from the ones that keep getting surprised.
Level 1 – Ad hoc
Notebooks in production, models shipped by hand, experiments tracked in a shared drive. One engineer knows how to retrain, and when they are on leave the model stays stale. No evaluation beyond the developer's own spot checks. No versioning of anything. Often no awareness that this is a problem until an incident.
Signs you are here: nobody can reproduce the current production model from scratch; "the model works fine" is a belief, not a measurement; model updates take a week of manual work per iteration.
Level 2 – Reproducible
Models can be rebuilt from versioned code plus versioned data. Experiment tracking is consistent (MLflow, Weights & Biases, Neptune, or similar). A model registry exists. Deploys are still mostly manual but predictable. Monitoring is limited to infrastructure metrics (latency, error rate) with little ML-specific observability.
Signs you are here: you can answer "what is running in production and how was it built?" in under an hour; any ML engineer can re-run a historical experiment; deploys happen weekly, not daily.
Level 3 – Continuous integration
ML pipelines are CI-tested. Data quality checks run on ingestion. Training and evaluation are automated on a cadence. A feature store (for tabular workloads) or a clearly-governed embedding and retrieval stack (for LLM workloads) is in place. Canary and shadow deployments are standard. Basic model monitoring – drift, performance – is wired up.
For LLM-backed systems at this level, eval suites exist and run on every release. System prompts are versioned in source control. Tool catalogues are governed. A per-request cost and latency telemetry exists and feeds the same observability stack as the rest of the platform.
Level 4 – Continuous deployment
Automated retraining triggered by quality signals or schedule, with evaluation gates that block deploys on regression. Traffic shifting managed by policy (canary ramp-up on success metrics, automatic rollback on anomaly). Feature pipelines under contract with producers. Training-serving skew actively monitored. GPU resources scheduled through a proper queueing system (Kueue, Volcano, or managed equivalents) rather than first-come-first-served.
For LLM systems at this level, evals include production-sampled inputs rolled back into the regression set weekly. Model swaps (GPT to Claude, frontier to small model) are a planned operational procedure with a known playbook, not a project. Shadow-mode evaluation on live traffic catches regressions before users do. Cost attribution per feature is available to product managers, not just platform engineers.
Level 5 – Closed-loop and governed
What level 4 does mechanically, level 5 does with the governance to survive regulatory scrutiny and the organisational design to keep velocity high. ML and AI systems have named owners with on-call rotations. Risk assessments are refreshed per material change, not per quarter. Documentation is generated from the pipeline, not maintained by hand. Retraining decisions are auditable. The organisation can answer a regulator's "show me how you control this" without a fire drill.
On the technical side, level 5 adds: automated red-teaming on a schedule, not just pre-launch; automated fairness and bias monitoring across protected dimensions; a clear, documented path for human-in-the-loop override on any high-stakes decision; and a cross-team catalogue where features, embeddings, prompts, and models are governed with lineage, ownership, and change review.
Level 5 is rare in 2026. Financial-services and healthcare teams operating under binding supervision are closest; most other organisations are between level 3 and level 4.
The self-assessment that actually helps
Instead of scoring yourself against the five levels as a whole, score across five axes. Most organisations are not uniform – they might be level 4 on deployment and level 2 on governance, or level 3 on classical ML and level 1 on LLM operations. The mismatch is usually where the next major incident will come from.
- Reproducibility – can you rebuild any production model from code and data, on demand?
- Evaluation – do you have a regression suite that fails closed, with calibrated metrics and a failure-mode catalogue?
- Deployment – are promotions gated on evaluation, with canary and rollback automated?
- Observability – do you have ML-specific monitoring (drift, quality, cost per call, latency p95/p99) alongside the standard infra stack?
- Governance – is there a named owner, an audit trail, and a risk assessment process that runs per material change?
The highest-leverage upgrade per tier
For most organisations, one investment dominates the next level of maturity.
- Level 1 → 2: adopt a single experiment-tracking tool and stop accepting notebooks as a deploy artefact.
- Level 2 → 3: build the evaluation regression suite and wire it into CI. Nothing moves as many downstream numbers.
- Level 3 → 4: replace first-come-first-served GPU allocation with a queueing system, and set up gated deploys.
- Level 4 → 5: add the governance layer – named owners, change review, documented risk assessments. It is the least fun and the most audit-durable.
A reality check
Most organisations we advise in 2026 are level 3 on classical ML and level 1-2 on LLM-backed systems, because the LLM operations stack is newer and the team muscle is still forming. That mismatch is normal; it is also the biggest risk carrier right now, because LLM workloads are where the reputational and regulatory exposure is highest. A level-2 LLM operation running on a level-3 platform produces exactly the kind of silent regressions that show up as front-page news.
Closing that specific gap – bringing your LLM operations up to the same maturity your classical ML has – is the single highest-ROI investment in 2026 for most mid-sized enterprise AI organisations. The playbook in earlier posts on this site (evals, system prompting, data contracts, FinOps) is the parts list. This maturity model is the sequencing guide.
