The AI Center of Excellence Is Dead. Long Live the AI Operating Model.

The AI Center of Excellence (CoE) was the standard playbook. Stand up a centralized team of data scientists. Give them a mandate. Let them prove value. Then scale.

It doesn't work. Not because the people are wrong, but because the structure is wrong.

A centralized AI CoE creates a bottleneck by design. Every use case funnels through a single team. That team becomes overloaded. Prioritization becomes political. Business units that don't get served start building their own shadow AI capability. Within two years, the organization has a CoE that handles a fraction of demand, three or four rogue data science teams building on different stacks, and no shared standards for anything — deployment, monitoring, governance, or quality.

This is not a people problem. It's an architecture problem.

What Went Wrong with the CoE Model

The CoE model made three assumptions that don't hold at enterprise scale:

Assumption 1: AI capability should be centralized. In reality, AI use cases are deeply domain-specific. A fraud detection model requires different data, different validation, and different operational characteristics than a customer segmentation model. Centralizing the talent doesn't centralize the context — and context is what makes AI work.

Assumption 2: Proving value will naturally lead to scaling. Proving value and scaling are fundamentally different challenges. A successful pilot proves that a model can work. Scaling proves that an organization can repeatedly develop, validate, deploy, monitor, and govern models. These are operating model capabilities, not data science capabilities.

Assumption 3: The hard part is building models. The hard part is everything around the model — the data pipeline that feeds it, the validation process that approves it, the platform that serves it, the monitoring that watches it, the governance that ensures it remains compliant, and the organizational clarity about who is accountable for each of these things.

The Operating Model Alternative

An AI operating model doesn't centralize capability. It distributes capability while centralizing standards.

The distinction matters. In an operating model approach:

A central platform team owns the shared infrastructure — compute, feature stores, model registries, deployment pipelines, monitoring frameworks. They don't build models. They build the platform that makes model development, deployment, and monitoring repeatable.

Domain teams own their own AI development. They have embedded data scientists or ML engineers who understand the domain. They build models against shared platform standards, using shared tooling, following shared governance.

A governance function owns the risk framework — risk classification, validation requirements, monitoring standards, escalation paths. This function sets the rules. It doesn't do the work. Model owners in domain teams are accountable for compliance. The governance function audits, advises, and escalates.

What This Looks Like in Practice

Demand and Intake

Use cases originate in business domains. A lightweight intake process scores them on business value, technical feasibility, data readiness, and risk tier. High-risk use cases (credit decisions, regulatory reporting) trigger enhanced validation requirements automatically. Low-risk use cases (internal analytics, operational dashboards) move through a lighter path.

Development Standards

Domain teams develop models using shared templates — model cards, experiment tracking, code review standards, bias testing requirements. The central platform provides the tooling. The governance function provides the templates. The domain team does the work.

Deployment Gates

Every model passes through deployment gates before reaching production. The gates vary by risk tier. A Tier 1 model (high regulatory impact) requires independent validation, shadow deployment, and sign-off from the model risk function. A Tier 4 model (low impact, internal use) requires peer review and automated testing.

The gates are defined centrally but executed locally. The domain team is responsible for meeting the gate criteria. The platform team provides the deployment infrastructure. The governance function audits compliance.

Monitoring and Lifecycle

Models in production are monitored against defined thresholds — data drift (Population Stability Index), performance degradation, fairness metrics. The monitoring infrastructure is centrally provided. The thresholds are set per model based on risk tier. When a threshold is breached, the response follows a defined escalation path — again, varying by risk tier.

The Organizational Shift

Moving from a CoE to an operating model is an organizational change, not a technology change. It requires:

• Clear role definitions. Platform team, domain teams, governance function — each with distinct accountabilities. Overlapping accountability is no accountability.
• Shared standards that are actually shared. Not a document in a wiki that nobody reads. Living standards embedded in tooling, templates, and deployment pipelines.
• Investment in platform capability. The central team's job shifts from "doing AI" to "making AI doable." This requires engineering investment in developer experience, deployment automation, and monitoring infrastructure.
• Executive sponsorship of the model, not just the outcomes. The operating model itself needs a sponsor who cares about whether it's working — not just whether individual AI projects are delivering results.

Why This Matters Now

The EU AI Act introduces regulatory requirements that make governance non-optional. Model risk management in banking (SR 11-7, BCBS 239) already demands it. As AI becomes embedded in more consequential decisions — credit, hiring, fraud, medical — the organizations that can demonstrate governance at scale will have a structural advantage.

You can't demonstrate governance at scale with a CoE. You need an operating model. The CoE was the right answer for 2018, when AI was experimental and the goal was to prove value. The operating model is the right answer for 2026, when AI is operational and the goal is to govern it at scale while continuing to deliver value.

The Center of Excellence served its purpose. It's time to replace it with something that actually scales.

For a practical governance framework that implements these ideas — risk classification, lifecycle governance, compliance mapping — see ai-governance-framework on GitHub.