Structural Models

How you structure AI ownership is one of the most consequential decisions in an AI transformation. Structure determines who has authority, where decisions get made, how fast the organization can move, and how well governance actually holds. Most organizations choose their structure by default rather than by design. The result is a structure that fits their early stage but creates serious problems as the portfolio scales.

There are three dominant models. Each has a legitimate use case and a specific failure mode. Knowing when to use each, and when to transition between them, separates organizations that scale AI successfully from those that plateau or regress.

The Three Models

Model 1: Centralized CoE

A single central team controls AI strategy, development, deployment, and governance. Business units submit requests. The CoE prioritizes, builds, and delivers.

When it works: Early-stage organizations that need to establish capability, set standards, and avoid the chaos of distributed experimentation. The CoE builds the first wave of production systems, establishes the reference architecture, and demonstrates what good looks like before handing the playbook to the business.

When it breaks: Once the portfolio grows beyond what a single team can handle without becoming a bottleneck. Business units with high-frequency AI needs will route around the CoE. Shadow AI appears. Talent concentrates in the center while the organization fails to build distributed capability.

Model 2: Hub-and-Spoke

A central governance and capability function (the hub) sets standards, owns the platform, manages the model inventory, and coordinates across functions. Embedded AI teams within each business unit or major function (the spokes) own execution within the governance framework.

When it works: At mid-to-large scale, where the portfolio is too large for centralized delivery but too complex for distributed ownership without governance guardrails. IBM's survey of more than 600 CAIOs found that where the CAIO drives a centralized or hub-and-spoke operating model, ROI on AI initiatives can be up to 36% higher than in decentralized structures (IBM IBV, 2025). IBM groups centralized and hub-and-spoke together in that comparison, so the finding argues for a central point of control rather than for hub-and-spoke over a CoE.

When it breaks: When the hub becomes administrative rather than enabling, issuing checklists instead of accelerators. When spoke teams lack sufficient AI expertise to execute without excessive hub involvement. When the governance framework is so heavy that spoke teams treat it as overhead rather than infrastructure.

Model 3: Federated

AI ownership is distributed across business units with minimal central coordination. Each unit hires its own AI talent, selects its own tools, and sets its own priorities. A nominal central function may exist but has limited authority.

When it works: In organizations where business units operate highly independently, have differentiated AI needs, and have sufficient AI maturity to govern themselves. Fast-moving consumer-facing divisions, for example, often cannot wait for central processes. Some highly regulated sectors run federated models within business-unit-level compliance structures.

When it breaks: When the organization needs a consolidated view of AI risk, when regulators ask for enterprise-wide AI documentation, or when redundant investments across units become visible to the CFO. Federated models also make it nearly impossible to build shared platform capability, because no single unit will fund infrastructure that benefits others.

Comparison Table

IBM's survey measured centralized and hub-and-spoke operating models together against decentralized ownership, so the two left-hand columns share one finding (IBM IBV, 2025). Every other row in this table is a judgement about how each model behaves, not a survey result.

Structural Diagrams

Centralized CoE

Hub-and-Spoke

Federated

Transition Paths

Centralized CoE to Hub-and-Spoke

This is the most common and necessary transition. The trigger is usually one of three signals: the CoE has a multi-month backlog, business units are building outside the process, or the CoE talent is overwhelmed and attrition risk is high.

The transition requires:

1. Auditing the existing portfolio to identify which capabilities can be productized as shared services vs. which require embedded expertise
2. Identifying and hiring AI leads within major business units, ideally from people who already understand the domain
3. Transferring execution accountability to the spokes while retaining governance and platform ownership in the hub
4. Defining the escalation path from spoke to hub for risk decisions, model approvals, and incidents

Expect 9 to 18 months for this transition to stabilize. The governance framework must be ready before spokes go independent, or they will develop local variants that are incompatible with each other.

Hub-and-Spoke to Federated (Partial)

Full federation is rarely the right outcome. What more commonly happens is that mature, large business units develop sufficient AI capability to operate more independently within the governance framework. The hub retains governance authority but reduces operational involvement.

This is not a transition to full federation. It is hub-and-spoke with lighter hub touch for mature spokes. The governance framework remains mandatory. The platform remains shared. The model inventory remains centrally tracked.

Federated to Hub-and-Spoke (Remediation)

Organizations that started federated often reach a point where the governance gaps become unacceptable, usually triggered by a regulatory event, a significant model failure, or a CFO-level reckoning with duplicate investments.

This transition is harder than building hub-and-spoke from the start. Business units have already established local processes, local vendor relationships, and local talent with local loyalties. Introducing central governance retroactively is a political exercise as much as an operational one.

The remediation path requires executive mandate, not persuasion. The CAIO must have authority to require model inventory registration, set minimum standards for new deployments, and sunset non-compliant systems on a defined timeline.

Maturity Indicators

Use these indicators to assess where your organization currently sits and whether the structural model is appropriate for your maturity level.

For team roles and decision ownership within these structures, see Operating Architecture.

Sources

1. IBM Institute for Business Value. "How Chief AI Officers Deliver AI ROI." 2025. Survey of more than 600 CAIOs across 22 countries and 21 industries, conducted Q1 2025 with Oxford Economics and the Dubai Future Foundation.

For the complete source list and methodology, see Sources & Methodology.