How to scale enterprise AI

Internal centers of excellence are emerging as the best way to go big

Enterprise AI at scale
  • Today, AI ownership in the enterprise is usually scattered among IT teams
  • New AI centers of excellence show promise for scaling enterprise AI projects
  • Successful AI ‘hub’ models work best when tightly integrated with business teams

In 2015, while many companies were making their first investments in artificial intelligence (AI) and machine learning, Capital One Financial Corp. was one step ahead. It was mapping out an organizational structure to help scale its AI efforts from prototype projects to bigger initiatives that would follow.

The company launched a Center for Machine Learning (“C4ML”)—a central development hub that took on all AI-related projects across the 50,000-employee company. It was basically a fulfillment shop. Business units submitted proposals or requests. The C4ML team then worked independently and cranked out solutions.

The model was prescient but flawed, as C4ML senior director Zach Hanif explained in a recent webinar. The team handled every aspect of the work, from teaching developers to training ML models. This constrained project speed and capacity, and stifled collaboration with business teams.

Three years later, Capital One overhauled C4ML by adding teams within each of its main business units: commercial and retail banking, auto finance and credit cards. The new AI “center of excellence” has increased project volume tenfold and significantly reduced development cycles.

Companies managing AI at scale are three times more likely to rely on a hub model.

Many companies today are making big bets on AI, but lack the internal structure and resources to help projects scale and succeed. While 90% of executives today say they’ve made investments in AI, according to a PwC study, less than 40% report seeing any gains over the last three years.

One reason for the poor returns is that control of AI development has been scattered between data and analytics teams, automation teams, individual business units, and outside vendors. (See box.) And just 8% of companies with AI initiatives have best practices in place that support scalability, according to McKinsey.

That’s why the center of excellence concept pioneered by Capital One looks promising. Nearly a quarter of large companies with AI initiatives—including JPMorgan Chase, Procter & Gamble, and Anthem—have AI centers in place.
While there’s no set of best practices yet to craft the ideal AI hub, a handful of strategies can help companies build a solid foundation and limit their risks.

Who owns AI development in the enterprise?

AI center of excellence24%
Data and analytics group19%
Enterprise-wide AI leader15%
Automation group14%
Business units13%
Outside providers11%
Source: PwC

Align structure with strategy

Gone are the days when CIOs could manage AI implementation like a skunkworks operation. There’s simply too much at stake. The global market for AI is projected to hit $203 billion by 2026, according to Fortune Business Insights. By 2030, PwC predicts those investments will contribute $15.7 trillion to the global economy.

Job one is plotting an AI strategy that can inform the organizational structure. That means identifying business problems that machine learning applications can solve, determining core requirements (especially with data), and identifying talent, internally and externally.

As strategy shifts to structure, a logical next step is to identify an “AI champion”—an executive-level advocate who can communicate the vision, take the lead on cultural or process changes, and commit resources.

“Where we see most success is with C-level sponsorship,” said Alex Fly, co-founder and CEO of Quickpath, an AI consultancy. “It definitely takes a high-level champion” with the ability to command budgets and lead change. In a 2018 Deloitte survey on the state of corporate AI, 45% of companies said they had appointed senior executives as AI champions.

“Something these people do a lot of is evangelizing AI, selling its abilities throughout the organization,” says Tom Davenport, professor of information technology and management at Babson College. “They need to be a good marketing and sales person.”
However, CIOs shouldn’t confuse “champion” with “cheerleader.” An executive sponsor also needs a realistic understanding of what AI can and can’t deliver.

“Sponsors and business stakeholders often have bloated expectations,” says Tom Debus, managing partner at data and AI consultancy Integration Alpha. “A few executives have asked me, ‘If I invest $2 million in your platform, can I send home 50 of my compliance officers?’ That’s not how you go about it.”

Expand the talent pool

AI development requires a spectrum of skills—preparing data, training machine learning models, coding, testing, and data visualization, to name a few. No single job profile covers it all.

That’s one reason to avoid recruiting expensive PhDs who could end up spending a lot of time on lower-level tasks like data cleaning and preparation. If companies don’t have clearly defined senior technical roles, high-priced AI talent isn’t likely to stick around very long, says Davenport.

Much of the work on an AI team can be handled by analysts who are familiar with easier-to-use machine learning tools. For example, one of the fastest-growing jobs on AI teams today is analytic translators. They serve as liaisons between business units and AI teams. “They know enough about the analytics and the capabilities on a higher level, but also understand business opportunities,” says Quickpath’s Fly.

Hub vs. spoke

AI management hasn’t matured enough for consensus to have emerged on the right degree of centralization—and whether a central hub model or a more decentralized approach will improve the chances of success.

That said, most experts suggest that a few core responsibilities should be centralized. According to a McKinsey study, those include data governance, recruiting, job training, defining ethical standards, and outsourcing of data and AI services that in-house teams can’t handle. McKinsey reports that companies managing AI at scale are three times more likely to rely on a hub model than those running smaller-scale efforts.

“Spoke” teams in business units, on the other hand, can handle tasks associated with AI usage: tracking user adoption, mapping out digital workflows, and measuring program performance.

At Capital One, machine learning experts from the central AI team work alongside data scientists and software developers from each business unit and meet daily to track progress and solve problems collaboratively. Under the new integrated approach, the center developed an AI-powered application that can predict when cardholders take out lines of credit for fraudulent purposes, or are likely to run up charges without paying the bills.

On its own, that may seem like a small win—but it’s the product of a management model that could pay off in a big way going forward.