AI requires companies to plan while they leap

ARTICLE | November 7, 2024

AI requires companies to plan while they leap

GenAI excitement is surging, but companies also need to create well-thought-out policies to govern it

By Evan Ramzipoor, Workflow contributor


Generative AI (GenAI) is at the apex of Gartner’s hype cycle list, and for good reason: One in five organizations say they are already using GenAI regularly in their businesses, while four in five say they have experimented with it, according to McKinsey's 2023 State of AI research.

Although the excitement is warranted, caution is too, says Jim Van Over, a field innovation officer at ServiceNow. He warns that executives who rush in too quickly, applying GenAI to as many use cases as possible before coming up with a governance strategy, may suffer for it. 

“You’ve got your vendors, your consultants, everybody under the sun telling you that GenAI will change your business, so it’s easy to get caught up in the hype,” says Van Over. Since GenAI is touted as a powerful tool that can do so much on its own, executives don’t think they need to do the legwork of coming up with a strategy for how and when to use it. Instead, companies are selecting vendors who sell AI tools and implementers who deploy them, hoping that the vendors have strategies themselves, he says. “And the reality is, they don’t either.”

Case in point, according to IBM’s 2023 Global AI Adoption Index: Just half of organizations surveyed say they have an AI governance strategy in place.

The thing about GenAI is that it cuts both ways, according to Samta Kapoor, EY Americas energy AI and responsible AI leader. “The good and bad about this technology is that you don’t have to be a data scientist to use it.” 

Indeed, one of the reasons ChatGPT captured the public’s imagination so quickly is that it requires so little training to get jaw-dropping results. And, since GenAI is so intuitive, executives believe it's easy to deploy and doesn’t demand the level of strategy and oversight that other new business technologies require, says Kapoor.

Embrace AI Faster and Responsibly With Robust AI Governance That Builds Trust          

Using a large language model (LLM) comes at a steep price. LLMs require server farms filled with energy-hogging graphics processing units (GPUs) to do their magic. Many AI vendors, including companies such as OpenAI, Google, Amazon, and IBM, employ either a token-based model or a subscription-based model to charge users for their services. In a token-based model, users are charged per token used, whereas subscriptions are typically measured in the number of users on an account per month.

But these pricing models aren’t always as straightforward as they might appear, says Van Over. For one, it’s often difficult to predict exactly how many GPUs are required to run a task on an LLM, because energy usage doesn’t become clear until after it’s been used. A team might estimate that a task will require only a few steps and therefore use up very little energy, but if it requires 10 additional steps, that’s a lot of additional energy—and cost.

To complicate things further, companies are buying GenAI tools and apps faster than they can calculate the cost of using them. Without the requisite calculations, companies can—and do—unwittingly spend tens of thousands of dollars overnight, says Van Over. For instance, a company might acquire a GenAI tool with the idea that only a dozen employees will use it, only to find out when the bill arrives that hundreds of employees have been using it. By 2025, about 90% of enterprise businesses that adopt GenAI will slow development because the cost will exceed the value generated, making further investment a tough sell, according to research by Gartner.


Beyond the financial concerns are those related to
security. Integrating GenAI into an existing tech stack can create security vulnerabilities or unintended outcomes, says Sean Hughes, ServiceNow’s AI ecosystem director. “Whenever you’re introducing new GenAI systems, you’re also introducing new risks, because you don’t know what you don’t know,” he says. “It’s not as simple as saying, ‘Let’s swap out this old model with a shiny new one.’”

For example, if a company integrates an unvetted LLM chatbot with its existing HR platform, crucial questions will remain, such as what kinds of employee data can the chatbot access? What is it doing with that data? Is it secure? How is the LLM being trained? What kinds of outcomes is it producing, and how might those outcomes impact employees?

Simply retiring an old model or layering in a new GenAI solution is an easy way to cause expensive problems later, says Hughes.

Deploying GenAI quickly, without burning through resources, requires the entire organization to work together to build a companywide GenAI vision.

Enterprise companies are more likely to face such challenges than smaller organizations, according to companies surveyed in IBM's AI index report. They cite difficulty deploying AI because the cost is too high, the projects are too complex, and the company lacks a holistic AI strategy.

Years before he joined ServiceNow, Eugene Chuvyrov was on the front lines of cloud adoption as a senior cloud architect, where he got a front-row seat to companies’ numerous mistakes.

“[Businesses] were ad hoc’ing this stuff,” he says. He saw firsthand how departments across organizations acted unilaterally to move their data to the cloud, built their own cloud control panels, and developed new processes on the fly. “There was never any enterprise wide strategy.”

The results, he says, were years of technical debt, sunk costs, and massive cleanup as companies realized they had overpaid for certain systems and lacked ones they desperately needed.

There are overlaps between the earlier cloud era and the current GenAI one, says Chuvyrov, who now spearheads product development in AI and other technologies at ServiceNow. Deploying GenAI quickly, without burning through resources, requires the entire organization to work together to build a companywide GenAI vision, he says. “The vision is a framework for people to control AI adoption rather than letting it control them.”

As part of that process, Van Over highlights the need for domain expertise within the organization. Companies should hire their own AI experts to evaluate and vet potential GenAI solutions, and there should be an open line of communication between the experts and leadership.

When a business considers a specific GenAI investment, leadership should identify the outcomes they’re looking for, what it’s going to take to achieve those outcomes, and whether the company can afford the financial and ethical risks, says Hughes.

Then leaders should design evaluation metrics by which they pick their AI vendors and communicate their strategy to the company, says Chuvyrov. Afterward, they should revise their strategy as needed for any future adoptions.

Chuvyrov suggests a safe way to get started is to identify low-risk use cases where GenAI can't cause major problems if things go awry. While experimenting, a company can draft a broad GenAI strategy and vision, including a data governance policy, to guide more ambitious implementations. Taking the time to create such policies will also allow for companywide conversations about how to use AI responsibly, put guardrails in place, and mitigate risks to customers, employees, and society at large.

Undertaking such efforts to ensure success doesn’t mean that companies must slow down adoption or otherwise be afraid to take the GenAI leap, says EY’s Kapoor. “I do understand the need to go fast,” she says. “That doesn’t need to stop. [Speed and strategy] can run on parallel tracks.”

Related articles

Enterprise AI Maturity Index 2024
ARTICLE
Enterprise AI Maturity Index 2024

Learn what AI Pacesetters do differently

Measuring GenAI’s business value
ARTICLE
Measuring GenAI’s business value

Here’s how to tell if your AI investments are paying off

The six biggest AI questions facing companies today
ARTICLE
The six biggest AI questions facing companies today

How to start filling in the blanks when it comes to the AI era’s most pressing unknowns

An Innovation Roadmap for IT Leaders
ARTICLE
An Innovation Roadmap for IT Leaders

How IT is taking a leadership position across the enterprise

Author

Evan Ramzipoor is a writer based in California.

Loading spinner