Put humans in the loop to generate real value from generative AI

Generative AI has shaken up the business and tech world, but the best tech relies on world-class talent to address the fundamental problems that haven’t been solved yet. At ServiceNow, we know the most constructive and value-creating strategies for generative AI are grounded in embedding human experience and expertise into the core capabilities.

In the Now Platform® Vancouver release, we focused on using generative AI to help deliver value in three key areas: productivity, experience, and agility. This involves incorporating humans in the loop to address challenges that employees experience every day.

New tech, same people

Generative AI generates value only if it solves specific problems for specific people. In our software design process, we define these people according to personas, such as developers, support agents, or line-of-business end users.

We track what these different personas need based on continuous feedback from our customers and our employees. The identified needs tend to reflect common desires across personas: higher productivity, user-friendly or frictionless experiences, and greater agility.

This persona-driven design process has led to Now Assist—our generative AI experiences. Now Assist defines how some features, such as code completion for developers, target one specific persona and how others, such as case summarization, apply to a range of personas, from IT support to customer service to HR, with nuances of execution.

This approach helps ensure that all Now Assist capabilities fit within the same workflows the personas are used to and familiar with.

All this helps maximize the chances that the people who use generative AI will distill consistent, lasting value from doing so. One of the best ways to test this is within an organization—which we did by deploying Now Assist among our IT, HR, and customer support teams to gauge and learn from their feedback.

It's worth noting that this persona-based approach informs development across all the features in the Vancouver release. Accounts Payable Operations, for example, automates invoice ingestion and purchase order matching to help deliver faster, more accurate invoice processing.

The product forms the final piece of the workflow from sourcing to payment for one of our newest personas—those in procurement and supplier roles. Like most of our new features, it was built based on extensive input and validation from customers experiencing the same challenges every day.

Experience, the difference

To solve specific problems for specific people, generative AI needs to deliver an experience that warrants people’s trust. This is best served by a humans-in-the-loop model, where users review all generative AI outputs before accepting them, as well as the processes and sources that surfaced those outputs.

This minimizes the chances of AI mistakes or hallucinations causing actual harm and builds users’ confidence in the technology. It also helps improve the tech’s fidelity.

Another way to keep humans in the loop is by tapping into their experiences and expertise to train the AI that eventually assists them. Large language models (LLMs), which power generative AI, are most effective when trained on domain expertise related to the specific problems we seek to solve.

In the case of ServiceNow domain-specific LLMs, this expertise comes in the form of ServiceNow code snippets from our developer ecosystem—literally decades of experience submitted by experts in their field. When taken at scale, these code snippets capture developer best practices and contextual approaches that training on open-source data cannot replicate.

From our tests so far, these kinds of specially trained LLMs perform much more accurately and intuitively on tasks such as translating natural language into well-formed and scalable ServiceNow code, compared to LLMs trained on open-source content. Specificity, backed by uniquely human expertise, makes all the difference.

Solutions people value

Finally, we need to ensure we balance generative AI capabilities with the quality of experience offered to users. That means creating consistency across the organization, as we aimed to do with our Now Assist capabilities, which can be applied in one go from IT to HR to developers.

It also means developing a frictionless user journey from installation to deployment at organizationwide scale. The ability to turn ordinary text into code (or an entire app) will fail to impress if the interface takes minutes to load. A poor experience all but guarantees failure of adoption, no matter how powerful the solution.

Ultimately, all this underscores the importance of keeping humans in the loop with generative AI—from identifying the specific problems it must solve to training it on the best practices for finding solutions to ensuring it delivers those solutions in a human-friendly experience. Generative AI will generate value only if it helps real people. That means we humans must play our part.

Put generative AI to work to generate value in your organization with humans in the loop.