LLM topic discovery in Virtual Agent

  • Release version: Zurich
  • Updated July 31, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of LLM topic discovery in Virtual Agent

    Large language models (LLMs) enhance the ServiceNow Virtual Agent by enabling it to understand and process user statements in plain language more effectively than traditional Natural Language Understanding (NLU). LLMs simplify setup and improve conversation quality by leveraging generative AI to discover topics matching user intent without the need for extensive NLU modeling.

    Show full answer Show less

    Key Features

    • LLM Topic Creation: When ServiceNow Otto for Virtual Agent is enabled, you can create LLM topics in the Assistant Designer Asset library by selecting LLM as the model type. This allows Virtual Agent to use generative AI to automatically find relevant topics from user input.
    • Topic Discovery: LLMs eliminate the need for building and maintaining complex NLU models, intents, and keywords. Instead, a clear, plain language description of the topic suffices for the LLM to match user utterances with the correct topic.
    • Entity Extraction: Unlike NLU, there is no requirement to predefine or map entities to input nodes. The LLM dynamically identifies relevant entities based on user intent within the conversation.
    • Topic Switching: LLMs enable seamless and natural language-based switching between conversation topics without requiring cancellation or restarting. This supports dynamic interaction flows, such as changing an order mid-conversation or handling unrelated small talk.

    Key Outcomes

    • Faster Setup and Deployment: Creating and configuring topics with LLMs is quicker and more straightforward compared to traditional NLU approaches, reducing development time and complexity.
    • Improved User Experience: Virtual Agent can handle multiple topic switches in a single session and extract entities more flexibly, resulting in more natural and efficient conversations.
    • Reduced Maintenance: The need for ongoing training and updating of NLU models is minimized, as LLMs leverage plain language descriptions and generative AI capabilities.
    • Enhanced Versatility: Virtual Agent can perform additional language-related tasks such as generating case summaries and resolution notes, expanding the scope of automation and assistance.

    Large language models (LLMs) enable Virtual Agent to process user statements in plain language. Conversations that use LLMs can outperform those conversations that use Natural Language Understanding (NLU), with easier setup.

    How LLMs work in Virtual Agent

    Watch this video to learn about LLM topics in Assistant Designer Asset library.

    When you create a topic in Assistant Designer Asset library, you can select LLM as the model type for your topic whenever ServiceNow Otto for Virtual Agent is turned on. Virtual Agent then uses LLM generative AI to discover topics that match the user's intent.

    Unlike NLU topics, LLMs don't require models, intents, or keywords to be linked to the topic. LLMs can discover topics and perform language-related tasks, such as text generation for case summaries and resolution notes, without months of training on NLU models. Overall, you can create, configure, and deploy LLM topics faster than working with NLU.

    With LLMs, Virtual Agent can do the following:

    • Perform topic discovery without needing a singular declared intent in a given topic.
    • Find intents without backup keywords as in NLU modeling.
    • Extract entity values without prior mapping as in NLU modeling.
    • Handle multiple conversation topic switches in a single conversation session.

    For more information about LLMs, see Large language models on the ServiceNow AI Platform®.

    Topic discovery

    With LLM topic discovery, topic authors no longer need to create and maintain complex NLU models and intents with backup keywords. The LLM does all of the heavy lifting for you. The only requirement is a robust, plain language topic description on the Properties tab in Assistant Designer Asset library. The LLM uses this description to find the best topic match for the user utterance. If there are multiple potential matches, the user will see a list of topics to choose from.

    For example, if a user asks Virtual Agent to calculate ride share fees, the LLM matches user utterance with an existing topic that calculates the ride share with a tip.

    Entity extraction

    With LLM topics, the LLM has all the information needed to determine if an utterance has the information to fulfill a request. Unlike NLU models, you don't need to associate entities with a user input node or add nodeless NLU entities as input variables to a topic. The LLM simply finds the entity that most closely fits the user intent.

    Topic switching

    Topic switching is faster and easier with LLMs compared to NLU topics. The LLM processes your requests to change intent made in natural language, and activates the appropriate topic.

    For example, if you start a conversation by asking for a mobile phone, you don't have to cancel the order first or restart the conversation. Instead, you can ask Virtual Agent to order a laptop instead. Virtual Agent immediately switches from the mobile phone topic to the laptop topic. Topic switching can be done during a query, but not within a catalog ordering flow.

    Another example is when a user might ask a casual question or engage in small talk. The question might be unrelated to the original request. You can create small talk topics or set up small talk filters to help the Virtual Agent match and launch the appropriate conversation for the switched topic. For more information, see Create a small talk topic and Configure small talk filters.

    Additional resources