LLM topic testing

Preview, test, and debug topics that use large language models (LLMs) in the Virtual Agent Designer chat window.

You can test your topic's functions by running your conversation in a chat test window as you work. The web (Service Portal) chat client is the default test window.

Large language model (LLM) topics are faster and easier to test compared to Natural Language Understanding (NLU) topics. For example, you don't need to retest your topic after updating utterances and retraining models as with NLU topics.

If you're using the Virtual Agent integrations with third-party messaging apps, elements in your conversation might appear differently in third-party messaging applications. Test your conversations in any third-party applications where you want to deploy Virtual Agent.

Note:

If the Now Assist panel or Microsoft Teams is configured for your environment, preview options for those channels display in the Test button list. Select Preview in Now Assist panel or Preview in Microsoft Teams in the list to test your topic in those environments.

Testing your LLM topic in the chat test window

Figure 1. LLM test options

Test LLM topics on the Virtual Agent Designer canvas by working with the Test options on the topic header bar. When testing from a single topic, the Include topic discovery option must be selected to work with the assistants. If the topic is associated with multiple LLM assistants, use the Assistant drop-down list to select which assistant you want to test. For more information about LLM assistants, see LLM assistants.

Note:

If your topic is missing any necessary materials, an incomplete badge appears in the corner of the flow diagram pane. A yellow or red warning badge also appears next to each node. The incomplete badge lists the total number of issues, while the local warning badges show how many issues are found in each node. If you select Test when there are issues, an Issues window opens with the number of issues present along with the details of each issue. These details include a full description and a hyperlink to each incomplete item.

Alternatively, you can test active (published) LLM topics on the Topics page. Use the LLM Assistant filter to restrict your topics to only topics associated with that assistant, and then select Test active topics. If you don't use the LLM Assistant filter, select Test LLM assistant topics from the Test active topics drop-down list. When using the Test active topics option or sub-options from the Topics page, topic discovery is enabled, so it's not listed as an option. When testing from the Topics page, the Assistant drop-down list appears in the chat test window. You must have previously established at least the default Now Assist in Virtual Agent assistant to see the Assistant drop-down list. When a topic is associated with just one LLM assistant, the Assistant drop-down list defaults to that assistant name. If you have multiple assistants, select which assistant you want to work with using the Assistant drop-down list. For more information about creating multiple LLM assistants, see Manage LLM virtual agents on the Assistants screen.

The chat test window opens in the Now Assist in Virtual Agent web client. The Assistant drop-down list and Include topic discovery option might display depending on where you originated the test from and if you have established any LLM assistants.

You can provide feedback to each utterance made by the LLM by selecting from the like thumbs-up icon () or dislike thumbs-down icon () options that appear when you hover over an utterance. All LLM-enabled user inputs aside from the Input Collector have these feedback options.

Figure 2. LLM utterance feedback options

The chat test window also displays adjoining tabs that provide details about your topic as you test it. The following tabs are available when testing LLM topics:

• Analyze test phrases - Results for topic discovery based on your input.
• Prompt discovery - List of all the prompts in the topic that are sent to the LLM.
• Variables - List of all the variables used in the conversation, such as input and Live Agent variables.
• Edit variables - Options for editing the variables used in this topic.
• Context - Options for specifying the context (using context variables) in which a topic is run.
• Logs - List of the processing performed.

Figure 3. Example LLM testing chat test window and test tabs

The following tabs appear when you test their related topic types on the Virtual Agent Designer canvas:

• The Analyze test phrases, Prompt discovery, Variables, and Logs tabs appear for all LLM topic types.
• The Edit variables tab appears for topic blocks and custom controls.
• The Context tab appears for topics, setup topics, or small talk topics when you select Include topic discovery.

If you run tests from the Topics page, the test window shows only the Analyze test phrases, Variables, Context (available by default, with no Include topic discovery option), and Logs tabs.

Analyze test phrases tab

When you test LLM-enabled topics, results for topic discovery appear based on your input. When you input a test phrase, you can see a variety of search results when Genius Results are enabled including skill (topic) discovery, Knowledge Base (KB) articles, and catalog items. Under the skills search results, variables and values may also be listed (such as the variable @laptop_make and the value macbook), depending on the topic. These Genius Results only appear if you’re testing a published topic and have selected the Include topic discovery option. If the Include topic discovery option appears inactive, publish the topic to work with topic discovery. A Search indexing in progress message might appear, but you can still test while the search indexing runs although topic discovery might not be updated. For more information about how Genius Results work, see Now Assist in AI Search.

Additionally, when testing active LLM assistant topics from the Topics page, you see skills results for Semantic search. Semantic search analyzes the meanings and context of your search terms and uses that information to find results with similar meanings. It improves search recall by interpreting natural language to more accurately reflect your search's intent. If semantic search is deactivated for topic discovery testing, those results are not displayed. For more information about semantic search, see Semantic vector search in AI Search.

Figure 4. Example Analyze test phrases tab for LLM topics

Prompt discovery tab

If your topic contains any prompts that are sent to the LLM, they’re shown in the Nodes category. The Nodes category lists LLM-enabled user input nodes on the canvas, their field names and types, and their contents.

You can modify the content of any of the prompts to develop optimal prompts for the conversation. Each prompt's status starts at Original prompt. If you alter a prompt's content, its status changes to Modified prompt and a revert icon () appears next to it. The total number of prompts modified is listed at the bottom of the tab next to a Save to topic button. To test the modified prompts, select Apply and restart. To undo changes to a prompt, select the revert icon next to the prompt. Alternatively, undo all changes by selecting Revert that activates when you modify any prompts.

Note:

If you select Save to topic, all changes are saved and all Modified prompt messages reset to Original prompt.

Figure 5. Example Prompt discovery tab

Figure 6. Example Prompt discovery tab with a modification

Variables tab

The Variables tab displays a list of all the variables used in the conversation and the associated values captured as the conversation progresses, so that you can follow along. A conversation can have these variable types:

• Input variables
• Script variables
• Live Agent variables
• Variables passed between a calling topic and topic block

The list is separated into sections by variable types. The following example shows the Input variables section. Notice that for the static list control, both the display label and value are captured for the selected choice.

Figure 7. Example list of input variables

The following example shows the Input variables section for the grouped list control. This variable information appears similar to the static list control, but the variables are separated by each group of the grouped choice.

Figure 8. Example list of grouped choice variables

Edit Variables tab

When testing topics, topic blocks, and small talk topics, you can edit the variables found in the topic's nodes.

Context tab

The Context tab appears when you’re testing topics, setup topics, or small talk topics, to specify a different context for the chat. Choose a context variable from the list. The variables contain contextual information that can be used to determine topic intent or control how chats are routed to live agents. For example, you could select portal from the list of variables and enter the portal name IT Express. The Context tab is unavailable when creating test cases.

For more information about defining context variables, see Configure context variables for storing chat-related information. For more information about live agent variables that are included with Virtual Agent, see Live agent chat context variables.

Figure 9. Example Context tab

Next steps

When you're done testing your topic, close the test chat window. If necessary, use the test information to adjust your topic to perform more accurately. For example, the results on the Analyze test phrases tab may return low scores or Unsure or Unknown confidence ratings. Improve scores by updating the topic description or instructions in the LLM-enabled user input nodes.