Using user search metrics with Virtual Agent

  • Release version: Xanadu
  • Updated August 1, 2024
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Using user search metrics with Virtual Agent

    Using user search metrics within the Virtual Agent application enables ServiceNow customers to analyze users’ search queries and the outcomes of those searches. This insight helps identify opportunities to enhance the Virtual Agent’s search relevance and overall user experience. The metrics are accessible via the User Search Metrics tab in the Conversational Analytics dashboard.

    Show full answer Show less

    Key Features

    • User Search Metrics tab: Provides key indicators about user queries and corresponding search results, answering critical questions such as which results were unhelpful, which queries returned no results, the most common user queries, and the most frequently selected search results.
    • Access control: Requires chat analytics admin or viewer roles to view the User Search Metrics tab.
    • AI Search integration: Virtual Agent topics can use AI Search to deliver relevant, intelligent search results, improving user engagement by suggesting helpful resources.
    • Contextual Search: Matches keywords to database content to find potential query matches, enhancing topic resolution capabilities.
    • Topic Links: Some Virtual Agent topics embed direct links to useful resources, allowing users to access information without additional backend searches.

    User Search Indicators

    The User Search Metrics tab visualizes several key indicators to help optimize Virtual Agent performance:

    • Queries with no clicks: Queries where users did not click any search results, measured as a percentage of total no-click queries.
    • Queries with no results: Queries that returned no search results, measured as a percentage of all no-result queries.
    • Trending queries: The most frequently asked queries by users, shown as a percentage of total trending queries.
    • Trending content: Search results that users clicked on most often, represented as a percentage of total trending search results.

    Practical Benefits for ServiceNow Customers

    By leveraging these metrics, customers can precisely pinpoint where the Virtual Agent’s search experience can be improved—whether by enriching the knowledge base, refining AI search configurations, or adjusting topic content. This leads to more meaningful search results, increased user satisfaction, and more efficient resolution of user queries through the Virtual Agent.

    By using user search metrics with the Virtual Agent application, you can see your users' queries and the results from their searches. With these metrics, you can identify the areas where you can improve Virtual Agent to offer more meaningful search results.

    Use the User Search Metrics tab in the Conversational Analytics dashboard to view key indicators about the queries that users searched on and the corresponding search results. By using this tab, you can get the answers to the following questions:
    • Which search results were not useful to the users?
    • Which queries did not have any search results?
    • Which were the most common queries from users?
    • Which search results did the users select most often?

    To access the User Search Metrics tab, you must have the chat analytics admin role or the chat analytics viewer role. The following example shows the data included in the User Search Metrics tab.

    Figure 1. User Search Metrics tab
    Conversational Analytics dashboard user search metrics tab.

    AI Search

    Virtual Agent topics may use AI search to suggest helpful resources to the user. When a user enters a query, AI search provides the most relevant results by using intelligent query features. To know more about AI search, see AI Search . To integrate Virtual Agent with AI Search, see Improving the user experience with AI Search.

    Contextual Search

    Contextual search matches keywords to an existing database search to find potential matches to a query. Virtual Agent topics may use contextual search to resolve queries. For more information, see Maximizing code reuse with topic blocks.

    Topic Links

    Some Virtual Agent topics have links to useful resources that are embedded within them. When a user invokes one of these topics, the link shows up without having to do any back-end search.

    User Search information

    The following table describes user search indicators visualized on the User Search Metrics tab.
    Table 1. User search indicators
    Visualization Description
    Queries with no clicks Queries that users did not click on any of the search results. Total % refers to the number of queries with no clicks expressed as a percentage of total number of queries with no clicks.
    Queries with no results Queries that did not yield any search result. Total % refers to the number of queries with no results expressed as a percentage of total number of queries with no results.
    Trending queries Queries that users most commonly asked. Total % refers to the number of trending queries expressed as a percentage of total number of trending queries.
    Trending content Search results that users clicked on the most. Total % refers to the number of trending search results users clicked on expressed as a percentage of total number of trending search results.