Legacy - Using user search metrics with Virtual Agent

  • Release version: Xanadu
  • Updated August 1, 2024
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Legacy - Using user search metrics with Virtual Agent

    This feature enables ServiceNow customers to analyze user search behavior within the Virtual Agent application by leveraging user search metrics. These metrics help identify which queries are common, which searches yield no results, which results users select most often, and which results are not useful. This insight allows customers to enhance Virtual Agent’s effectiveness by improving search relevance and user experience.

    Show full answer Show less

    Note that the existing Conversational Analytics dashboard, which includes the User Search Metrics tab, is being prepared for deprecation. A new Conversational Analytics dashboard is available in the Platform Analytics experience, compliant with Government Community Cloud (GCC) and FedRAMP requirements. Migration guidance is provided for existing users.

    User Search Metrics Tab

    The User Search Metrics tab within the Conversational Analytics dashboard provides key data points related to user queries and search results, answering critical questions such as:

    • Which search results were not useful to users?
    • Which queries returned no search results?
    • What are the most common user queries?
    • Which search results were selected most frequently?

    Access to this tab requires either the chat analytics admin or chat analytics viewer role.

    Users can also utilize Remote Tables to view user search data in dashboards, providing additional flexibility in analytics.

    Search Technologies Used

    • AI Search: Uses intelligent query features to suggest relevant resources dynamically.
    • Contextual Search: Matches keywords to an existing database search for query resolution.
    • Topic Links: Embedded links in certain Virtual Agent topics that provide direct access to resources without backend search.

    User Search Metrics Data Explained

    The User Search Metrics tab visualizes summarized user search information, including:

    • Queries with no clicks: Queries where users did not select any search results, shown as a percentage of total such queries.
    • Queries with no results: Queries that returned no search results, expressed as a percentage of total.
    • Trending queries: The most frequently asked user queries, shown as a percentage of all trending queries.
    • Trending content: Search results most often clicked by users, expressed as a percentage of all trending search results.

    Practical Benefits for ServiceNow Customers

    By leveraging this feature, customers can:

    • Gain clear visibility into Virtual Agent search performance and user behavior.
    • Identify gaps where users receive no or irrelevant results.
    • Prioritize improvements to search results and Virtual Agent topics based on actual user data.
    • Plan for migration to the updated Conversational Analytics dashboard to maintain compliance and future support.

    By using user search metrics with the Virtual Agent application, you can see your users' queries and the results from their searches. With these metrics, you can identify the areas where you can improve Virtual Agent to offer more meaningful search results.

    Important:

    Conversational Analytics dashboard is being prepared for future deprecation. It will be supported until deprecation but will no longer be available for installation. A new Conversational Analytics dashboard in Platform Analytics experience, which meets the compliance requirements of Government Community Cloud (GCC), and thus FedRAMP authorized, is available. See Conversational Analytics dashboard in Platform Analytics experience.

    For details on the deprecation process, see the Deprecation Process [KB0867184] article in the Now Support Knowledge Base.

    If you are an existing user of this dashboard and want to migrate analytics data to the new dashboard, see Migrate data to Conversational Analytics dashboard in Platform Analytics experience [KB1651556].

    Use the User Search Metrics tab in the Conversational Analytics dashboard to see the key details about the queries that your users searched on and their search results. By using this tab, you can get the answers to the following questions:
    • Which search results were not useful to the users?
    • Which queries did not have any search results?
    • Which were the most common queries from users?
    • Which search results did the users select most often?
    Note:
    You can also use Remote Tables to view user-search data in Dashboards. For more information, see Remote tables.

    To access the User Search Metrics tab, you must have the chat analytics admin role or the chat analytics viewer role. The following example shows the data included in the User Search Metrics tab.

    Figure 1. User Search Metrics tab
    Virtual Agent Analytics user search metrics tab.

    AI Search

    Virtual Agent topics may use AI search to suggest helpful resources to the user. When a user enters a query, AI search provides the most relevant results by using intelligent query features. To know more about AI search, see AI Search . To integrate Virtual Agent with AI Search, see Improving the user experience with AI Search.

    Contextual Search

    Contextual search matches keywords to an existing database search to find potential matches to a query. Virtual Agent topics may use contextual search to resolve queries. For more information, see Maximizing code reuse with topic blocks.

    Topic Links

    Some Virtual Agent topics have links to useful resources that are embedded within them. When a user invokes one of these topics, the link shows up without having to do any back-end search.

    User Search information

    The following table describes the summarized User Search information that is visualized on the User Search Metrics tab.
    Table 1. User search data
    Visualization Description
    Queries with no clicks Queries that users did not click on any of the search results. Total % refers to the number of queries with no clicks expressed as a percentage of total number of queries with no clicks.
    Queries with no results Queries that did not yield any search result. Total % refers to the number of queries with no results expressed as a percentage of total number of queries with no results.
    Trending queries Queries that users most commonly asked. Total % refers to the number of trending queries expressed as a percentage of total number of trending queries.
    Trending content Search results that most users clicked on. Total % refers to the number of trending search results users clicked on expressed as a percentage of total number of trending search results.