Create an AI Search Tuning evaluation

  • Freigeben Version: Australia
  • Aktualisiert 12. März 2026
  • 4 Minuten Lesedauer

    • Create an evaluation to measure two search applications' result relevancy for a specified set of search queries. Compare the performance of AI Search and Zing text search, or compare the performance of two different AI Search applications.

    Vorbereitungen

    The AI Search Tuning ServiceNow® Store application must be installed on your instance. For details on installing this application, see Install AI Search Tuning.

    Role required: ais_admin

    Warum und wann dieser Vorgang ausgeführt wird

    An AI Search Tuning evaluation lets you compare search result relevancy for two search applications. You can create an evaluation for either of the following use cases:
    • Compare search result relevancy for Zing and AI Search applications that have the same search sources
    • Compare search result relevancy for two AI Search applications that share the same search sources
    The evaluation includes the following settings:
    1. The two search applications to compare — a baseline application and a comparison application
    2. The set of search terms and the search language to use when evaluating search result relevancy for the two selected applications
    3. The search users assigned to rate and review search results for the selected queries in the two selected applications
    4. A deadline for completion of the rating and review stages

    Prozedur

    1. Navigate to All > AI Search Tuning > Admin Homepage.
    2. On the Start new tuning goal card, select View setup.
      The AI Search Tuning wizard appears.
    3. On the wizard's 1. Set up new evaluation page, specify basic parameters for your evaluation.
      1. In the Evaluation details section, enter a name and description for your evaluation.
      2. Enter a reviewer deadline date or select one using the calendar picker tool .
      3. In the Which search applications do you want to compare? section, select the appropriate search engine for your baseline search application.
      4. Select the baseline Zing or AI Search application for your relevancy comparison.
      5. Select the AI Search application that you want to compare with your selected baseline search application.
        Hinweis:
        You can only select an AI Search application that uses the same search sources as your baseline search application. If the selection list is empty, no evaluation is possible for the selected base system application.
      6. Wahlweise: To review a selected search application's configuration settings, select its View search application details link.
      7. Advance to the next page by selecting Continue to 2. Choose search terms.
    4. On the wizard's 2. Choose search terms page, specify the source for search terms you want users to rate and review in the two selected search applications.
      • To retrieve search terms in a specific language from the Search Event [sys_search_event] table, choose Select from logs and choose your desired language.
        Hinweis:
        The system filters out search terms that only include record numbers or that are 3 characters or less in length.
      • If you have the admin role and want to retrieve search terms from another table, choose Advanced Configuration, specify the table and column containing search terms, and optionally select Filter search terms with condition builder to define filter conditions that terms must satisfy.
      1. In the Maximum number of search terms field, enter a integer limit.
      2. In the Maximum suggested search results per application field, enter an integer limit.
      3. Advance to the next page by selecting Continue to 3. Refine search terms.
    5. On the wizard's 3. Refine search terms page, modify the set of search terms.
      1. Select Load search terms.
      2. Review the loaded search terms.
      3. Wahlweise: Use the search field and Search icon to refine the set of search terms displayed, or select the Clear input icon to view the full list of loaded search terms.
      4. Wahlweise: To remove unwanted search terms, select them, then select Delete and confirm by selecting Delete All.
      5. Wahlweise: To add new search terms, select Add new search terms, then enter your new terms, selecting Add after each, and finally select the Close dialog icon .
      6. Advance to the next page by selecting Continue to 4. Assign reviewers.
    6. On the wizard's 4. Assign reviewers page, select reviewers for your evaluation.
      1. In the Reviewers field, select one or more search users to rate and review search results for the evaluation.
        Hinweis:
        You can only select users with the sn_ais_data.tuning_reviewer or ais_admin roles as reviewers.
      2. Advance to the next page by selecting Continue to Review and Submit.
    7. On the wizard's Review and Submit page, save your evaluation.
      1. Confirm that your evaluation settings are correct.
        Hinweis:
        If a setting is incorrect, select the link for the appropriate wizard page, correct the setting, and then return to the Review and Submit page.
      2. Select Save, then select Got it.

    Nächste Maßnahme

    Notify your selected reviewers that they can begin rating and reviewing search results for the evaluation. For more details on this procedure, see Rate and review search results in an AI Search Tuning evaluation.
    Hinweis:
    Don't modify the configuration of either of your evaluation's search applications until review is complete. Mid-review changes to search application configurations can cause unexpected behaviors in the reviewing UI.