AI Search FAQ

[Loic1]

Looking for the Quick Start Guide? Head this way.

Check out the AI & Intelligence forum to get the best content about ServiceNow's AI products!

---

As of the Xanadu release

Table of contents:

Getting Started
Genius Results
External data & Connectors
Using AI Search in Portals

---

Getting Started with AI Search

Is AI Search available out of the box?

Yes. See the full quick start guide for more information about plugins, entitlements and activation.

What is needed to start with AI search? What kind of effort are we looking at? 

AI Search comes with some OOB configurations, covering a majority of use cases for Knowledge and Catalog. For the full quick start guide, head this way.

Will we be able to enable AI Search in a PDI soon?

AI Search is not currently available for PDIs (Personal Developer Instances). You will have to request it in a sub-production environment.

On a fresh install should we start by using Zing to give a chance for machine learning to gather enough data to work properly?

AI Search learns based on interactions with AI Search - starting with Zing will not engage the ML for AI Search.

Does using the AI Search eliminate the need for the synonym dictionary overall?

It does not, synonym dictionary is still a very effective way for searching to understand your organization’s synonyms and acronyms, customers can configure synonym dictionaries on the AI Search Profiles.

Is AI Search more flexible to configure the weight of results? More than what Zing had to offer?

The AI Search relevancy is calculated without "weight", it offers a better default relevancy model than Zing and uses Machine Learning to improve relevancy over time based on the users' behaviors. Beyond that, AI Search also provides a feature called “Results Improvement Rules” which allow customers to boost, deboost, promote or block search results based on specific scenarios.

Is search sources where you can indicate specific knowledge bases or catalogs to return in a portal?

Yes, the AI Search > Search Sources do exactly that. The AI Search needs to be enabled on that portal (i.e. the checkbox 'Enable AI Search' is true).

Can I use a dot walking condition to filter my content on a search source (for example filtering KB based on Author.Active is true)?

Yes, you'll have to enable indexing of referenced table fields for the indexed source first, using the dot_walk_fields attribute.

Can I use AI Search to search for Incidents and Requests?

Yes, the Task [task] table and tables that extend it can be used as Indexed Sources, they just need to have a retention policy.

Does AI Search in mobile work in parallel to AI Search for the portal? For example, if an external search source is created once, it can then be related  to both portal and mobile?

Yes this is how this works. You can use different Search Application records linked to different sources to drive the behavior of your mobile apps and portal(s).

What's the difference between no_text_index and not_searchable?

See the answer here.

In “Results Improvement Rules” how does this part work: allow customers to boost, deboost, promote or block search results based on specific scenarios?

Yes, You can create your conditions to select what content to boost/block in the search results. More details here. It also applies to auto-complete resuls.

Are there any real/perceived delays in Search Signals being recorded and visible in the respective tables (sys_search_signal_event , sys_search_signal_result_event)?

Search Signals can take up to a minute to reflect in the respective tables, however, if a script blocker is being used, that will prevent the AI Search callback for signal processing and the Search Signal will not be captured.

Can you recommend any documentation on how to best actively manage AI Search after go live? I am referring to features like auto complete, type corrections, stop words, Promoted results, etc.

See this article.

Is AI Search also available through Virtual Agent?

Yes, see this article for more details.

Are there any tips for setting up a search for content that may exist in a PDF document that is attached to a knowledge article?

You can enable the indexing of attachments for a table with indexed source attributes (docs). When enabled, AI Search evaluates the relevancy including the attachments. As a best practice, only use attachments under 5MB and limited to 10,000 words.

Does AI Search read Meta fields in catalog items?

Yes. See this article for more details.

Search reporting options with AI search: Are the sys_search_event tables still used or are there new ones for reporting?

Yes, queries are still logged in that table. We recommend enabling User Experience Analytics to access OOB reporting. More details can also be found in the docs regarding Search signal tables.

If we have multiple languages in the instance, does the search pull up only what my language is set for articles, or all show articles that have different languages?

AI Search provides better relevancy in 6 languages by leveraging language-specific processing that Zing lacks. Because of this, by default, we only search in the user's session language. However, there is a mechanism to designate content as "global" to make it searchable by all available starting in San Diego, and a workaround to enable the same in behavior in Quebec/Rome. See this article for more.

Is there a module available to debug and better understand why a user might be getting a certain result with AI Search?

Yes, AI Search also released a store app “AI Search Advanced Management Tools”, after installing this store app, a preview UI will be available for customers to debug any specific search query and get explanations on search behavior.

Will enabling AI Search in non-production environments affect the search in the production environment?

No.

Some users are experiencing slowness to get search results or it takes long to get results. What do I do?

Performance issues of search are often caused by long-running user criteria. To find criteria causing issues, navigate to System Diagnostics > Stats > Slow Scripts, filter the Label with starts with user_criteria, sort by date.

Is it possible to invoke the AI Search from outside ServiceNow using an API?

It is possible to add external sources for your search via the ingest API, but do not offer an external query API.

Is it possible to create an external API for AI Search?

AI Search is the search for ServiceNow. We strive to create the best search experience within the platform and are continuously working to improve in this regard. There is currently no plan to offer an external query API.

---

Genius Results

Does the Q&A snippet only show the first 3 to 4 lines of the article?

Q&A will find the snippet within a longer Knowledge article that answers the user’s question. This is what gets displayed in the card so that the user doesn’t need to click through to the article and read the entire article.

Is there a capability to search the KB and bring back the relevant content and not the entire article?

Yes, this is the Q&A Genius Result.

Can you explain how genius results differ from pinning content (using keywords)?

We have three Genius Results types out of the box, supported by our machine learning capabilities: Catalog Item, People Search, and Q&A for finding answers to questions within longer Knowledge articles.
"Pinning" a result (technically we refer to this as "promoting" a result via Result Improvement Rule), lets you choose a specific result to appear first in the standard (non-Genius) search results when certain configurable trigger conditions are met.

In order to use the Q&A Genius Results, do we need to "train" the AI search?

Q&A Genius Results work out of the box without any configuration or training.

---

External data & Connectors

Can AI Search already combine information from multiple sources?

AI Search can combine information from multiple sources. The architecture is as described here, where connectors are configured or developed, and the information is indexed in the AI Search Index.

Is there a connector with Drupal sites?

Not yet, customers and partners can use AI Search REST API and Integration Hub Spokes to build a connector for it.

Can Search integrate with Workday and Alight?

Currently, we do not yet have connectors for Workday and Alight. Customers and partners can also leverage our API and Integration hub to index documents from Workday and Alight.

Where can I find all the connectors available?

Please go to the ServiceNow app store to find connectors from ServiceNow or our partners.

Does the search of external sources, like SharePoint, require the purchase of integrationHub, and thus, incur transaction costs?

Some integrations may require the purchase of plugins or an integrationHub license and external document ingestion is counted as an integration transaction. Please check with your Account team for more details.

Google search as a search source, is it possible?

Not really. Google and enterprise search are very different search use cases, they are designed, maintained, and supported very differently.

---

Using AI Search in Portals

Can I use AI Search in my portal?

AI Search is currently available for Service Portal, Employee Center (ESC), Consumer Service Portal, and Customer Service Portal. AI Search is used in the following widgets: Homepage Search widget, Typeahead Search widget, Faceted Search widget, Catalog Homepage Search widget, Knowledge Breadcrumbs widget. If these widgets were customized, they can be reclassified following this procedure.

Can I change the look and feel of the search results in my portal?

You can customize the look and feel of the AI Search experience in Service Portal by updating the relevant C... or create a CSS include to override the theming for AI Search in Service Portal.

Can I change the navigation URL when I click on a result?

See how to achieve this using the EVAM configuration in this detailed article. 

[Hugo Tristant1]

Does Q&A Genius Results works also for Knowledge block contained into Knowledge articles ?

According to my test it does not unfortunately. How to make this available ? When would this be available OOTB ?

[Gerard Dwan]

Hi Hugo, 

Unfortunately, that functionality isn't available today. Can you add it to the Idea Portal? 

[Sandhya V]

Hi,

What configuration is needed if we have to get  articles in Related Search Results based on the product using AI search?

[Gerard Dwan]

Hi Sandhya, 

AI Search isn't currently supporting contextual search (where Related Search Results come from). Would you mind opening an idea in the idea portal for this? 

[Hugo Tristant1]

Related to :

Does Q&A Genius Results works also for Knowledge block contained into Knowledge articles ?

According to my test it does not unfortunately. How to make this available ? When would this be available OOTB ?

Done: See following link.
https://support.servicenow.com/ideas?id=view_idea&sysparm_idea_id=303a7ede1b8d99501e579979b04bcb9d&s...

Please upvote

[Max Nowak]

@Gerard DwanWhen adding a task table as indexed source, would you recommend using field mappings to map description to text and short description to title? Or is that irrelevant?

[Gerard Dwan]

@Hugo Tristant1 That's correct. Yes, please upvote to have this considered. 

[Gerard Dwan]

@Max Nowak - I recommend including a text and title mapping for every indexed source. It helps with two things: 
1) Dynamic hit-highlighting and teaser text for display purposes 

2) Better relevancy behavior. The OOB relevancy model heavily weights title + text fields. 

Hope this helps! 

[Joakim St_ckel]

Hi! Lovely to see this!

I'm currently working on setting up some specific portal pages to be searchable. I've managed to setup the search source etc. and added a filter to specify which pages we want to be able to actually find.
Problem is though, they've gotten the same glyph/icon as the Knowledge Articles when searched for...

How do I change this? The search result currently only displays the short description as well, which is quite fine. But the glyph... How is this configurable?

[Gerard Dwan]

Hi @Joakim St_ckel - 

I see your question on this post and think it makes sense to respond there. 

[Jar]

Are there any plans to enhance the baseline AI Search analytics dashboard. Right now it feels very basic and it  would nice to be able group data by user specific information (e.g. country, location, department) and to also be able to export the data.

I know there are system tables we can build our own reports on but we've been told this could cause performance issues and ServiceNow would not guarantee to support us if we did encounter such issues.

[Shamus Mulhall]

Hi @Jar - These are certainly great ideas and we wouldn't want to lose them. Would you mind opening an idea in the idea portal for this? 

[Jar]

Hi @Shamus Mulhall - I would love to but I don't seem to have the necessary access to the idea portal. I'll have to check with our admin if I can get such access.

Cheers

[Racheal Wiafe]

hi team

The link to the architecture description is broken. Can you please share on additional info on Architecture of AI Search?

Related question:

Can AI Search already combine information from multiple sources?

AI Search can combine information from multiple sources. The architecture is as described here, where connectors are configured or developed, and the information is indexed in the AI Search Index.

[vivian1]

Hi @Gerard Dwan, I would love to know if ServiceNow considered the idea "Q&A Genius Results works for knowledge blocks"? Thank you!

[Gerard Dwan]

Hi @vivian1, We're introducing some big changes to Q&A Genius Results in upcoming releases. We haven't handled the knowledge block indexing for articles quite yet, but we are watching that idea. 

For context for people later, this is the link to the idea. Continue the upvoting! 

[Lisa Baremore]

Hello!

Does AI Search work in the Knowledge Hompage? We are testing in a dev environment and though we configured the kb_knowledge table to use AI Search as opposed to Zing search, it doesn't seem to be working in this UI. I can't find any documentation that specifically explains how to configure it here. We have knowledge search sources, etc. but what else would be missing? Thanks!

Thank you!

[Astha5]

Hi @Loic1 , @Gerard Dwan ,

If we enable AI search on the platform, does it also enable AI search on APIs. Here are some details for your context-

In our project we are using Knowledge Management and Catalog APIs to search for Knowledge and catalogues within ServiceNow from an external bot. Currently the search is keyword search, which is not bringing up responses in line with the Portal that has AI. The customer is asking to check if AI can be enabled for API search as well to get consistent response as portal search results.

Thank you

Astha

Thank you

Spoiler

[Gerard Dwan]

Hi @Lisa Baremore ,

AI Search does not support the Knowledge Homepage currently. It is supported in Service Portal based portals, Next Experience workspaces and global search, mobile, and virtual agent. 

Best,
Gerard

[Gerard Dwan]

Hi @Astha5 ,

There is not a query api for AI Search. The same use case is supported by using Virtual Agent and having AI Search as a fallback. 

Best,
Gerard

[RaghuBellamkond]

Hi @Loic1, @Gerard Dwan,

Can you please share some documentation or videos that explain the AI synonyms dictionary? I'm seeking guidance on how this dictionary operates and how we can enhance it to improve search results.

[Shamus Mulhall]

Hi @RaghuBellamkond 

You may find this article covering tuning AI Search helpful.  It does cover the use of synonyms as well.

https://www.servicenow.com/community/ai-intelligence-articles/ai-search-tuning-making-ai-search-anal...

[estherg]

Hi,
A new customer is very interested in AI Search and them have asked us two questions about it:
* What level of AI, AI Search has?
- Level 0: Deterministic
‍No required training data, no required testing data
- Level 1: Learned‍
Static training data, static testing data
- Level 2: Self-learning
Dynamic + static training data, static testing data
- Level 3: Autonomous (or self-correcting)
Dynamic training data, dynamic test data
- Level 4: Intelligent (or globally optimizing)
Dynamic training data, dynamic test data, dynamic goal

* On the other hand, it's needed that the customer creates synonyms dictionary or AI Search is able to find some using NLU/AI? For example if I search "laptop" I see "PC" for example, and it is not confugured in the dictionary of synonyms OOTB. How this works in ServiceNow? I don't have found any documentation about this in specific.

Thank you so much!

[Ashley Snyder]

@Lisa Baremore Only Zing Search is supported on the Knowledge Management v3 homepage. We are beginning to deprecate the Knowledge V3 homepage as of Washington DC and redirect new customers to the Knowledge Management Service Portal, which also only supports Zing search at this time. We do have it on our roadmap to support AI Search within the Knowledge Management Service Portal. Next Experience and Core UI fulfillers/agents can use AI Search within the Global Search menu to utilize AI Search when searching for knowledge articles.

[Brian Bakker]

@estherg 

Here are the answers we provided for the following 2 queries -

Query 1:
What level of AI, AI Search has?
- Level 0: Deterministic
‍No required training data, no required testing data
- Level 1: Learned‍
Static training data, static testing data
- Level 2: Self-learning
Dynamic + static training data, static testing data
- Level 3: Autonomous (or self-correcting)
Dynamic training data, dynamic test data
- Level 4: Intelligent (or globally optimizing)
Dynamic training data, dynamic test data, dynamic goal

Answer: Level 3: Autonomous (or self-correcting)
Dynamic training data, dynamic test data, because the search signals will include data on how search users interact with the search input field, auto-complete suggestions, facet and navigation tab filters, Genius Result answer cards, and search results. AIS uses data from these search signals to intelligently tune relevancy models on a continual basis, every 30 days. It is not Level 4: Intelligent (or globally optimizing), because these relevancy model tuning processes occur separately for each search profile, and so not done at a global scale, as each search profile could be searching different parts of the Platform, and so you want to tune the relevancy model based on the search signals generated for that search profile.

As per our documentation Machine learning relevancy in AI Search

Search signals and machine learning relevancy tuning
AI Search UX components record signals associated with user searches. These search signals include data on how search users interact with the search input field, auto-complete suggestions, facet and navigation tab filters, Genius Result answer cards, and search results. To learn more about how search signals are recorded and stored, see Search signals.

Machine learning relevancy uses data from these search signals to intelligently tune relevancy models on a continual basis. Every 30 days, AI Search computes a new version of each relevancy model, iteratively modifying its parameters and regression testing it against aggregated search signal data for the search profile. When this tuning process is complete, AI Search compares the existing and new relevancy models to see which one produces better matches for user search behavior as recorded in the historical signal data.

If the new relevancy model produces better results with the signal data, AI Search uses its modified parameter values to perform A/B testing evaluations of live search traffic for the search profile. These evaluations test individual parameter changes to verify that they produce better search relevancy.

If the new model outperforms the original model in both the historical search-match comparison and the A/B testing, AI Search sets it as the active relevancy model for the search profile, overwriting the existing relevancy model. The updated relevancy model remains in use until the next tuning cycle begins.

These relevancy model tuning processes occur separately for each search profile. Changes made to the relevancy model in one search profile don't affect relevancy models in other search profiles.

Query2:
On the other hand, it's needed that the customer creates synonyms dictionary or AI Search is able to find some using NLU/AI? For example if I search "laptop" I see "PC" for example, and it is not configured in the dictionary of synonyms. How this works in ServiceNow?

A. AIS only used NLU for "People Genius Results" and not on the search term used for searching. NLU was also used for "Catalog Genius Results", but we switched back to using a script, because the Catalog Item names can contain the same phrases used in the associated AI Search Genius Results NLU Model for AIS that produced inconsistent results.

If you use the Search Term "laptop", but you also want to search for "PC", then you would need to create a dictionary synonym, as it does not use NLU when performing an actual search. We recommend to review the Community Article on AI Search Synonyms.

[Brian Lancaster]

Any idea why AI Search is bridging back catalog items that are no longer active?

[Community Alums]

Hi @Gerard Dwan 

Good morning, I've a short question regarding AI Search.

Does AI Search use the feedback table part of article relevancy tuning? https://docs.servicenow.com/bundle/washingtondc-servicenow-platform/page/product/knowledge-managemen...

Context: We're streamlining the feedback mechanism for article to provide one way of feedback to users rather than 3 ways as provided, we would like to see, if this is anyway impacting the ai search relevancy, 

Thanks,

Bhanu.

[Prinssi]

Regarding Genius Results: Is there a system property or some other out-of-box setting that we can configure so that we can hide the Genius Results card instead of seeing the "No results generated" message?

If Genius Results cannot be generated, that card still takes up a notable amount of screen real estate.  I would rather not display this card, and go straight to the results, instead of displaying a lot of negative space on the page.

EDIT: For anyone else experiencing this issue, you need Washington D.C. Patch 5 per this KB: KB1647708: [WP4] AI Search - Search on a service portal displays a blank Genius Results card at the .... We are on patch 6, and the card no longer displays when there is not a Genius result.

[Kass3m]

@Loic1 Would it be recommended to use personas as search profile such as Employee search profile vs a manager search Profile?

[Brian Bakker]

@Prinssi 

This is a defect, and is being fixed.

[Brian Bakker]

@Kass3m 

Search Profiles are linked to Search Applications, and you should not be used as personas for your search profiles, as Content Security in AIS will determine what records are returned in the Search results. Search Profiles are used to store search experience settings for a Search Application. If you review the out-of-box Search Profiles, you see they will be linked to different Service Portals, or other applications that use AIS.

[Kass3m]

thanks @Brian Bakker so we should not put a Manager Search profile or HR Profile to enhance our AI search?

[Nisha15]

Hi
I have two queries if someone can help:
1. Can we show the profile photo here on search itself?
2. Can we add email and designation below user name(currently shows user).

[Brad_B]

We are preparing to upgrade to Xanadu.  I have two questions regarding AI search in Xanadu.

1) Will the knowledge category filter be available as a hierarchical list (as it was in Zing), instead of a flat list?  We want to show the list of knowledge categories as a tree structure with sub-categories nested under their parent categories.

2) Will AI search results include knowledge blocks?  In our current version, the knowledge blocks are not indexed with the article, so those articles are not displayed in search results.

[BenS24570419223]

What types of documents can AI search index from an external source? .doc .pdf .ppt ...etc. 
Can AI index videos or video transcriptions and meta tags and use those results as a source?

[Shamus Mulhall]

Here are the list of supported file formats

https://www.servicenow.com/docs/bundle/xanadu-platform-administration/page/administer/ai-search/conc...

AI Search has an ingest api that can be used, currently AI Search will not index video files, but a transcript and metadata can be indexed.

https://www.servicenow.com/docs/bundle/xanadu-api-reference/page/integrate/inbound-rest/concept/exte...

[pavan38]

Hi Gerard Dwan,

We have a business requirement for which we need a solution. Could you please let us know if it's feasible?

We are managing two instances, A and B, both configured with SSO. Although they share the same user base, the roles and permissions for these users differ between instances.

In Instance A's AI Search-enabled portal, we need functionality that allows a user to search for terms like "Outlook" and retrieves relevant knowledge articles from Instance B. Importantly, only those articles that the user is permitted to access should be displayed (with user criteria applied to the articles residing in Instance B).

Your insights would be greatly appreciated. Thank you.

[narramounik]

hello @Shamus Mulhall @Brad_B @Brian Bakker @Gerard Dwan
'Taxonomy - Q&A Genius Result' EVAM view configuration already exists in the EVAM view config bundle 'Taxonomy Portal Search Bundle' which was connected to EVAM definition 'ESC Portal Search'.

But in Employee center portal when we search with any knowledge article , genius results not showing, while the genius results working fine for catalogs.

[Kevin Paterson]

Hi 

With respect to stored search signals and the relevancy model training where does the training happen? 

What is the data flow from a governance perspective.

Is it isolated to our instance?  

Search signals

[Brian Bakker]

@Kevin Paterson 

Relevancy model training happens in our AI infrastructure in the same datacenter where your instance resides using encrypted communications . Once the training completes, the data is removed from the AI infrastructure, as we do not persist any customer data in our AI infrastructure, and the trained model is uploaded to the instance where it undergoes A/B Test Evaluation.

I hope this answers your questions.

Regards,

Brian

[Kevin Paterson]

@Brian Bakker 
Is there a specific date that the training happens as it says every 30 days in the support docs. So I can possibly check returned results after a training cycle?

https://www.servicenow.com/docs/bundle/washingtondc-platform-administration/page/administer/ai-searc...