ServiceNow AI Search returning irrelevant Knowledge Articles

[AftabS]

Hi everyone,

I am trying to implement semantic search using ServiceNow AI Search for Knowledge Articles.

So far, I have configured the following:

• Created a Search Profile

• Configured the required search source

• Added Synonyms

• Configured Result Improvement Rules

• Configure Search Application

• Tested the search using the API/payload

However, I am facing an issue with the search results.

When I send my search query using the payload, the response contains some irrelevant knowledge articles along with the relevant ones. I only want the results that are actually relevant to the user's search query.

For example, if I search for a specific topic, the response contains the expected KB articles, but it also returns other KB articles that are not related to the search.

I have attached screenshots showing:

1. My Search Profile configuration

2. Search Source configuration

3. Synonym configuration

4. Result Improvement Rule configuration

5. The request payload

6. The response showing the irrelevant KB articles

Could someone please help me understand:

• Why are these irrelevant Knowledge Articles being returned?

• Is there a way to configure AI Search so that only highly relevant KB articles are returned?

• Should I use Result Improvement Rules, Search Source conditions, or some other AI Search configuration to filter out irrelevant results?

• Is there a relevance/threshold setting that I can configure for the API response?

Any guidance on how to troubleshoot and improve the relevance of the returned KB articles would be greatly appreciated.

Thanks in advance!