Why Search retrieval not giving proper results in AI agent?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
3 weeks ago
I set up a search profile to retrieve similar or duplicate incidents through search retrieval, and you can review the search source, its configuration, the indexed source, field mappings, and the semantic search setup. That said, the results are not coming back correctly and are consistently showing fewer than 10 records, even though there are 37 matching records. Also, the document matching threshold is set to 60.
.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
3 weeks ago
Hi @KM SN
It's worth separating the 37 from what the tool returns. That number is how many records your search source condition matches, so it's the pool available for indexing rather than a target for retrieval. Your tool is set to Hybrid with a document matching threshold of 0.6, so only chunks scoring 0.6 or above against the query come back. Fewer than 10 out of 37 is normal at that threshold.
You can confirm it with one test. Drop the threshold to around 0.3, run the same query, and see whether the count rises. If it does, retrieval is fine and the cut-off is doing the filtering, so you can tune it to a level that suits your use case.
Two other things you can check. Your Indexed Source carries its own condition, separate from the one on the Search Source, so if it's narrower then some of those 37 were never indexed. Comparing the two is quick. Second, your chunking configuration decides which fields get embedded, and if Short description and Description aren't included there, the semantic side has very little to match on and hybrid falls back mostly to keyword.
If records look completely absent rather than just low scoring, you can check the property sn_ais_assist.dpr_ingestion_completed. When it's false, ingestion hasn't finished and a reindex on the search profile helps. Opening sn_aia_tools_execution also shows you exactly what the tool returned on each run. One last thing, retrieval respects read access for the user the agent runs as, so it's worth confirming that user can see all 37.
Two useful references:
https://www.servicenow.com/community/ceg-ai-coe-articles/a-field-guide-to-evaluating-analyzing-and-d...
https://www.servicenow.com/community/now-assist-articles/ai-agent-tools-getting-the-most-out-of-your...
thanks
