Now Assist Cache for AI Search

[Lisa71]

Per this document https://www.servicenow.com/docs/bundle/zurich-platform-administration/page/administer/ai-search/conc... if we enable the second level cache, it won't trigger Now Assist for searching? It will still search from the search sources besides the 2 level cache, just don't use Now Assist LLM service to generate the explanation in the upper side of the result, right? Thanks. 

[rpriyadarshy]

Hi Lisa

Yes

• AI Search for Now Assist Q&A Genius Results uses two query-time caches to improve search performance and efficiency.  Also lowering Now Assist entitlement consumption.
• The first-level cache stores key-value pairs in memory for fast access, matching search queries and Knowledge article sys_ids to generated summaries.
• The second-level cache is a table indexed for AI Search, storing cache entries with semantic vector search, allowing retrieval of answers even if the search query isn't an exact match but is similar in meaning.
• Caching benefits include faster response times, reduced consumption of Now Assist entitlements, increased likelihood of returning answers, and improved consistency for similar searches.
• Content security restrictions ensure that cached results are only returned for Knowledge articles the user has access to.
• Cache operational modes can be set to "off" (first-level cache and Now LLM Service), "offline" (first and second-level caches, no LLM Service), or "online" (not detailed in the provided documents). Administrators can configure these modes using the sn_ais_assist.semantic_cache_mode system property.
• The Update Semantic Cache scheduled job runs daily to populate the second-level cache with frequent queries and purge unused, unpinned entries older than seven days. Frequently used answers can be pinned to prevent purging and improve performance.

Regards

RP