Wildly Inaccurate Results Using NowAssist Searches
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
4 hours ago
We recently enabled Now Assist for our fulfillers and, overall, many of the core capabilities are working as advertised. Features such as record summarization, content generation, and email generation have generally been well received by our teams.
Where we're running into challenges is with the Now Assist Panel's search and retrieval capabilities, particularly when users attempt to query ServiceNow data using natural language.
A few examples:
- When asking Now Assist to return remediation tasks associated with a specific Configuration Item (CI), it consistently returns records tied to different CIs than the one requested.
- When asking for incidents I've opened in the last 12 months, it may return problem records instead of incidents, or it appears to use my name in an unrelated field as part of the search criteria(see attached).
- In general, the results often seem disconnected from the intent of the prompt and can be difficult to trust without manually validating the returned records.
We're trying to better understand whether this is expected behavior, a configuration issue, or simply an area where prompt engineering is required.
For those who have successfully deployed Now Assist:
- Have you found ways to improve the relevance and accuracy of record retrieval from the Now Assist Panel?
- Are there specific configurations, indexing considerations, data source settings, AI Search tuning, or or other configuration/techniques that made a noticeable difference?
- Have you identified prompt patterns that consistently produce better results?
- Are there any known limitations with querying transactional data (Incidents, Problems, Tasks, Vulnerabilities, etc.) that we should be aware of?
We're currently evaluating adoption across a broader user base, and the search experience is one of the biggest areas preventing wider confidence in the tool.
I'd appreciate any lessons learned, best practices, or troubleshooting recommendations from others who have been down this path.
