Tracking successful searches in SN search bar

[DaveThibode]

I am curious how people are tracking if their users are getting successful and useful search results in ServiceNow. We are fairly new to SN. We do not have a lot of Knowledge articles; however we did make Employee Center pro our Intranet and the landing page for any user who opens a browser. We do have an API to SharePoint and when someone searches in SN, it also returns results from SharePoint. I can get a list of "Searched" items but how to I monitor if that is the results they expected? How are others tracking this and improving search results accuracy for their end users? 

Thank you

Dave

[Shahjay]

Hi Dave,

Since you're on EC Pro and getting SharePoint results inside ServiceNow, you're almost certainly on AI Search already (the SharePoint connectors only work with AI Search, not Zing). So you've got a lot more to work with than a list of search terms. The AI Search Analytics dashboard is where I'd start.

To get there: All > User Experience Analytics, click Launch Dashboard on your Employee Center portal, then the Search Analytics icon. It's part of the Advanced AI Search Management Tools app from the Store, which also needs the Usage Insights API app. You'll need ais_admin plus one of the analytics, portal analytics or web analytics viewer/admin roles. If the dashboard loads empty or won't load, check the doc page on configuring Service Portal to send analytics data. That's usually the missing step.

The lists I'd actually spend time on:

• Queries with no results is your content gap list. With only a few KB articles, this will probably be the most useful one early on. It's basically a backlog of articles people want and can't find.
• Queries with no clicks is your relevance list. Results came back but nobody picked one, so either the wrong content ranked first or the titles didn't look like the answer.
• Average click position tells you how hard people work to find things. If they're regularly clicking the 5th or 6th result, the content exists but isn't ranking well.
• Top clicked results shows what's really being used, including whether SharePoint content is winning over your own articles.

To improve things, work in the search profile your Employee Center uses (AI Search > Search Experience > Search Profiles). Add synonyms for the internal jargon people actually type. Use result improvement rules to boost or promote the right content for high-volume queries that keep missing. Run the problem queries through Search Preview before and after so you can see the difference. One gotcha: rule changes only apply once you publish the profile, and synonym dictionaries have to be published separately. Then watch those same queries over the next few weeks and see whether the no-click count drops.

A click isn't proof the user got the right answer, though. Someone can click, read, and still raise a ticket. For KB articles, article feedback fills some of that gap. SharePoint clicks take the user out of ServiceNow, so you lose sight of them after the click.

Also worth knowing: the dashboard only keeps 180 days and caches for an hour. If you want longer-term trends, you'd report on sys_search_event and the search signal tables. They're system tables, so reporting on them has to be enabled first.

[DaveThibode]

Thank you for the quick response. we have actually not turned on AI yet in our environment, as we just went live earlier in the year. we set up the service now. data fabric that connects to SharePoint and scans our SharePoint environment once a week, which allows us to search what is in SharePoint.