Set up AI Search before you start with Now Assist. Why would search come before AI?

kotiswaroop · 4 hours ago

1. 🎯 Why AI Search comes BEFORE Otto

Matt Dodd (ServiceNow), "Getting ready for Now Assist / ServiceNow AI":
- "AI Search is at the heart of Now Assist"
- "it's advisable to review and activate it prior to attempting to implement Now Assist."
Otto's context comes from one of three places:
- Trained: what the model learned. Great at language, but generic
- Retrieved: fetched from your instance when someone asks
- Given: the record in front of it
💡 If it wasn't retrieved or given, it was guessed.
AI Search does the retrieving, so the quality of your search setup shows up in the quality of Otto's answers
ServiceNow already helps: the AI Search RAG pipeline rewrites long questions, retrieves, re-ranks, then responds
👉 Your job: the right content, in the right front door, for the right user

2. 🗺️ The map: one engine, many front doors

Indexed sources: the tables AI Search indexes (Knowledge, Catalog Items…). Semantic indexing is set here
Search sources: filtered slices, e.g. published knowledge that hasn't expired
Search profiles: a group of search sources, plus settings
Front doors (search application configurations): portal, global search, workspaces, Virtual Agent, Otto panel
💡 Each front door can point to a different profile.
When results differ between two places:
- Check which profile each front door uses first
- The docs suggest checking that your chat assistant "uses the same search configuration as your portal's search"

3. 🚪 Inside a front door: what each part decides

Search application configuration: the front door's own search settings
- Docs: it "contains a reference to a search profile, along with settings that affect how search results are displayed to users of the application."
- On my lab: Search Profile · Enable Typo Handling · Genius Results Limit · facets · sort · auto-complete
Search profile: how results are generated
- Docs: "Search profiles contain settings for a search experience, including synonyms, stop words, typo handling, Genius Results, and result improvement rules."
- Plus: search sources (what can be searched)
Assistant (chat front doors only): which tool answers
- It picks a KB answer, a catalog item, a skill or an AI agent
💡 The profile decides what can be found. The assistant decides what gets used.

4. 😅 Typo handling: "Monday" became "money"

What happened on my lab
- I added new knowledge articles
- Searched "…every 10 minutes since Monday"
- AI Search changed "Monday" to "money"
Why, per the docs
- The typo handling dictionary "derives a list of auto-correction terms from indexed content found in search sources linked to its search profile."
- "By default, typo handling dictionaries refresh every 24 hours."
- My articles were brand new, and the word list refreshes on a schedule
The fix
- The refresh "has the same effect as republishing the dictionary's linked search profile."
- I republished the profile, changed no setting, and "Monday" stayed "Monday"
Also worth knowing
- Not corrected: words of 1–2 characters, ALL-CAPS words, anything with a digit (record numbers and codes are safe)
- Users see the corrected term and can search for their original term instead
- You can exclude a term from the dictionary
- Switch it off for one front door: Enable Typo Handling

5. 🔧 How to configure it right: 8 checks

Semantically indexed?
- "Hybrid search applies to any indexed source that's included in the semantic index."
- Other tables: configure semantic indexing, then "you must reindex content from that indexed source."
Hybrid actually on?
- On by default on a first install of Otto for AI Search
- "Upgrading an existing installation doesn't change current hybrid search settings."
- For record numbers or codes, keyword search may give more predictable results
Did you publish?
- "To make a search profile's settings active in search queries, you must publish it."
- Synonym dictionaries publish separately
- Remember Monday: publishing also refreshes the typo words
Know what synonyms do
- "…semantic vector search doesn't rely on your synonym dictionaries to find term equivalences."
- Synonyms still help the keyword side
Write for retrieval (Ashley Snyder, knowledge best practices)
- "Focus on one topic for each article, and cover it fully"
- "The LLM can't click links or go to other pages to summarize info"
- "Minimize the use of table formatting"
- Semantic chunks default to 250 words
Attachments
- "The maximum file size for binary files is 25 MB"
- Keyword indexing "processes up to the first 1MB of text."
- Test with your own files
Keep the wrong content out
- Result improvement rules: boost, block, promote
- AI CoE article: "Review promoted content quarterly to ensure it remains relevant."
Test as a real user
- Applied: ACLs, Before Query business rules, domain separation, user criteria
- Not supported: "scripted field-level ACLs or conditional field-level ACLs"

6. 🧪 See what Otto was given: Search Preview

Path: All › AI Search › Search Query Settings › Search Preview (New)
Pick the search application of the front door you care about
Ask a real user question
Open Genius Results › LLM Context, which "shows a list of all the search results that were sent to the LLM"
My lab: the right article was sent, plus 4 other articles and 10 catalog items
Search as user: see what a colleague sees
💡 Answers vary. Judge what was retrieved and sent, not one answer's wording

7. 🤖 Search found it. Did Otto use it?

The test: same question, in the real Otto panel, as an agent user (itil role)
Run 1, settings as they were
- AI Search ranked the right article first (re-ranker score 0.989)
- Otto started the "Wrap-up and resolve incident" AI agent workflow and asked for an incident number
- Copilot mode, so nothing changed without approval
Run 2, "Prioritize AI agents during skills discovery" off
- Otto picked Get Help, which answers from ServiceNow product documentation, not your KB
- On my lab it had no active endpoint, so it failed
Run 3, Get Help deactivated
- "Searched for relevant documents and actions" → "Generated the response"
- The source was the GlobalConnect article. Every step matched it
💡 The search was right all along. The front door needed two settings.
The setting, per the docs
- It lets "AI agents take priority over other assets, such as knowledge bases and Q&A modules, when the assistant discovers skills."
- "By default, all chat features, except web search mode, are turned on."
- Current docs: the assistant's chat features. On my Zurich lab there was no screen, so I changed it on the assistant's configuration record

8. 💡 Why it's designed this way (and that's a strength)

These assistants are built to get work done, not only to answer
"When you query a Virtual Agent, it reasons, plans, and executes across a wide variety of options."
Docs example: answer from the knowledge base, then send an email with the details. A search box can't do step two
Release notes: agent priority helps "steer users toward the AI agent experience instead of a static article."
👉 These were the settings on my lab. Decide per front door, rather than switching them off everywhere

9. 🚪 Four questions for every front door

Who uses this door? Information, or getting work done?
What can it discover? KB answers, catalog, skills, AI agents, Get Help
When two things overlap, which one wins?
Did you test it as a real user? Read View AI Steps and the sources

💡 Neither choice is wrong. Just make it on purpose.

✅ The verdict: 8-point checklist

Know your map: front door · profile · assistant
Index what matters: semantic coverage, then reindex
Check hybrid, especially after an upgrade
Publish: profiles, dictionaries and the typo words
Write for retrieval: one topic, fewer tables
Curate: keep outdated content out
Test as a real user: Search Preview AND the real front door
Set each front door on purpose: information or getting work done

Set up the search before you judge the AI. Grounded answers start with what search found.

🎥 Watch the full walkthrough: https://youtu.be/PJ5DQsd8YVE 💬 Over to you: which of these did you find on your own instance?

📚 Sources

ServiceNow Docs:

ServiceNow Community: