Hybrid Search(enhaced chat)Field Mapping, Semantic Index configuration & Knowledge Graph Dependency

SheljaS · 3 weeks ago

Hi Everyone

We are using Enhanced Chat in Patch 7 and I would like clarity on Index Source configuration, especially around Field Mapping, Semantic Index configuration, and Related List settings.

Context

We are on Enhanced Chat
Index Source is already configured
Patch version is Patch 7
Users run natural language searches such as
How many incidents were raised last month

Observations

In the Index Source configuration, the following options are available
Field Mapping
Semantic Index configuration
Related List configuration

After reviewing the documentation, the purpose of these configurations is still not very clear, especially in the context of Enhanced Chat.

Current Behavior

When Knowledge Graph is not configured, Enhanced Chat does not return incident data for prompts like
How many incidents were raised last month

After configuring Knowledge Graph, the same prompt returns correct incident information.

Clarifications Needed

I would like to understand the following points

The role of Field Mapping in Enhanced Chat in Patch 7
The role of Semantic Index configuration in Enhanced Chat
Whether Field Mapping and Semantic Index are mandatory or optional
Whether these configurations have a direct dependency on Knowledge Graph
Whether Enhanced Chat requires Knowledge Graph to return structured or analytical data such as incident counts
Whether there is any supported way to get these results without configuring Knowledge Graph
If yes, what is the minimum required configuration
If no, whether Knowledge Graph is mandatory for such use cases

I have already gone through the official documentation but could not find clear explanations or examples that cover this behavior.

Any clarification, documentation references, or real world implementation experiences would be very helpful.

Thank you

Naveen20 · 3 weeks ago

Field Mapping — Controls how AI Search treats specific fields (title, text, sort_by_date) for relevancy ranking and result highlighting. Optional; all fields are indexed by default. It improves search result quality but isn't required for Enhanced Chat to function.

Semantic Index — Enables vector-based (meaning-aware) search instead of just keyword matching. Also optional; enhances retrieval accuracy for natural language phrasing but is still a document retrieval mechanism.

Why Knowledge Graph is needed for your use case — "How many incidents were raised last month" is a structured data aggregation query, not a document search. AI Search (Field Mapping + Semantic Index) finds matching records — it cannot count, aggregate, or filter by date ranges. Knowledge Graph enables natural language queries against structured ServiceNow data conversationally.

Bottom line:

Field Mapping and Semantic Index → optional, improve search/retrieval quality, no dependency on Knowledge Graph
Knowledge Graph → mandatory for analytical/aggregation queries (counts, status lookups, "who is my manager" type questions) via Enhanced Chat
No supported OOTB alternative exists to get incident counts in Enhanced Chat without Knowledge Graph

Minimum setup for KG NLQ:

Activate Knowledge Graph plugin
Set sn_ais_assist.enable_knowledge_graph_nlq = true
Set sn_ais_assist.kgnlq_schema_name to your graph (OOTB Now_user_graph_nlq or a custom one covering Incident data)

The OOTB graph is user-centric — for incident analytics, you'll likely need to extend or create a custom graph schema that includes the Incident table.

SheljaS · 3 weeks ago

How can I confirm whether all fields are indexed by default? Is there any documentation that mentions this? I also want to confirm one more thing: if I add a new field to an out‑of‑box table and configure the field settings and mapping—for example, if I add an Opened by field—what will be the result? Is it necessary to add it in this way? Additionally, if I want this field to appear in AI Search results, do I need to add it to the Indexed Source in the Search Source portal? Finally, could you please explain how AI Search works—for example, when a prompt is entered, how it interacts with the Knowledge Graph and then retrieves data from the Indexed Source?

SheljaS · 3 weeks ago

How can I confirm whether all fields are indexed by default? Is there any documentation that mentions this? I also want to confirm one more thing: if I add a new field to an out‑of‑box table and configure the field settings and mapping—for example, if I add an Opened byfield—what will be the result? Is it necessary to add it in this way? Additionally, if I want this field to appear in AI Search results, do I need to add it to the Indexed Source in the Search Source portal? Finally, could you please explain how AI Search works—for example, when a prompt is entered, how it interacts with the Knowledge Graph and then retrieves data from the Indexed Source?

Naveen20 · 2 weeks ago

1. Confirming "all fields are indexed by default"

A ServiceNow employee confirmed: "All fields are indexed by default. You need to specify fields you do not want to be indexed." Additionally, ServiceNow's own KB best practices doc states: "All fields on the document are considered for document retrieval, however the relevancy model will weight these items more in its scoring: KB Number, Title, Content, Keywords."

The official product doc reference is: AI Search > Field Settings for AI Search (Zurich bundle: docs.servicenow.com/bundle/zurich-platform-administration/page/administer/ai-search/concept/field-settings-ais.html

2. Adding a custom field (e.g., Opened By) to Field Settings & Mapping

If you add opened_by to Field Mapping, you're not making it "searchable" — it already is by default. What you're doing is one of these:

Mapping it to a special role like title, text, or sort_by_date — which changes how AI Search weights and displays it.
Marking it not_searchable — which excludes it from text-based search matching.
Enabling dot-walk indexing — for reference fields, you need to enable indexing of referenced table fields using the dot_walk_fields attribute if you want to search by the underlying field values (e.g., searching by the opened_by user's name rather than just the sys_id display value).

So for opened_by specifically: the display value is already indexed. If you want to search by the user's email, department, etc., you'd need searchable_dot_walk_fields or dot_walk_fields in Advanced Configuration. Simply adding it to Field Mapping without a map_to value or attribute change has no practical effect.

3. Making a field appear in AI Search results

There are two separate concerns:

Searchability (can the field be matched against a query) — handled by the Indexed Source and its field settings. Already enabled by default.
Display in results (can the field value show up in the search result card) — handled by the EVAM (Experience, View, and Model) configuration tied to your Search Application. Custom fields in EVAM are for items not directly on the table, using ai_search_teaser_title and ai_search_teaser_text for highlighted results.

You do NOT need to re-add the field to the Indexed Source for it to be searchable. But for it to display in the search result UI, you configure the EVAM view template in your Search Application Configuration.

4. How Enhanced Chat processes a prompt (simplified flow)

When a user types a prompt in Enhanced Chat, the pipeline works roughly like this:

Step 1: Intent classification — The LLM determines whether the query is a search/retrieval question (e.g., "How do I reset my password?") or a structured data question (e.g., "How many incidents were raised last month?").

Step 2a: If retrieval → AI Search pipeline

Query goes through the Search Profile and its associated Search Sources
AI Search performs lexical + semantic matching against the Indexed Sources
Field Mapping determines relevancy weighting (title/text fields rank higher)
Semantic Index (if configured) enables meaning-based matching
Results are passed to the LLM, which synthesizes an answer (RAG pattern)

Step 2b: If structured data query → Knowledge Graph NLQ

When an employee asks something like "Are there incidents affecting payroll?" the AI interprets the question through the Knowledge Graph, which translates questions into structured logic.
The KG schema maps entities and relationships, converting natural language into a structured query against ServiceNow tables
Results (counts, records, relationships) are returned to the LLM for response generation

Step 2c: User context enrichment (if enabled)

Knowledge Graph can also inject user context (location, department, manager) into search queries to personalize results

The key takeaway: AI Search and Knowledge Graph are parallel paths, not sequential. AI Search handles unstructured retrieval; Knowledge Graph handles structured data querying. Enhanced Chat routes to one or both depending on the prompt intent.