Semantic index configuration for indexed sources
Summarize
Summary of Semantic index configuration for indexed sources
ServiceNow’s AI Search offers a generalized Retrieval-Augmented Generation (RAG) framework that enables administrators to configure semantic indexing settings for records indexed from ServiceNow AI Platform® tables. These configurations determine how content from indexed sources is processed and indexed to support semantic vector search, which improves search relevancy by finding results with meanings similar to the search terms.
Show less
Key Features
- Indexed Source: A reference to an existing indexed source containing field values or attachments to be semantically indexed.
- Embedding Models: One or more embedding models define how content is encoded into vector representations, enabling semantic similarity matching during search.
- Chunking Strategy: Text from indexed fields and attachments is broken into smaller chunks, which helps reduce search load and enhances context and relevancy for semantic vector matches. Chunks can span sentences and paragraphs.
- Semantic Index Fields: These specify the fields or attachments from the indexed source to be semantically indexed, including the order in which fields are evaluated, allowing precise control over the indexing process.
Activation and Practical Use
The semantic index configuration capability is provided by the AI Search Semantic Controller plugin (com.glide.ais.semanticsearch), which is automatically activated when any Now Assist application is installed. To confirm activation, navigate to All > AI Search > AI Search Index > Indexed Sources and check for the Semantic Index Configuration related list on an indexed source record.
Administrators can create multiple semantic index configurations per indexed source, but should be aware that each additional configuration adds indexing performance overhead. Proper configuration allows tailored semantic indexing that enhances search accuracy and relevance for users leveraging AI Search capabilities.
The AI Search generalized RAG (Retrieval-Augmented Generation) framework offers a streamlined way to configure semantic indexing settings for records indexed from ServiceNow AI Platform® tables.
Semantic index configuration overview
AI Search admins can configure semantic indexing settings for an indexed source. These settings specify how AI Search indexes content from the indexed source for use with semantic vector search. The group of semantic indexing settings for a particular indexed source is called a semantic index configuration.
- Indexed source
-
A reference to an existing indexed source with field values or attachments that you want indexed for semantic vector search.
For more information on indexed sources, see Indexed sources in AI Search. To learn more about semantic vector search, see Semantic vector search in AI Search.
- Embedding models
-
A list of one or more embedding models for the system to use when indexing content from the indexed source for semantic vector search.
An embedding model specifies how information found in your indexed content is encoded in a vector map. Semantic vector search uses the encoded information from the vector map to find search results that have meanings similar to those of your search terms.
- Chunking strategy and related parameters
-
A chunking strategy and related parameter values that together determine how content from the indexed source's selected fields and attachments is handled during indexing for semantic vector search.
Chunking is the process of breaking text down into smaller portions (called chunks) during indexing. By chunking your content, AI Search reduces search load and improves context and relevancy for semantic vector matches.
The following image shows how a two-paragraph block of sample field value text might be broken into chunks for semantic indexing. As shown, chunks can contain multiple sentences and may span paragraph breaks found in the original text.
- Semantic index fields
-
References to one or more semantic index fields that provide semantic indexing settings for content from the indexed source.
Each semantic index field defines semantic indexing settings for a single field from the indexed source table, or for attachments from that table. You can specify the order in which semantic index fields are evaluated when indexing content from the indexed source for semantic vector search.
Activating semantic index configuration
Semantic index configuration functionality is provided by the AI Search Semantic Controller plugin (com.glide.ais.semantic_search). This plugin is automatically activated for your instance when you install any Now Assist application.
You can verify whether semantic index configuration is activated by navigating to and selecting an indexed source record. If you see the Semantic Index Configuration related list on the Indexed Source form, the plugin is activated.