Semantic index configuration for indexed sources
Summarize
Summary of Semantic index configuration for indexed sources
The AI Search generalized RAG (Retrieval-Augmented Generation) framework in ServiceNow provides a streamlined approach to configure semantic indexing settings for records indexed from AI Platform® tables. This configuration enables AI Search to index content for semantic vector search, improving search relevance by understanding the meaning behind search terms rather than relying solely on keyword matching.
Show less
Key Features
- Semantic Index Configuration: A set of settings applied to a specific indexed source, defining how its content is processed for semantic vector search.
- Indexed Source: A reference to an existing indexed source containing field values or attachments to be semantically indexed.
- Embedding Models: One or more models that encode content into vector representations, enabling the semantic vector search to find results with similar meanings to user queries.
- Chunking Strategy: A method that breaks down text into smaller chunks during indexing to reduce load and enhance context and relevancy of search results. Chunks can span multiple sentences and paragraph breaks.
- Semantic Index Fields: Settings applied to individual fields or attachments in the indexed source, with the ability to specify the evaluation order during indexing.
- Multiple Configurations: Support for multiple semantic index configurations per indexed source, though additional configurations increase indexing performance costs.
- Activation: Functionality is enabled via the AI Search Semantic Controller plugin (com.glide.ais.semanticsearch), automatically activated when installing any Now Assist application.
Practical Application for ServiceNow Customers
AI Search administrators can configure semantic indexing settings by accessing indexed source records in ServiceNow. Verification of the semantic index configuration activation can be done by checking for the Semantic Index Configuration related list on the Indexed Source form. Proper configuration of embedding models, chunking strategies, and semantic index fields allows customers to tailor semantic vector search behavior, leading to more contextually relevant search results across indexed content.
This configuration is essential for customers aiming to leverage AI-enhanced search capabilities within their ServiceNow instance, improving user experience by delivering more meaningful and precise search outcomes.
The AI Search generalized RAG (Retrieval-Augmented Generation) framework offers a streamlined way to configure semantic indexing settings for records indexed from ServiceNow AI Platform® tables.
Semantic index configuration overview
AI Search admins can configure semantic indexing settings for an indexed source. These settings specify how AI Search indexes content from the indexed source for use with semantic vector search. The group of semantic indexing settings for a particular indexed source is called a semantic index configuration.
- Indexed source
-
A reference to an existing indexed source with field values or attachments that you want indexed for semantic vector search.
For more information on indexed sources, see Indexed sources in AI Search. To learn more about semantic vector search, see Semantic vector search in AI Search.
- Embedding models
-
A list of one or more embedding models for the system to use when indexing content from the indexed source for semantic vector search.
An embedding model specifies how information found in your indexed content is encoded in a vector map. Semantic vector search uses the encoded information from the vector map to find search results that have meanings similar to those of your search terms.
- Chunking strategy and related parameters
-
A chunking strategy and related parameter values that together determine how content from the indexed source's selected fields and attachments is handled during indexing for semantic vector search.
Chunking is the process of breaking text down into smaller portions (called chunks) during indexing. By chunking your content, AI Search reduces search load and improves context and relevancy for semantic vector matches.
The following image shows how a two-paragraph block of sample field value text might be broken into chunks for semantic indexing. As shown, chunks can contain multiple sentences and may span paragraph breaks found in the original text.
- Semantic index fields
-
References to one or more semantic index fields that provide semantic indexing settings for content from the indexed source.
Each semantic index field defines semantic indexing settings for a single field from the indexed source table, or for attachments from that table. You can specify the order in which semantic index fields are evaluated when indexing content from the indexed source for semantic vector search.
Activating semantic index configuration
Semantic index configuration functionality is provided by the AI Search Semantic Controller plugin (com.glide.ais.semantic_search). This plugin is automatically activated for your instance when you install any Now Assist application.
You can verify whether semantic index configuration is activated by navigating to and selecting an indexed source record. If you see the Semantic Index Configuration related list on the Indexed Source form, the plugin is activated.