Indexed sources in AI Search
Summarize
Summary of Indexed sources in AI Search
Indexed sources in AI Search allow ServiceNow customers to designate specific ServiceNow AI Platform® tables or external document sets containing alphanumeric text and string fields that should be searchable. AI Search ingests and indexes this content to enable efficient text-based search across selected records or documents.
Show less
Indexed source types
- Internal indexed source: Retrieves and indexes text and string fields from records in a specified ServiceNow AI Platform table and its child tables, excluding certain system tables. Remote tables cannot be indexed internally.
- External indexed source: Retrieves and indexes content from external repositories or remote tables referenced via external content schema tables, enabling searches beyond ServiceNow-native data.
Search performance considerations
Several factors affect search speed and indexing efficiency:
- Index size: Larger indexes take longer to search; avoid indexing unnecessary content.
- Number of indexed sources: More sources increase search time even if the index size is constant.
- Number of indexed fields: More fields indexed can slow search results retrieval.
- Indexing frequency: Frequent updates to indexed content compete with search operations for resources, impacting search response times.
Retention policies and filter conditions
To optimize index size and update frequency, customers can configure retention policies (e.g., exclude records older than two years) and filter conditions (e.g., exclude records by status). These settings help keep search results current, reduce index size, and limit compute resource consumption by avoiding indexing frequently changing or obsolete records. Note that retention policies are mandatory for sources indexing Task [task] tables or their extensions.
Indexed source configuration
- Attributes: Control indexing behavior at the record level.
- Field settings: Control indexing behavior for individual fields within records.
These configurations help tailor what content is indexed and how it is processed for search.
Indexing behavior and specifics
- AI Search automatically indexes changes (create, update, delete) to records in configured source tables and child tables.
- Unmodified records are indexed only after a full table index operation.
- Numeric fields are not directly searchable but can be used for faceting and filtering; to search numeric values, copy them into text or string fields.
- Knowledge article indexing defaults to including knowledge block content, but administrators can exclude this if desired.
- Only one indexed source can be active per ServiceNow AI Platform table. Duplicate indexed sources created by plugins or apps are inactive by default and must be managed manually.
Additional capabilities
- Indexed source guardrails: Limit the number of task and alert records indexed to optimize index size and performance.
- Semantic index configuration: AI Search supports a generalized RAG (Retrieval-Augmented Generation) framework to configure semantic indexing for platform table records.
- Catalog item variable indexing: Supports activation and configuration of indexing searchable content from variables on Catalog Item records.
Practical benefits for ServiceNow customers
By carefully defining indexed sources, retention policies, and field settings, customers can efficiently make relevant ServiceNow and external content searchable with AI Search. This enhances knowledge discovery, improves search relevance, and optimizes system performance by controlling index size and update frequency.
Indexed sources designate ServiceNow AI Platform® tables and external document sets with alphanumeric text and string field content that you want to make searchable. AI Search ingests text and string fields from table records or external documents and stores their searchable alphanumeric content in its search index.
For instructions on creating an indexed source, see Create an indexed source.
Indexed source types
- Internal indexed source
- An internal indexed source retrieves alphanumeric content and metadata from text and string fields on ServiceNow AI Platform records. It includes a unique name and a reference to a ServiceNow AI Platform table with records that you want to make searchable. AI Search extracts and indexes searchable alphanumeric content and metadata from text and string fields on records in this table and in any of its child tables that you configure for indexing.
- External indexed source
- An external indexed source retrieves alphanumeric content and metadata from text and string fields of documents in an external repository or a remote table. It includes a unique name and a reference to an external content schema table instead of a ServiceNow AI Platform table. For more details on configuring indexed sources for external content, see Indexing and searching external content in AI Search.
Search performance considerations for indexing
- Index size
- Indexing more content produces a larger index, which takes more time to search. Avoid indexing content that isn't needed for search.
- Number of indexed sources
- An index with more indexed sources takes longer to search than one with fewer indexed sources. This is true even if the two indexes are the same size.
- Number of indexed fields
- Increasing the number of fields you index across your indexed sources makes the system take longer to find search results. This effect is independent of index size and number of indexed sources.
- Indexing frequency
- The more often your indexed content is synchronized and updated, the more often search will compete with indexing for compute resources, increasing search response time. This is especially pertinent for indexed sources with frequently modified fields.
Indexed source retention policies and filter conditions
To limit the size of your index and the frequency of index updates, you can define retention policies and filter conditions for your indexed sources.
As an example, you can define a retention policy for an indexed source to exclude records that are more than two years old. This policy keeps your search results more current and reduces the size of your index. Changes made to the excluded records don't trigger index updates, so this policy also reduces indexing frequency.
Similarly, you can define a filter condition for an indexed source that excludes source table records with a specific status, such as Open. This filter condition reduces the number of records indexed from the source table, which in turn reduces the total amount of data you index. Excluding open records that have frequent updates also reduces indexing frequency.
AI Search also uses your retention policy and filter condition settings to automatically purge stale records from the index, reducing its size.
To learn more about creating retention policies and filter conditions for your indexed sources, see Indexed source retention policies and filter conditions.
Indexed source attributes and field settings
You can configure attributes and field settings for an indexed source to control indexing behavior for source records. Attributes control the indexed source's behavior at the record level, while field settings define its behavior for individual fields on indexed records. For more information, including lists of available attributes and field settings, see Indexed source attributes for AI Search and Field settings for AI Search.
Indexing content from an indexed source
Indexing content from knowledge articles
When indexing content from records in the Knowledge [kb_knowledge] table, AI Search defaults to including content defined in knowledge blocks. Administrators can override this default behavior and configure AI Search to exclude content from knowledge blocks when indexing knowledge articles. For details on making this change, see Exclude knowledge block content from the AI Search index.