--- sourceDocument: Australia ServiceNow AI Platform Administration sourceDocumentLink: https://www.servicenow.com/docs/r/australia/platform-administration Release : - australia ft:locale : - en-US ft:publication_title : - Australia ServiceNow AI Platform Administration ft:clusterId : - platadm bundleId : - platadm workflow : - Platform --- # Indexed source retention policies and filter conditions # Indexed source retention policies and filter conditions {#ariaid-title1} Release version: Australia Updated September 30, 2026 ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 2 minutes to read Summarize ![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete. ## Summary of Indexed source retention policies and filter conditions This content explains how ServiceNow customers can control the records indexed by AI Search from source tables by using retention policies and filter conditions. These settings help optimize search performance by limiting the volume of indexed data and enabling automatic purging of stale records. Show full answer Show less ## Indexed source retention policies Retention policies restrict indexing to source records updated within a specified time frame, reducing the size and scope of the AI Search index. For example, selecting a two-year retention policy excludes records last updated over two years ago. When a record exceeds this retention period, AI Search marks its indexed counterpart as stale and eligible for purging. Retention policies are mandatory for indexed sources based on the Task \[task\] table or its extensions, and optional for other sources. ## Indexed source filter conditions Filter conditions further refine indexed data by indexing only records that meet all specified criteria. This reduces index size and indexing frequency by excluding certain records, such as frequently updated open records filtered by status. When a record no longer meets filter conditions, its indexed record is marked as stale. Note that filter conditions do not reduce the number of indexing events triggered by changes in the source table, but they limit the number of records indexed per event. Filter conditions apply only to fields indexed by AI Search. Fields like Tags and fields on referenced records are not indexed by default and require explicit configuration to be used in filter conditions. ## Purging stale records AI Search automatically purges stale records daily to maintain index efficiency. Administrators can also manually trigger purge operations through scheduled jobs to remove stale indexed data immediately. To limit the set of records indexed from source tables, you can configure retention policies and filter conditions for your indexed sources. AI Search also uses these settings to automatically purge stale records from the index, optimizing search performance. ## Indexed source retention policies {#retention-policies-conditions-ais__section_fdt_qqk_2rb} Indexing large source tables, such as the Task \[task\] table and tables that extend it, can add significant numbers of records to the AI Search index. To limit the set of source table records indexed based on the time since they were last updated, select a retention policy for your indexed source. AI Search only indexes source records updated within the time period defined for the retention policy. For example, if you select a two-year retention policy, AI Search excludes source records that were last updated more than two years ago. When the time since a source record's last update exceeds the limit from the indexed source's retention policy, AI Search marks the corresponding indexed record as stale. Note: Retention policies are required for indexed sources that index records from the Task \[task\] table or tables that extend it. They are optional for other indexed sources. ## Indexed source filter conditions {#retention-policies-conditions-ais__section_zpd_sqk_2rb} To limit the set of records indexed from a source table, define filter conditions for your indexed source. AI Search only indexes records that match all defined filter conditions. Adding filter conditions can reduce the number of records indexed from a source table and reduce indexing frequency. As an example, if you exclude frequently updated open records based on their status, AI Search doesn't index data from those records. Excluding these records reduces your index size and the compute resources needed to index changes from the source table. When a source record no longer satisfies the indexed source's filter conditions, AI Search marks the corresponding indexed record as stale. Note: Adding filter conditions doesn't reduce the number of AI Search indexing events generated for the indexed source. The system generates indexing events for every change to an indexed source table, unless the change is to a column that has a `no_text_index` field setting defined in the indexed source. The filter condition can reduce the number of records indexed for each event, however. Indexed source filter conditions only operate on source record fields that are indexed. Fields in the following table aren't indexed by default. {#retention-policies-conditions-ais__table_hhm_5w2_tkc__entry__2} | Field | Description | |-|-| | Tags | The Tags field on source records isn't indexed by default. To use tags in an indexed source filter condition, you must first configure indexing for tags in the indexed source. For details on this procedure, see [Enable indexing of tags for an AI Search indexed source](https://www.servicenow.com/docs/w6aX5knliRVxq05LEL6weg "Configure an indexed source attribute to enable indexing of content from tags found on records from an indexed source."). | | Fields on referenced records | Fields on referenced records aren't indexed by default. To use a referenced record field in an indexed source filter condition, you must first configure indexing for the field in the indexed source. For details on this procedure, see [Enable indexing of referenced table fields for an AI Search indexed source](https://www.servicenow.com/docs/wP1WDY4IIkT5PRtYxPDE4w "Configure an indexed source to index field values from tables referenced by fields in the source table. You can index these field values for use in filters and EVAM search result configurations, index them for search, or both."). | [Table 1. Fields not indexed by default] {#retention-policies-conditions-ais__table_hhm_5w2_tkc} ## Purging stale records {#retention-policies-conditions-ais__section_yqg_yqk_2rb} AI Search automatically purges stale records from the index daily. Users with the admin role can manually purge stale records. For details on this procedure, see [Purge stale records from the AI Search index](https://www.servicenow.com/docs/_is7nyG_uCGs5sMmJkaprQ "Execute a scheduled job to delete stale records from the AI Search index."). * **[Purge stale records from the AI Search index](https://www.servicenow.com/docs/_is7nyG_uCGs5sMmJkaprQ)** Execute a scheduled job to delete stale records from the AI Search index.