Indexed source retention policies and filter conditions

  • Release version: Australia
  • Updated July 2, 2026
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Indexed source retention policies and filter conditions

    This guide explains how to manage the records indexed by AI Search in ServiceNow by configuring retention policies and filter conditions on source tables. These settings help optimize search performance by limiting the data indexed and automatically purging stale records.

    Show full answer Show less

    Indexed source retention policies

    Retention policies control which records from large source tables (like the Task [task] table and its extensions) are indexed based on the time since their last update. AI Search only indexes records updated within the specified retention period. For example, a two-year retention policy excludes records not updated in over two years.

    When a record exceeds the retention period, AI Search marks its indexed counterpart as stale. Retention policies are mandatory for sources indexing from the Task [task] table or its extensions, while optional for other sources.

    Indexed source filter conditions

    Filter conditions further refine the subset of records indexed by AI Search by including only those that meet specified criteria. This reduces index size and indexing frequency by excluding irrelevant or frequently changing records, such as open records filtered by status.

    When a record no longer meets the filter criteria, its indexed record is marked stale. Note that filter conditions do not decrease the number of indexing events triggered by table changes, but they do reduce the number of records processed per event. Changes to columns marked with notextindex do not trigger indexing events.

    Purging stale records

    AI Search automatically purges stale records from the index daily to maintain efficiency. Administrators can also manually execute a scheduled job to purge stale records as needed, ensuring the index remains optimized and up to date.

    To limit the set of records indexed from source tables, you can configure retention policies and filter conditions for your indexed sources. AI Search also uses these settings to automatically purge stale records from the index, optimizing search performance.

    Indexed source retention policies

    Indexing large source tables, such as the Task [task] table and tables that extend it, can add significant numbers of records to the AI Search index. To limit the set of source table records indexed based on the time since they were last updated, select a retention policy for your indexed source.

    AI Search only indexes source records updated within the time period defined for the retention policy. For example, if you select a two-year retention policy, AI Search excludes source records that were last updated more than two years ago.

    When the time since a source record's last update exceeds the limit from the indexed source's retention policy, AI Search marks the corresponding indexed record as stale.
    Note:
    Retention policies are required for indexed sources that index records from the Task [task] table or tables that extend it. They are optional for other indexed sources.

    Indexed source filter conditions

    To limit the set of records indexed from a source table, define filter conditions for your indexed source. AI Search only indexes records that match all defined filter conditions.

    Adding filter conditions can reduce the number of records indexed from a source table and reduce indexing frequency. As an example, if you exclude frequently updated open records based on their status, AI Search doesn't index data from those records, reducing your index size and the compute resources needed to index changes from the source table.

    When a source record no longer satisfies the indexed source's filter conditions, AI Search marks the corresponding indexed record as stale.
    Note:
    . Adding filter conditions doesn't reduce the number of AI Search indexing events generated for the indexed source. The system generates indexing events for every change to an indexed source table, unless the change is to a column that has a no_text_index field setting defined in the indexed source. The filter condition can reduce the number of records indexed for each event, however.

    Purging stale records

    AI Search automatically purges stale records from the index daily. Users with the admin role can manually purge stale records. For details on this procedure, see Purge stale records from the AI Search index.