AI SEARCH - Is there a limit on how many words are indexed for searching

[maggieo]

Some of our customers have long documents (39 pages+) , and when searching for a word that is found on the last pages, AI search is not returning any results.

Is there a limit for the length or word count in AI Search? 

[rohansargar]

Hello,

Yes, ServiceNow AI Search does have limits on document length and indexing that could be causing this issue.

1. Tokenization & Indexing Limits
AI Search tokenizes and indexes text, but there are limits on how much of the document is processed.
If a document is too long, only a portion of the content might be indexed, truncating the later parts.
2. Maximum Document Size
ServiceNow AI Search has a limit on the maximum size of a document that can be indexed.
For long documents (39+ pages), only a portion (e.g., the first few thousand words) might be considered.
3. Search Query Length Limits
AI Search uses vector-based similarity and keyword-based indexing.
If the target word is located in a part of the document that wasn't indexed, it won't appear in search results.

Mark Helpful if you got solution for your issue.

Best Regards,

Rohan.

[maggieo]

Thank you @rohansargar , I really appreciate your response.

Is there an sn document where I can found what those limits are? I mean when you mention if the document is too long, how long is considered too long?

[rohansargar]

Thank you @maggieo for marking as helpful.

ServiceNow's official documentation doesn't specify explicit limits on document length or word count for AI Search indexing.

But you can refer : http://servicenow.com/docs/bundle/yokohama-platform-administration/page/administer/ai-search/concept...

Best Regards,

Rohan.

[Gerard Dwan]

For documents, use the attachment section of this docs page for info: https://www.servicenow.com/docs/bundle/yokohama-platform-administration/page/administer/ai-search/co...

By default, AI Search indexes searchable content and metadata (such as file size and date) for attachments found on source table records. Indexing supports the following attachment file formats:

• Active Server Page Extended (.aspx)
• Hypertext Markup Language (.html, .htm)
• Microsoft Excel (.xls, .xlsx)
• Microsoft PowerPoint (.pot, .potx, .ppt, .pptm, .pptx)
• Microsoft Word (.doc, .docx, .dot, .dotx)
• Plain Text (.txt)
• Portable Document Format (.pdf)

Note: AI Search ignores attachments larger than 25 MB in size. Standard indexing only indexes content from the first 1 MB of the attachment's data, while semantic vector indexing indexes the attachment's full content.

This is not configurable.