Zing Text Search is a powerful tool within the ServiceNow Platform that searches multiple tables within ServiceNow via a single transaction. Text Search enables us to find things easily without knowing which table or column the data is stored in. Search brings a logical collection of results grouped by table. I am going to show you how to configure the Auto Stop Words and rebuild text indexes for a table.
You will find all the tables that are indexed under System definition > Text indexes.
Stop words are common words that won't appear in search results or be queried for search. Stop words are of 2 types, global stop words (implemented out of box) and automatic table specific stop words. Once the frequency of any word on a table reach a particular threshold, the system automatically designates it as a stop word. This auto stop threshold can be enabled by clicking the table names under System definition > Text indexes. Enabling Stop Words on a table helps to reduce the size of the ts_ tables which will help to speed up the search process as well. Properly tuning Stop Words will improve performance of your text search operations dramatically. I've seen reductions in search times of 40x in some cases!! The ServiceNow product documentation has additional information on Zing text indexing and search engine .
Note that ServiceNow Zing Text Search can be replaced with AI Search in almost all areas of the ServiceNow platform. AI Search is the recommended search engine as it is faster and more accurate than Zing Text Search. New development on Zing Text Search stopped in the Rome release of ServiceNow and the product is in maintenance only mode. See Migrate Zing search application configurations to AI Search for more information.
How to configure auto stop words followed by regeneration of text indexes:
I am using this Task table as an example here
1. Set threshold for auto stop words:
a. Navigate to System Definition > Text Indexes > For eg: Open Task table record
b. Check 'Auto stop' checkbox and then set "Auto threshold" to the desired size (see notes below about how to estimate this value).
c. Save or Update.
2. Reset Text Search Caches
Navigate to System Definition > Text Indexes
In Related Links at the bottom select "Reset Text Search Caches"
3. Run "TS Index Stats" job
a. Navigate to System Scheduler > Scheduled Jobs > Open the "TS Index Stats" record
b. Select "Execute Now"
c. Monitor job until it completes (either by watching the worker on the node that picks it up or checking the "state" from sys_trigger_list)
4. Set Stop Mode if stop words are generated. If the stop words are not generated, lower the "auto threshold" value and repeat the above 3 steps.
a. Once "TS Index Stats" finishes, go back to the ts_index_name table
b. Select the task record
c. Scroll down to the stop words that were generated and go to the "Stop Mode" column, do a multi-select and change all the records to:"Neither Index Nor Query". This will stop the system from wasting processing overhead and disk space by indexing words that you will not be searching on. Also, if there are any words that are frequent enough to be stop words but, for whatever reason, you still do not want them to be stop words, you should set the Stop Mode to "Not a Stop Word". This can happen for certain company names or product names that occur frequently but will still need to be included to produce meaningful search results. Be warned that if a stop word is set to "Neither Index Nor Query" and you decide to change it to a different Stop Mode later you should also regenerating the index for the entire table or the results will be incomplete. If your goals include reducing writes to the database, to avoid replication lag for example, then using "Neither Index Nor Query" will be important to avoid database write activity for changes related to Stop Words.
See detailed instructions below:
4.c.i. Copy URL only for the stop words of a table and open in another window .
4.c.ii. Right click on column header "Stop Mode"
4.c.iii. Select "Update All" ( This appears only if you open the stop words in new window )
4.c.iv. A new form opens where select "Neither Index Nor Query" from drop down and click on "Update" ) (Neither index nor query tells the system not to index the word or be queried.)
5. Regenerate Indexes
a. Go back to the ts_index_name > task record. Select the related link 'Regenerate Text Index'
b. Verify that the "text index event process" is processing and finishes. Monitor the job until it completes.
c. An event named text_index.all will be created.
Repeat above steps for all tables you are planning to configure auto stop words and regenerate text indexes.
Regeneration of indexes may take a few hours (or even days!) depending on the size of tables and other factors. If your table is over 10 million records, you should be very cautious. Make sure you test on a sub-production first for estimated timings. It will almost always take LONGER on your production instance because production has more work going on and this contention usually causes processes to run more slowly. Text Index regeneration can cause performance degradation during the process and should be planned after business hours. Also, during text index regeneration, the text index tables will be incomplete. This means that search results on the table whose text index is being regenerated will not be accurate until regeneration completes.
How to Pick a Stop Words Threshold
The usual recommendation for large tables (over 1 million records) is to implement stop words with a threshold of 50,000. This will take any word that occurs 50,000 times or more and place it into your auto stop word list.
But setting stop words to 50,000 is not a one-size-fits-all solution. Some customers may find that 50,000 words is way too small. Others may find it way too big. As you might imagine, it all depends on how many words are out there. A customer with 1,000,000,000 task records will probably want a higher threshold than a customer with only 100,000 tasks. It is going to be different for each customer and for each table.
That being said, there are a few things that can help you make a decision when adjusting your stop word thresholds:
1. Consider if you really need to raise the threshold or can you just make exceptions for certain words? Suppose you set your stop words at 50,000 and then your users start complaining that certain words aren't brining back any of the results they expect. This can happen when a word they want to use is now a stop word. If there are certain words that are very common but you still want them to be indexed anyway you can make an exception for those words. You can do this through the Text Indexes module by opening the record for the table you want to fix, finding the word in question in the Ts index stops related list, and changing the Stop Mode of the record to "Not a Stop Word". If the Stop Mode was "Index but do not Query" then there is nothing more to do. That word will now show up in search results for that table. However, if the Stop Mode was set to "Neither Index nor Query" then you will need to click the "Regenerate Text Index" button on the same form after you have set the word to "Not a Stop Word". Caution: This will re-index the text search database for the whole table - a process that is resource intensive and may take hours or even days to complete for large tables.
2. Before you decide on a Stop Words threshold you should probably have a look at the frequency of the words that are currently being stopped and confirm that your adjustment will be a higher number than the frequency of words that you do not want to be stopped. You can see the frequency of each word in the Comment field of each record in the Ts index stops related list.
3. You may need to adjust the stop words thresholds of your large tables as your tables grow. If you are a new customer your task table might double in size every couple months. The stop words threshold that worked fine when you first implemented is now way too tight.
In the end, it all comes down to this: the higher your stop words threshold the slower text search will operate. I've seen customers who have gone as high 250,000 words for their threshold on some tables with no issues. Hope this helps!
Quick note on Zing versions
There are 4 versions of Zing Text Search Index. As of 2025, Version 3 is the most appropriate version for almost all tables. Version 4 might be helpful for certain text heavy tables like the Knowledge Article table (kb_knowledge). However, it is recommended NOT to upgrade any of your tables to Version 4 unless you have a specific business requirement to do so. There is no simple downgrade path once you upgrade to Version 4. See Configure a table for indexing and searching and Index Multiple Tables for product documentation regarding Zing V4.
14 Comments
