How to Access Semantic Vectors from AI Search Index for Clustering in Predictive Intelligence?

Stefan3000
Tera Contributor

‎03-22-2025 03:54 AM

Hi everyone,

 

I’m working on clustering incidents in ServiceNow using the semantic vectors generated by the AI Search semantic indexing (e.g., based on the short_description or description fields of the incident table).

 

My goal is not to cluster based on the raw text but rather on the vector embeddings that are already generated and stored by the AI Search index.

 

Is there a way to access these semantic vectors from within the Predictive Intelligence module (or elsewhere) so that I can apply clustering algorithms like k-means and analyze similarity within clusters?

 

Any insights or best practices would be greatly appreciated!

 

Thanks in advance,

Stefan

0 Helpfuls
  • 1,317 Views
1 REPLY 1

Bernard Esterhu
Tera Expert

‎03-27-2025 11:39 PM

Hi @Stefan3000 

We have been investigating ServiceNow clustering for more than a year.

firstly, the AI Search tool does not create vector based indexes. The AI Search, in most cases, is text matching, not semantic search (i.e. vector based). See below for info on where Semantic Search is available:

 

BernardEsterhu_1-1743143627696.png

BernardEsterhu_2-1743143650058.png

I have also never been able to actually see the content index tables to investigate this deeper.

Your other option is to make use of the Predictive Intelligence clustering feature. This originally used word-2-vec or paragraph-2-vec text vectorisation, but has recently changed to GUSE (Google universal sentence encoder) for embedding generation. There are many limitations to the clustering feature, especially if you would like to use if for continuous clustering. The default clustering algorithm is k-means, but there is also an option to use DBSCAN and HDBSCAN. However those two did not look to be mature solutions when we tested them.