Predictive Intelligence - Using the Cluster Insight Table to improve your analysis

[Lener Pacania1]

Predictive Intelligence can help you find patterns in text data using clustering.  To leverage the full power of clustering you need to do the following: (1) pick the right clustering algorithm & word corpus (2) select purity fields (3) enable the cluster insight table.

I've provided links to items (1) and (2), we'll focus on (3) and enable the cluster insight table in this article.

1.  Go to All > Predictive Intelligence > Clustering > Solution Definitions > open an existing cluster solution and enable the "Create Cluster Insight table".  Update Frequency will specify how often new records are added to existing clustering.  The benefit here is you don't need to re-train the entire cluster.   Training frequency will re-train the entire cluster, and move the positions of the clusters (aka redefine the centroids).  You only want to re-train your clusters if you have a lot of new data that you want to mine.

2. Because we selected the "Create ClusterInsight table" we need to Update & Re-train the model.

3. Once the cluster model has been re-trained go to the Cluster Visualization.  In this case I am interested in the cluster with Cluster Concept = password reset need new help request voicemail please urgent voice.  The  cluster concept contains the top ten words that are most frequently found in all the records of the cluster.  In my example below the Size : 1105 means that there are 1105 records in this cluster that have a combination of the words in the cluster concept.  The Purity tells me that 85.34% of the records below to the ServiceDesk, 85.16% have category = request and 51.67% have priority of 2.  The purity fields are helpful to make sure that these records are being categorized correctly.

4. Click into the cluster with Cluster Concept = password reset need new help request voicemail please urgent voice.  This will open the cluster insight table.  This shows us a LIST view of all 1,105 records in the cluster. 

5. Now I can do some fine tuning and modify the filter to search for how many of these records have "voicemail" & "urgent"  in the short description.  You can do everything that you normally do in a list view, such as sort, create filters, and create graphs.   This narrows down the records to 398.

6. So now I wonder, how many "voicemail" & "urgent" issues exist within all the clusters.  Maybe we have the same issues handled by different groups.  Let's modify our filter and remove the "cluster is 16" and hit run.  Wow, so this looks like an issues spread across multiple assignment groups.  We have 10k+ records with this type of issue.  

In summary - The Cluster Insight table allowed us to identify a pattern in our incidents/cases that we might be able to automate with ServiceNow's Virtual Agent, Flow Designer workflow, or RPA.

[Vinay52]

Hello Lener,

Does Cluster Insight Table List view only shows Short description and description fields? If i want to see the Close Notes as well along with these 2 fields what needs to be done? I dont see adding any other fields apart from Cluster Field1, 2,3 etc. and what are these Cluster Field1, Cluster Field 2.. etc? They are showing empty values. thanks 

[Micael]

Thank you Lener, great article, however I have the same issue as Vinay52. How can I see in the list view the description and resolution notes? The short description is the only field from the incident that’s being displayed in the list view. I added the description and resolution notes to Fields but, I don’t have an option to add them to the list view.

[Lener Pacania1]

Hi Micael, Vinay52

Did you try adding the resolution notes & description as you input fields to the cluster solution definition?

I'm able to see resolution notes and description in my cluster insight list view.  I just created the below solution definition in my Tokyo instance.  -Lener

Here is my solution definition

HTH -Lener

[Micael]

Lener, yes, it worked. Once we deployed the model to production, the input fields in the cluster solution definition came in. Not sure why in our test environment it was not consistent. I also learned how to run reports from the model. Through the report tool, I was able to bring additional fields not included in the input field. Thank you very much for the article.