If Conversational Analytics tables are blank, where is the data coming from?

[Michael Scepano]

Why is the Virtual Agent Conversational Analytics table blank? These tables are used by and feed into the Virtual Analytics dashboard, of which there is data that can be seen. However, if you try to view the table data in list view, there are no records.

Eg, on the Virtual Agent Dashboard (https://[YOUR_INSTANCE].service-now.com/now/conversation/analytics/home) you can click on your profile at the top right -> Edit page -> and navigate to the 'Conversations' page. You can see that the data source for the list is sn_ci_analytics_st_conversation . Note: there are a suite of tables, 27 specifically, that have the sn_ci_analytics prefix - these tables, and certainly the one mentioned above, is used only for Conversational Analytics.

This OOTB functionality works, as if you navigate to the conversations page on VA Analytics, you can see the data is populated. However, if you navigate to the data source table sn_ci_analytics_st_conversation and show the records in list view https://[YOUR_INSTANCE].service-now.com/nav_to.do?uri=%2Fsn_ci_analytics_st_conversation_list.do%3F , you see that there are no records.

One of the reasons for the existence of this table as opposed to using just the sys_cs_conversation table, is due to maintaining anonymity. One of the differences is that the conversation table shows the userID under 'Consumer', whereas the analytics table has a hashed user ID instead (encrypted for anonymity).

So, going back to my original question, my best theory for why the sn_ci_analytics_st_conversation table is blank is again due to anonymity, that maybe we are not supposed to be able to view these records in list view?

However, the weirdness gets further - under the columns (VA Analytics -> edit page -> Conversation), you can see: date,duration,user_index,channel,language,end_state,topics,favorite . These are the columns being selected to show on the dashboard, from the sn_ci_analytics_st_conversation table as that is the source. These match what is available in the database schema, but I also decided to add sys_id to the dashboard so I could see what they are, ultimately to help me track down where this data is. The dashboard successfully returns the sysID of those records within the dashboard, but when I then search for the sysID in SNOW, even using a the global sysID search in the SNUtils browser extension, no results are found.

If we delve deeper into the rabbit hole, we also notice that the OOTB functionality shows the chat transcript and topic timeline if you click on a sn_ci_analytics_st_conversation record within the VA dashboard. Where is this data coming from? According to the schema, there are no reference variables in that table - it must be linking to another table as the data is correct, but it's not showing us how and where this is occurring.

As a final note, as I wanted to extract the more detailed information that exists in the sys_cs_conversation table so I changed the dashboard source to that, which worked to pull the data, but then a 500 Internal Server Error continually reappears, I believe related to the transcript/topic timeline that now ceases to work. I can't see anywhere in the edit page settings where this is configured - it seems to only be done automatically with the sn_ci_analytics_st_conversation table but there must be more to this story.

So my question is, where is the data coming from? I would really like to know where the data is coming from and how this is working, so I can ultimately combine and match the data from the sys_cs_conversation table with the functionality of that transcript/topic timeline view.

Kind regards,

Michael.

[Amit Gujarathi]

Hi @Michael Scepano ,
I trust you are doing great.

To access the Conversations page on the Virtual Agent Dashboard, follow these steps:

1. Go to the Virtual Agent Dashboard at https://[YOUR_INSTANCE].service-now.com/now/conversation/analytics/home.
2. Click on your profile at the top right corner.
3. Select "Edit page" from the drop-down menu.
4. Navigate to the 'Conversations' page.

The data source for the list on the Conversations page is the "sn_ci_analytics_st_conversation" table. It is important to note that there are 27 tables with the "sn_ci_analytics" prefix, but the one mentioned above is specifically used for Conversational Analytics.

Although the Conversational Analytics feature functions correctly on the Virtual Agent Analytics dashboard, the table itself appears to have no records when viewed directly at https://[YOUR_INSTANCE].service-now.com/nav_to.do?uri=%2Fsn_ci_analytics_st_conversation_list.do%3F.

One reason for having a separate table like "sn_ci_analytics_st_conversation" instead of using the "sys_cs_conversation" table is to maintain anonymity. The conversation table displays the userID under 'Consumer,' whereas the analytics table uses a hashed user ID (encrypted for anonymity).

The best theory for why the "sn_ci_analytics_st_conversation" table is blank is that, due to anonymity concerns, we may not be permitted to view these records in list view.

However, there are additional peculiarities to consider. Under the "VA Analytics -> edit page -> Conversation" section, the following columns are displayed: date, duration, user_index, channel, language, end_state, topics, and favorite. These columns are selected to be shown on the dashboard, and they come from the "sn_ci_analytics_st_conversation" table as the data source. These columns align with the database schema, but I also added "sys_id" to the dashboard to track down the data. Interestingly, the dashboard successfully returns the sysIDs of those records. However, when searching for the sysID in SNOW, including using the global sysID search in the SNUtils browser extension, no results are found.

If we dig deeper, we notice that the OOTB functionality on the VA dashboard displays the chat transcript and topic timeline when clicking on a "sn_ci_analytics_st_conversation" record. The source of this data is not apparent in the schema, as there are no reference variables in that table. It suggests that the data is linked to another table, but it doesn't reveal how and where this linkage occurs.

[Michael Scepano]

Hi Amit,

Thanks for confirming the results I experienced, but in all honesty you've mostly just restated what I've already said - particularly the 4 step instructions on how to access Conversational Analytics.

I do appreciate your time though, and it's always great to have a second opinion to confirm. I will not however be marking this as the solution though - in this case, it's a bit audacious to ask.

Cheers,

Michael.

[Ramiro4]

Hi @Michael Scepano,

Did you have any update or explanation on this topic from servicenow? 

Thanks!

[Michael Scepano]

Hey Ramiro,

Sorry for the delayed response. So I spoke to SNOW and they essentially said that those tables are not supposed to be reported on - all of these tables with the sn_ci_analytics prefix are used by VA analytics for the VA Dashboard - there's a bit of smoke and mirrors behind it all, but essentially they exist for a reason, but that we're not supposed to be able to view them. The thing with ServiceNow fulfiller is that it's still the front-end...