Virtual Agent topic trigger script?

[Michael Patten]

I am looking for a way to track successful topics in Virtual Agent / MS Teams. 

Currently, the 'Anything else' topic asks if the Virtual Agent provided the correct information the user was looking for. A script in the 'Anything else' topic adds a plus 1 to the topic tracker. If the user says 'No', it will subtract 1.

However, if a topic requires user input and the user fails to continue the conversation, the 'Anything else' topic is not presented, and the topic tracker is not updated. 

I am looking to add a few lines of code to update the topic tracker when the unknown script calls the topic based on an NLU match. Would anyone happen to know what the script is called or where to find it? 

[Victor Chen]

The method that calls the topic based on NLU is vaSystem.topicDiscovery(). However, the Conversational Analytics Dashboard can track topics usage and completion for you. 

[Michael Patten]

Is the vaSystem.topicDiscovery editable?

[Chris D]

I don't have an answer for you, sorry, but I do have some things to add to the discussion...

Question: What's the "topic tracker" you're referring to? In Quebec, I actually implemented a custom version of "Deflection Tracking" - basically a "Was this Helpful? Y/N" field on the Conversation Task table and a topic block to set it - before ServiceNow implemented their own ootb version in Rome. Seems like this might be part of the ootb implementation of it...?

Food for thought: Coming from my experience with our custom deflection tracking which has the same problem you're describing (as does the ootb tracking) - what if the user doesn't answer "Was this helpful?" - you can assume whatever you want with no response but it sadly isn't going to make good data because it's not data - it's an assumption.

There at least a few scenarios at play where the user doesn't respond and they mean different things:

• The user DID get the help they needed, and as such, they closed/ignored VA as soon as they got their answer/help. This would be a Deflection, or in my case, Was this Helpful = Yes
• The user DIDN'T get the help they needed, but they were frustrated with the VA and gave up by closing/ignoring it. This one is NOT a Deflection - Was this Helpful = No.
• And then you have the possibility that the users unintentionally closed/ignored the VA because they forgot about it and/or got distracted. We have no idea if this is a Deflection or not.

So what I'm saying is just that you want to be cautious with the data you are considering presenting because, as uncomfortable it seems to have empty data, it actually does make sense in this case to be empty/undefined because we can't tell what the data should actually be.

[Michael Patten]

The topic tracker is a field I added to the interaction table to track topics provided.

The ootb tracking as well as my custom 'Anything else' topic all have the same problem with abandoned conversations.

'Was this Helpful' has been a back and forth and is a bit subjective. There is a weekly true-up of missed utterances based on the 'open_nlu_predict_log'. The utterances are reviewed, and topics are created and/or intents are updated to ensure accuracy on the VA. Therefore, if a user enters an utterance and VA returned a topic, that's a +1, unless otherwise stated back completing the conversation.

Based on the scenarios:

• The user DID get the help they needed, and as such, they closed/ignored VA as soon as they got their answer/help. +1 (Helpful = Yes)
• The user DIDN'T get the help they needed, but they were frustrated with the VA and gave up by closing/ignoring it. +1 (Helpful = Yes)
• And then you have the possibility that the users unintentionally closed/ignored the VA because they forgot about it and/or got distracted. +1 (Helpful = Yes)

I have found a work around: I created a topic block that adds the +1 and added to the beginning of each topic. If the topics were not helpful and/or not the indented topic, the user will need to answer the 'Was this Helpful?' with 'No'. No, will subtract 1 from the count and provide a feedback survey.