Is there an easy way to identify if a topic was triggered from an utterance or from the show me everything menu?

[Ulrich1]

We would like to give our users some context when they triggered a topic through an utterance, e.g. to ensure they ended up in the correct topic and give them an option to quickly exit if it is the wrong one, however this information is not needed when they intentionally choose a topic from the show me everything. How can we achieve this?

Is there a way to even consider the NLU prediction confidence? e.g. show such a message only if the prediction confidence is below 80%?

We are aware about how to optimize the NLU to prevent such issues to some extend, but there are always some cases, where you end up in a wrong topic, which can be very frustrating for the customers if they don't immediately realize that they are wrong.

[Chris D]

Universally? No. (See paragraph 3 for an ootb option that may meet your needs very similarly though.)

You could no doubt add a custom script to the start of each and every topic to check if an NLU prediction was made and manually do your confirmation - I suppose the best thing way to do this would be a topic block that you could add to the start of all your topics, but that's still quite a bit of overhead, especially if you have a lot of topics already. And yes, you can use the prediction confidence in this script if you'd like. You can use vaVars._topic_current (details below) to get the prediction and utterance, though not the confidence.

vaVars._topic_current

Same as vaVars._topic_results. Provides details of the current Topic the conversation is in, in JSON format. Details like the Sys ID of the current Topic, Sys ID of the Conversation Task, name of the current Topic. Example output: "{"sys_id":"5e85248c2f50a050b0c2d5ea2799b6b4", "task":"f327e4802f90a050b0c2d5ea2799b63d", "entities":null, "modelId":null, "name":"Test Topic", "prediction":null, "title":"Test Topic", "type":"normal", "utterance":"test topic"}"

(From 50+ (Un)documented Virtual Agent variables (vaInputs, vaVars, vaContext, vaSystem) - Virtual Agent, ...)

Afaik, the confidence would need to be queried on the open_nlu_predict_intent_feedback table. If you're using that data, keep in mind a) you shouldn't need do a confirmation when confidence = 1(00%) and b) you shouldn't need to do a confirmation if "Shown = true (and Auto-selected = false*)" - what that means is that multiple intents matched within the confidence delta (default was 5%)* of each other, so the system already asked the user to confirm their topic - i.e. it did what you wanted it to do.

*If your concern is specifically that you have similar topics and users happen to be getting to the wrong one so you want them to use this ootb confirmation (more), you could play around with the com.glide.cs.intent_confidence_delta property, which is what was defaulted to 5% (0.5) before Quebec, though ServiceNow has it default to 0 which is their recommendation. But theoretically, you could make this delta super high, let's say 20%, which would mean that if you have an intent that matches 90% and another one that matches 70%, the system would display this confirmation message and make the user choose between the two (or none of the above).

I think the reason it's recommended to keep this at 0 is because it forces you to maintain a tip-top NLU model which is best for the user experience, but it sounds like may achieve a good chunk of what you want. See this for more details about how confidence delta works: Virtual Agent Topic Discovery - Virtual Agent, Natural Language Understanding (NLU) - Article - Serv...

Using ootb functionality to address this issue, there's a few things that come to my mind:

• Using the "Greeting response" for each topic
  • Unimportant for users manually starting topics, but such a simple but important underrated feature for setting the context for users who start topics via NLU (or topic switching). This indirectly acts as your confirmation for users to ensure they're in the right topic.
    
    • i.e. if they type in "my printer is broken" and the first message of the topic is "When did this issue occur?", the user has no idea if they started the printer topic or the cell phone topic. Throw in a simple greeting like "I see you have an issue with your printer... I can help report this issue to the hardware team after you answer a few questions."
• Educating users on how to end/restart a conversation
  • This way, if the greeting message doesn't meet the user's expectations, they can try a different option

[Chris D]

Wow, I wrote up that big explanation yesterday and after seeing the answer on the new, similar post here - How to check if a VA Topic was triggered from NLU or if a user selected it from topic picker? - Virt... - I feel silly because I suggested vaVars._topic_current which was so much less simple than vaSystem.getSearchText(). 

From that topic:

The conditions are:

NLU: return !gs.nil(vaSystem.getSearchText())

Manual selection: return gs.nil(vaSystem.getSearchText())

That still doesn't get you the confidence, so besides that one better method to use, my other points still stand 🙂