My experience with Virtual Agent

[Aditya Raute]

I'm writing this blog after a fairly successful implementation of Virtual Agent in my workplace. I'll highlight some key areas where we were falling short and the steps we took to improve in those areas. If you are struggling with the same things with Virtual Agent, hopefully, this blog serves as an ideation instigator.

I'll briefly walk you through the shortcomings we faced previously.

Improper Intent Identification

The two major causes for this was that the NLU capabilities were under-utilised — Model was unable to figure out the meanings of company-specific jargons (duh!) — and that the intents themselves were too few as the scope of the implementation was too narrow. We did not communicate this narrow scope to the users, which meant a bad User Experience.

Unreliable AI Search implementation

AIS is an important weapon in the arsenal of Virtual Agent. It is responsible for answering un-configured queries. However, in our case, we did not effectively use Genius Results and other search capabilities to our advantage, which hampered the UX overall. Since the scope of our topics was anyways narrow, this proved a major detractor.

---

With these challenges to combat, it was necessary to devise a detailed plan for next release stage and fix the issues faced. ServiceNow's Impact Accelerator team was a great help throughout the entire process. At the end, we had a decent product with reasonable adoption. Highlighting some steps we undertook.

Scope - narrow and deep over wide and shallow

For the ServiceNow Virtual Agent, it is important to understand that scope is managed by two avenues — one by defining topic flows, second by implementing AI Search. We made it a point to widen the topic flow scope only slightly and providing an in-depth conversational experience for all topics in scope. This ensured that whenever users conversed with the VA for those topics, they had a complete experience with resolution and steps to undertake further. Remember, having a wide range of topics with not much value in the conversation will always irritate the user.

Let AIS be the hero

Having AI Search cover a wide range of knowledge articles and catalog items - we succeeded in maintaining our VA as the one-stop solution for majority of user queries. AIS is a lucrative fallback option that will make you categorically move away from traditional topic flows. AIS, with its genius results, and the ability to boost and promote certain results serves as a reliable safety net when NLU does not pick up the utterance.

NLU best practices

Often NLU models are shoddily defined with low-context training utterances. It is important to define NLU in detail keeping up with the latest best practices. Use entities and vocabulary source abundantly. They not only help in better topic identification but are able to spot crucial information in the user's original utterance.

One suggestion to ServiceNow, it is often required that entity values be used in scripts. They should make entities more easily accessible in scripts. The current way is too bulky and prone to errors.

Befriend Dispatcher Topics

Dispatcher topics make topic creation and maintenance exponentially easier, while providing a great way of interfacing with the topics to the user irrespective of the quality of context in the utterance. Here is a fantastic implementation of Dispatcher topics. We took this one notch higher and opted for multi-level dispatcher topics for one of the deeper topics. This narrow focus enabled us to cover the topic in depth while elevating user experience.

---

To summarise, while there is a certain scope of improvement for Virtual Agent as a product, it is definitely possible to create a user-friendly experience and use Virtual Agent to your advantage. Focus on automating conversations and you'll start seeing the value that VA brings to the table.