Do you have a Natural Language Understanding (NLU) powered Virtual Agent (VA) and a desire or means to enter the Now Assist universe? As of Washington Patch 3, you can migrate your NLU- or keyword-based topics to a Large Language Model (LLM) assistant in a low-touch manner.
Why migrate from NLU to LLM?
- No more NLU overhead – First, migrating to LLM will render NLU model management obsolete. LLM assistants do not rely on Natural Language Processing (NLP) technologies such as intent detection and entity recognition to process and collect user inputs. Rather, they rely on topic descriptions which are used for discoverability. This means you will no longer have to manage models and their contents in NLU Workbench and can simply write descriptions and prompt instructions in the same language that end users will use to engage with the LLM assistant. Gone are the days of manually creating utterances, intents, and entities, and labelling large quantities of data to tune your domain-specific models.
- New concept of Assistant & Seamless Setup – With our guided setup experience, you can create an LLM-powered assistant within minutes, after which you can port over your NLU topics using the migration tool. LLM topics can be associated with an “assistant” which constitutes a grouping of LLM topics and configurations, from information sources and branding to chat experience and display location. By migrating your NLU topics to your new LLM assistant, your topic descriptions will be automatically written based on any existing utterances associated with the NLU topics of interest. Note: Keyword topics are not supported for migration.
- LLM Authoring in Human Language – You will be able to leverage LLM nodes such as the Input Collector to better facilitate data collection, tone setting, and conversational fluidity. This means you can express, in human language, what types of data to collect from end users (formerly entities) and what kind of tone to use with them. End users can reply in whatever order makes sense for them irrespective of the order in which inputs were configured in the LLM topic, so the conversation flows less like a fixed decision tree and more like a real-world dialogue.
- Let the LLM do the Talking – Migrating to LLM also means you don’t need to spend time writing static messages and can fully lean on the LLM which will generate dynamic messages to end users in the tone and manner that you specify in the written prompt instructions (e.g., “fulfill laptop orders like a pirate”).
Overall, LLM topic authoring allows for more flexibility, adding a human touch while keeping the focus on writing quality prompt instructions.
|
NLU topics |
LLM topics |
|
NLU topic authoring requires multiple steps in multiple locations and ongoing management of the ML model in NLU Workbench.
• Create 10-15 utterances. • Create intents and topic mappings. • Create entities and vocabulary sources. • Label data and tune model. • Manually write static messages. |
LLM topic authoring empowers you to express what you want the assistant to do in natural language within VA Designer.
• Write a description that describes what the topic does that will be used for discovery. • Write prompt instructions using everyday language for the LLM to dynamically generate messages. |
Limitations
The LLM migration feature is actively being enhanced. As of the Washington DC release, you may still observe the following limitations:
- Zero-Touch or Bust – We are actively working towards zero-touch migration. Soon, you should be able to migrate your NLU topics automatically to LLM without needing to resolve any validation issues in VA Designer. In the meantime, you may encounter issues noted in a migration log for you to resolve in VA Designer before publishing.
- Role Configuration – In order to access the tool, you must have the right role assignment. Once you are assigned the correct role (virtual_agent_admin or sn_vad_genai.topic_migration_admin), there may be a slight delay in accessing the tool.
- Topic Description review – You can review the topic descriptions that are automatically generated for you if there are utterances associated with the intent mapped to NLU topics of interest. In the future, you will be able to review Topic Descriptions as its own step in the guided Topic Migration experience. This step will allow you to gauge how likely a topic description is to be discovered and to iteratively improve the description based on a match.
- Read-Only Topics – You are not allowed to delete read-only topics, so it is recommended to first duplicate any read-only topics before migrating them. If you migrate a read-only topic, you can’t delete it!
- Setup Topics – Setup topics, which are also read-only, are not included in the migration list. Duplicate them and change their topic type if you wish to migrate them.
- Entity Support – Entity vocabulary sources tied to an NLU-based input are not currently supported in the migration process, but a workaround is being investigated for the Xanadu release. More information to come soon.
Roadmap
Safe harbor: In the Xanadu+ release, there will be support for the automatic migration of notifications and proactive triggers that are connected to migrated topics to continue bridging the gap between the low-touch process and a true zero-touch migration experience.
To facilitate issue resolution, we have defined error types that may surface in your migration log if any issues arise during the migration. You can download the migration log as a csv file and review it as you get your LLM topics ready to be published.
In Xanadu / November, we are adding a new field for topics that allows for a separate internal name and display name. The display name appears in the "Show me everything" list or when the topic is promoted. When migrating to LLM, we'll append a prefix or suffix only to the internal name, so the display name will remain the same as the NLU topic. This will allow you to control the button labels that appear in Show Me Everything and Promoted Topics.
Further enhancements on the horizon include support for entity vocabulary sources.
We are excited to continue bridging the gap—stay tuned!
