Claire_Conant
ServiceNow Employee
Options
- Post History
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
4 hours ago
Your Virtual Agent is live, but the topics you built aren't resolving users' questions the way you expected. Maybe the NLU isn't matching intents correctly, or the conversation flow hits a dead end, or the Microsoft Teams integration isn't working. Getting Virtual Agent right after go-live almost always takes another pass. None of this means something's broken in the build. It usually means one specific layer between intent, topic, and channel needs tuning, and knowing which layer to check first is what turns this into a quick fix instead of a guessing game.
Start with what your users are reporting
Before diving into specific pitfalls, find out what your users are really experiencing. Checking Virtual Agent analytics can show you how often users are successful in getting the right answers (conversation completion rates) and where your users are giving up (abandonment points). Use the NLU Workbench testing page to see what the model is predicting in response to user queries. Then, use the following table to match what you're seeing to the right section.
|
What you’re seeing
|
Where to start
|
|
Bot matches the wrong topic to a user’s question
|
|
|
“I want to be sure I got this right” disambiguation prompt on clear queries
|
|
|
Conversation stalls or loops without resolution or handoff
|
Missing fallback paths in your topic flow (item 3)
|
|
“I’m having technical issues” error in Microsoft Teams
|
Public role on greeting and setup topics (item 4)
|
|
Topics work in Service Portal but not in Teams
|
Channel-specific configuration and role requirements (item 4)
|
|
Now Assist LLM topic is published but doesn’t trigger
|
Topic description and assistant assignment (item 5)
|
|
No data on which topics are failing
|
Virtual Agent analytics — identify abandonment points, then revisit the matching item above
|
If you’re still setting up Virtual Agent for the first time, start with the product documentation on planning and configuring Virtual Agent.
1. NLU intent overlap across models
What you may have missed: Testing with all models active at the same time.
You’ve created multiple NLU models with separate intents, each connected to a different topic. Training looks good in the NLU Workbench, and individual intents predict correctly when tested in isolation. But when multiple models are active in the same entry point, the NLU engine evaluates intents across all of them simultaneously. If two intents in different models use similar utterances, the engine returns competing predictions with close confidence scores. The result is either the wrong topic triggering, or the disambiguation prompt appearing for queries that should match cleanly.
What to try instead: Use a single NLU model per entry point whenever possible. Consolidating intents into one model lets you use the Cross-model Conflict Review tool in NLU Workbench to detect overlapping utterances before they cause problems in production. If you do need multiple models, test them together using the Virtual Agent Simulator in Virtual Agent Designer—not just the Workbench. Also check the CS Topic Language (sys_cs_topic_language) table to confirm there’s only one binding per topic per language, since duplicate bindings can cause unexpected prediction behavior.
2. Confidence threshold defaults that changed between releases
What you may have missed: Verifying threshold settings after a platform upgrade.
Starting with the Tokyo release, the com.glide.cs.intent_confidence_threshold system property was set to 0 by default for ServiceNow NLU. Virtual Agent no longer uses that global property to filter low-confidence predictions. Instead, each NLU model has its own confidence threshold. Go to All > NLU Workbench > Models, open your model, then go to Manage your model content > Settings. If you upgraded from an earlier release and assumed the system property was still enforcing a floor (the previous default was 0.6), low-confidence predictions may be slipping through unfiltered.
A related setting, com.glide.cs.intent_confidence_delta, controls how close two competing predictions need to be before the bot prompts the user to choose. If this value is too low, the bot auto-selects between closely scored intents without confirming.
What to try instead: Set the confidence threshold on each NLU model individually rather than relying on the system property. If a specific intent is consistently triggering at low confidence, apply a per-intent override using the glide.mlpredictor.option.nlu.confidenceThresholdOverride property—this lets you set a minimum confidence score for individual intents without affecting the entire model. After any platform upgrade, verify these values haven’t reverted to defaults.
3. Conversation flows that leave users stranded
What you may have missed: Testing with unexpected inputs.
Your topic flow handles the expected path: the user asks a question and then the flow walks them through resolution. When users respond in unexpected ways, provide input the flow doesn’t handle, or reach a branch condition that evaluates to false, they hit a dead end: no resolution, no escalation, no next step. This is especially common in flows that rely on conditional branching without a default “else” path, or in flows that don’t include a live agent handoff when self-service fails.
Users experiencing this don’t always report it clearly. They just stop using Virtual Agent.
What to try instead: Add a fallback path to every conditional branch in your topic flow. Include a live agent handoff option at any point where the self-service path might not resolve the issue. Test flows by deliberately entering unexpected inputs, not just the inputs that match your design. Check Virtual Agent analytics for conversation abandonment points, which often indicate where users are hitting dead ends.
4. Topics that work in Service Portal but fail in Microsoft Teams
What you may have missed: Testing in your actual deployment channels.
Topics built and tested in the Virtual Agent Designer and Service Portal can fail in Teams with behavior differences that Service Portal doesn’t surface. The most common failure is the “I’m having technical issues and won’t be able to continue this conversation” error. This typically has one of three root causes:
- Greeting topic is missing the Public role. If you customized the greeting topic and didn’t assign it the Public role, Teams users—especially guest users—can’t access it. The fix: Open the greeting topic in Virtual Agent Designer, add the Public role to the Roles field, save, and republish.
- Custom setup topics that aren’t public. The same role issue applies to other setup topics: Anything Else, Closing Conversation, Error Handling, and Live Agent Support. If any custom versions of these topics lack the Public role, Teams users get the error.
- AI Search fallback topic enabled without AI Search active. If your setup topics include the AI Search fallback topic but AI Search isn’t activated on the instance, the topic fails silently in Teams.
Beyond the “technical issues” error, attachment handling can crash the Teams session, and topics that rely on UI elements not supported in the Teams messaging channel may behave unpredictably.
What to try instead: Test topics in Teams separately from Service Portal—don’t assume portal success means Teams success. If you see the “technical issues” error, check the greeting topic and all custom setup topics for the Public role first. Verify that any fallback topics reference active features on your instance.
Note: Teams requires a Teams Admin role on the Microsoft side and only supports users with Office 365 accounts—free Microsoft Teams accounts without an Office 365 subscription are not supported.
5. Now Assist LLM topics that are published but not discoverable
What you may have missed: Testing discovery in the actual Now Assist interface.
Your LLM-based topic works when tested directly in Virtual Agent Designer. But it doesn’t trigger when users type matching queries in the Now Assist interface. Now Assist in Virtual Agent uses a fundamentally different discovery mechanism than NLU-based Virtual Agent. NLU topics are discovered through utterance matching; LLM topics are discovered through AI Search and the topic’s natural-language description. If the topic description is too generic, or if it conflicts with content that AI Search surfaces from your knowledge base, the topic won’t be selected.
Additionally, NLU-based topics and LLM-based topics are not interchangeable. NLU topics can’t run in the Now Assist chat client, and LLM topics can’t run in the NLU-based Virtual Agent. If you migrated topics but didn’t verify which assistant they’re assigned to, they may be published to the wrong client entirely.
What to try instead: Write detailed, specific topic descriptions that use the natural language your users would actually type. Test discovery using the Now Assist interface on Service Portal, not just the Designer test console. Verify that your topic appears in the correct assistant’s active topic list (check Now Assist Panel > default topics). If you migrated NLU topics to LLM topics, confirm they’re active in the Now Assist assistant, not just in the legacy NLU Virtual Agent.
Where to go from here
After go-live, keep an eye on Virtual Agent analytics weekly for two signals: topics with low completion rates (which often point to conversation dead ends) and low-confidence intent matches (which indicate NLU tuning gaps). These two metrics tell you where to focus your next round of iteration.
For NLU-specific issues, the testing page in NLU Workbench is the fastest way to see what the model actually predicts for a given query, including competing intents and confidence scores.
Fixing Virtual Agent issues becomes much simpler when you can identify which layer is responsible. Start with the symptom, trace it to the right layer, and you'll usually find a clear path to resolution.
Note: Licensing, usage, and entitlements can vary by release, setup, and contract. Always confirm the specifics for your environment.
