Model management
Summarize
Summary of Model management
The Model management feature in the NLU Workbench enables ServiceNow customers to efficiently manage the entire life cycle of their Natural Language Understanding (NLU) models. This life cycle includes building, testing, tuning, and publishing models tailored to various applications such as Virtual Agent, AI Search, or Issue Auto Resolution. The process is divided into distinct phases that guide customers through creating and refining their models to ensure optimal performance and deployment readiness.
Show less
Key Features
- Model Creation: Customers can create models by copying prebuilt read-only models, importing training data via CSV files, or starting from scratch. Models are created within the appropriate application tab in the NLU Workbench.
- Phased Model Management: The model life cycle is segmented into phases (Build and train, Test and publish, Tune) that are available based on the model’s application, providing a structured approach to model development and improvement.
- Build and Train: Add and manage model content such as intents, entities, vocabulary, and test utterances. Training uses realistic utterances to help the model learn user interactions effectively.
- Test and Publish: Test the model’s performance using the NLU Workbench - Advanced Features plugin, identify improvements, and publish the model to make it available for use by other ServiceNow applications.
- Tune Your Model: For Virtual Agent models with the advanced features installed, use the Expert Feedback Loop to incorporate actual user utterances, refining accuracy and relevance. Issue Auto Resolution models have a dedicated tuning interface.
- Model Settings: Modify model metadata such as name and description, and adjust the confidence threshold, which controls how confident the model must be before predicting an intent.
Practical Considerations
- Ensure all necessary NLU plugins, especially NLU Workbench - Advanced Features and Intent Discovery, are installed for full functionality.
- The phases and options presented are context-sensitive and will only appear when applicable to the model’s application.
- Testing and performance monitoring depend on advanced plugins available from the ServiceNow Store.
Key Outcomes
By following the Model management phases, ServiceNow customers can systematically build robust NLU models that understand diverse user intents and extract relevant entities. This approach ensures models are well-trained, thoroughly tested, and continuously improved using real user data, leading to more accurate and reliable natural language interactions across ServiceNow applications such as Virtual Agent and AI Search.
Manage your NLU model's life cycle in the NLU Workbench. Model management phases guide you through the iterative process of building, testing, and publishing your model.
Bringing your NLU model from creation to deployment requires multiple steps, separated into phases. You can return to earlier phases when you want to adjust and maintain your model.
The phases available for your model depend on the model's application. The system will display a phase, button, or function only when it applies to your model's application.
Create a model
To create a model for Virtual Agent or AI Search, navigate to . The Virtual Agent tab opens by default. Select the appropriate tab for the model you want to create.
- Use prebuilt model: Copy one of the included read-only models, and add content specific to your business.
- Import data from CSV: Upload a CSV file that contains training utterances and matched intents.
- Start from blank: Go through the process of setting up a new model from scratch.
To get started, see Creating models.
Model management phases
After creating a model, access its management phases by navigating to . Select the tab for your model's application, then the name of the model to open the Model details page on the model overview.
There are three phases on a Virtual Agent model's overview page: Build and train your model, Test and publish your model, and Tune your model. These phases guide you as you build and improve your model.
Build and train your model
Build the model by adding and managing content:- Intents: Add more intents to broaden the range of user requests that your model can understand.
- Entities: Add more entities so that your model can extract more contextual details from your users' requests.
- Vocabulary: Add vocabulary to enable the model to better understand words and phrases that are specific to your business, such as industry terms and acronyms.
- Test set: Add test utterances and their expected intents to your model's default test set.
To learn more, see Build and train your model.
Train your model using utterances that the model is likely to encounter from your users. To learn more, see Train and try your NLU model.
Test and publish your model
Test your model to gauge the performance and identify areas for improvement.
For more information on testing and thresholds, see Test and publish your model.
When you're satisfied with the results of testing, publish your model to make it available for use by other applications. For more information, see Publish your NLU model.
Tune your model
If NLU Workbench - Advanced Features is installed, and your model is created for Virtual Agent, the Tune your model phase is enabled. With this phase, you can use Expert Feedback Loop to incorporate actual user utterances into your model.
For more information, see Tune your model.
If your model is created for Issue Auto Resolution, you will be taken to IAR Tuning by selecting the name of your model in the IAR tab of the NLU Workbench homepage. For more information, see Issue Auto Resolution Tuning in NLU.
Model settings
Use the Settings page of the model overview to change the name and description of the model. You can also modify the confidence threshold of the model. The confidence threshold determines how confident the model must be to predict an intent.
For more information, see NLU model settings.