NLU Model Optimize was introduced in Rome release for English models as part of NLU Workbench - Advanced Features plugin to help further improve the performance of customer-created models.
Model Optimize should be run after running batch tests and tuning the model against those results. Then we can use the same test set to further tune the model with the Optimize feature. This feature is available from the NLU Batch Testing module and also as part of the Publish phase in NLU Workbench in the San Diego release.
Once model optimization completes, we can review the prediction results before and after optimization, then choose to accept or decline the optimization as needed. It is also recommended to review and compare the prediction results of the current and optimized model prior to publishing the model.
Some key pointers about NLU Optimize:
- Available in English as of Tokyo release
- Requirements: test set >100 utterances, covers > 25% of intents in the model
- Out of the box read-only models come pre-Optimized and Optimization is not available for these models
- Ensures optimal performance by leveraging ServiceNow Language Model using advanced algorithms from the most recent research
- Helps improve the prediction capabilities of your models by minimizing irrelevant detection and skipping such predictions
- Perform Optimize as a final step from Batch Testing tool or Publish phase (San Diego) before the model is ready to be published
- Optimize does not modify the physical model (intents, utterances etc.) and it is recommended to re-optimize when changes are made to the model
- NLU Model Optimize - FAQs
How Optimize helps improve model performance
Virtual Agent Academy: Improve your NLU performance with Model Optimization
Additional NLU Related Resources:
- NLU Documentation
- Now Learning – NLU Fundamentals
- Virtual Agent Academy
- Virtual Agent and NLU Quick Start Guide
- In-Depth Guide rails to building good NLU Models
- NLU FAQ, best practices, and general troubleshooting (San Diego release)
- NLU Best Practices – Using Vocabulary & Vocabulary Sources
- NLU Testing Capabilities and Techniques for your NLU Models
- Best Practices: single v. multiple NLU models
- NLU Model Optimize – FAQs
- Using NLU Model Optimize to Tune your Model
- Migrating VA and NLU between instances with update sets
- Virtual Agent and NLU Implementation
- Guided Overview to Implementing Multilingual NLU Models in NOW platform
Additional NLU troubleshooting KBs:
