NLU model settings
Summarize
Summary of NLU model settings
ServiceNow’s NLU model settings enable you to customize key aspects of your natural language understanding (NLU) models within the NLU Workbench. These settings allow you to manage your model’s identification details and optimize how the model predicts intents based on confidence thresholds, helping enhance model accuracy and relevance.
Show less
Model Identification Settings
Within the model settings page, you can update the model’s name, short description, and business area to better reflect its purpose within your organization. However, the model’s language, purpose, and scope are fixed after creation and require creating a new model if changes are needed in these areas.
The “Ignore punctuation” option is enabled by default to reduce variability caused by different punctuation in user utterances, promoting more consistent intent predictions. It is recommended to keep this option active for best results.
Confidence Threshold Management
The confidence threshold defines the minimum confidence score (expressed as a percentage) that an intent must have to be predicted for a given utterance. Properly setting this threshold is critical:
- A threshold set too low may result in false positives by predicting irrelevant intents.
- A threshold set too high may exclude valid intents from being predicted.
You can configure the threshold in two ways:
- Automatic: The system dynamically selects and updates the optimal threshold based on ongoing test results, especially during the test and publish phase.
- Manual: You may set the threshold manually and optionally accept system recommendations if they improve model performance.
Prebuilt models come with pre-tuned thresholds specific to their use cases, which are not adjustable.
Threshold Recommendations and Testing
During model testing, the system may provide recommended threshold values to improve prediction accuracy. Recommendations are shown only if:
- The test set has at least 60% coverage with a minimum of 5 utterances per intent.
- The test set contains at least 100 utterances.
- The model is not prebuilt.
- The recommended threshold offers better results than the current setting.
If you apply the recommended threshold, the system automatically retrains the model and updates prediction results accordingly. Visualization tools help compare current and recommended thresholds’ impact on model predictions.
Practical Benefits
By effectively managing these settings, ServiceNow customers can optimize their NLU models to provide more accurate intent recognition, reduce false positives, and tailor models to specific business areas without needing to recreate models unnecessarily. The confidence threshold management ensures model predictions are both reliable and actionable in real-world conversational scenarios.
Change your NLU model's name, description, or confidence threshold on the Settings page of the model overview.
Access the model's settings by navigating to . Select the tab for your model's application, then your model's name.
On the model's overview, select the Model settings tab.
Model settings
In the upper section of the model settings page, you can change the model's name, short description, and business area. You cannot change the model's language, purpose, or scope. To make a model with a different language, purpose, or scope, see Creating models.
By default, the Ignore punctuation check box is active. Ignoring punctuation makes it so that there is less variance between predicted intents and confidence scores for utterances with slightly different punctuation. For best results, keep the check box active.
Model threshold settings
Here you can adjust how the confidence threshold works in your model.
A threshold is a confidence score represented by a percentage. The confidence threshold of a model determines what intents from that model will be predicted for a given utterance. For example, if the model threshold is 65%, then an intent will be predicted for an utterance only when the intent has a confidence score that is at least 65%. Setting a threshold that is too low may increase the false positives by predicting intents that should not be a match for an utterance. On the other hand, a model threshold that is too high may filter out intents that you do want to get predicted. Finding the ideal threshold improves your model's ability to predict intents correctly.
- Automatic - Allow the system to choose the optimal confidence threshold for your model. The value is updated dynamically based on test results. This happens in the Test and publish your model phase, where your model's default test set is used.
- Manual - You can manually set the confidence threshold. The system may also recommend a better threshold for the model during testing. You can choose to accept recommendations.
Prebuilt models come with a tuned threshold. The confidence threshold on prebuilt models was chosen specifically for that model.
- The test set has a Test Coverage score of at least 60%, with at least 5 test utterances per intent. For more information, see Test set creation and management.
- The test set has at least 100 utterances.
- The model is not a prebuilt model.
- The recommended threshold would have better results than the current threshold.
Test results with a recommended threshold contain a second graphic. The second graphic shows the prediction percentages with the recommended thresholds applied.
Applying the threshold recommendation may improve the prediction percentages of your model. Select Apply recommendations to change the threshold. The system automatically retrains the model, and the test results show the prediction percentages with the new threshold.