Test and publish your model

  • Release version: Zurich
  • Updated July 31, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Test and publish your model

    This process enables ServiceNow customers to assess and improve the performance of their Natural Language Understanding (NLU) models by testing them against predefined test sets and publishing the trained model for use with other applications like Virtual Agent. It is crucial for ensuring your model accurately interprets user intents before deployment.

    Show full answer Show less

    Testing Your Model

    Testing your NLU model involves running it against a default test set to evaluate how well it predicts intents from user utterances. Test results, shown via bar charts and detailed tables, categorize prediction outcomes into four types:

    • Correct: Model accurately predicted the intent or correctly identified irrelevant utterances as having no intent.
    • Correct among multiple: Model predicted the correct intent(s) but also included incorrect ones.
    • Missed: Model failed to predict any intent where one was expected.
    • Incorrect: Model predicted an intent that was wrong.

    Understanding these results helps identify areas for model improvement. Testing also influences the model’s confidence threshold, which determines how confident the model must be to assign an intent.

    Note: Testing requires the Multi-model Batch Testing feature available with the NLU Workbench - Advanced Features application from the ServiceNow Store.

    Publishing Your Model

    After training and satisfactory testing, you can publish the model, making it accessible to applications like Virtual Agent. The Publish button is only enabled once the model has been trained.

    Multi-model Batch Testing

    This advanced feature allows you to test multiple models simultaneously and against various test sets beyond the default one, providing greater flexibility and comprehensive evaluation. Access this feature via NLU Workbench > NLU Advanced Features > Multi-model Batch Testing.

    Practical Steps for ServiceNow Customers

    • Navigate to NLU Workbench > Models, select your model, and open the Test and publish your model card.
    • Run tests to view performance summaries and detailed prediction results.
    • Use test results to fine-tune your model by adjusting training data or confidence thresholds.
    • Once satisfied with model accuracy, publish the model to enable integration with Virtual Agent and other applications.
    • Consider leveraging Multi-model Batch Testing for broader testing needs.

    Assess the performance of your NLU model to identify areas for improvement. Then publish your model to make it available to other applications such as Virtual Agent.

    Summary usage

    Test your Virtual Agent or AI Search model against its default test set to see how the model responds. Test results provide information you can use to improve your model.

    Note:
    Testing your model requires the Multi-model Batch Testing feature, available with the NLU Workbench - Advanced Features application from ServiceNow® Store. For more information, see Install NLU Workbench - Advanced Features.

    To test your model, navigate to NLU Workbench > Models. Select the tab for your model's application, then select the name of the model. In the Test and publish your model card, select View phase. Test and publish your model phase card

    Overview of testing and publishing your model

    The Test and publish your model phase opens in the Overview page by default. Buttons for Run new test and Publish model are located here.

    Test and publish your model overview page

    Overview provides information about a previous test run, with bar charts summarizing the test results.

    If you have earlier test runs, you can view those by selecting from the Test run date list.

    Test run date pulldown

    To drill down into the test results table, select the Detailed results tab. Each test utterance is listed in Detailed results, with its prediction.

    Understanding test results

    The test results show how your model responded to the utterances in the test set.

    Test results for a model test in the NLU Workbench.

    The bar chart shows the prediction percentages for correct, correct among multiple, missed, and incorrect:
    Percentage Description
    Correct The percentage of utterances for which your model correctly predicted the intent.

    When the model predicts no intent for utterances marked as Not relevant, that result is counted as Correct.
    Correct among multiple

    For utterances that had more than one intent predicted.

    The percentage of utterances for which the model correctly predicted the intent or intents, but also predicted intents that did not belong to the utterance.
    Missed The percentage of utterances for which your model did not predict an intent, even though there was an expected intent.
    Incorrect The percentage of utterances for which your model predicted an intent that was not correct.

    Testing can affect the model's confidence threshold. The confidence threshold determines how confident a model must be to predict an intent for an utterance. For more information on confidence thresholds, see NLU model settings.

    For information about utterances which should not have any intent predicted, see Irrelevance detection in NLU.

    Publish model

    The Publish model button makes the current version of the model available to other applications such as Virtual Agent.
    Note:
    If the model has not been trained, the Publish model button is unavailable. Return to the Build and train your model phase to train the model before publishing.

    For more information on publishing your model, see Publish your NLU model.

    Multi-model Batch Testing

    In the Test and publish your model phase, you test your model against its default test set. With Multi-model Batch Testing, you can test against other test sets, test multiple models at once, and see your test results. To use Multi-model Batch Testing, navigate to NLU Workbench > NLU Advanced Features > Multi-model Batch Testing.

    For more information, see Multi-model Batch Testing.

    For more information about test sets, see:

    For information about the process of testing, see Test your model.