NLU Workbench properties

  • Release version: Yokohama
  • Updated January 30, 2025
  • 3 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of NLU Workbench properties

    The NLU Workbench properties in ServiceNow control various settings for the Natural Language Understanding (NLU) application, enabling customers to optimize their NLU models and processes. These properties are accessible by users with theadminornluadminroles viaAll > NLU Workbench > Settingsin the application navigator. They help control limits on utterances, vocabulary sizes, intent discovery parameters, conflict detection thresholds, feedback loop configurations, and model training schedules.

    Show full answer Show less

    Model Settings

    • Maximum utterances per intent: Recommended to keep under 200 to maintain balanced model size (must be >5 and ≤300).
    • Maximum records in Table and List vocabulary sources: Keep under 100,000 for Table vocabulary and under 1,000 for List vocabulary to ensure performance.
    • Enable pre-built vocabularies for software and hardware names: Allows the system to recognize these entities automatically, improving recognition accuracy.

    Advanced Settings

    • Intent Discovery classification record limits: Use between 10,000 and 500,000 records for high-quality results.
    • NLU performance analysis minimum records: At least 5,000 records recommended for reliable analysis.
    • Conflict Detection thresholds: Moderate (default 0.85) and Critical (default 0.95) thresholds must be decimals between 0 and 1, with the critical threshold higher than moderate.
    • Batch test import file row limit: Maximum 10,000 rows per import file to ensure smooth batch testing.
    • Expert Feedback Loop settings:
      • Display up to 300 utterances from Virtual Agent chat logs for feedback.
      • Require at least 100 user-reviewed utterances before tuning the model.
      • Fetch up to 3,000 Virtual Agent chat log records by default; can increase to 50,000 if needed for high NLU usage.
      • Size limits on label candidate and labeled data tables set to 10,000 for pruning purposes.
    • Scheduled NLU model training: Disabled by default; enabling it schedules training during off-peak hours to avoid instance impact, with notifications upon completion.

    Practical Implications for ServiceNow Customers

    These properties allow you to fine-tune your NLU models by controlling data input sizes, vocabulary recognition, training schedules, and feedback loops. Proper configuration ensures balanced intent models, efficient processing, and higher-quality intent discovery and performance analysis. Using pre-built vocabularies enhances entity recognition, while conflict detection thresholds help identify ambiguous intents. Managing batch test sizes and feedback data improves testing and model tuning workflows. Scheduled training minimizes system disruption.

    Adjust these settings thoughtfully to match your organization’s data volume and usage patterns for optimal NLU application performance and model accuracy.

    Refer to these system properties for the Natural Language Understanding (NLU) application.

    NLU Workbench properties and their usage

    To access your system properties, use the admin or nlu_admin role and the following path in the application navigator: All > NLU Workbench > Settings.

    Table 1. Model Settings
    Label and Name Default value Plugin Recommended usage
    Maximum number of utterances per intent

    glide.nlu.utterances_per_intent.value_limit

    		 200 NLU Workbench Use fewer than 200 utterances per intent to keep your model well balanced in terms of intent size.
    Note:
    Value must be greater than 5 and less than or equal to 300.
    Maximum number of records in a Table vocabulary source

    glide.platform_ml.api.max_nlu_lookupsource_records

    		 100,000 NLU Workbench Keep the value under 100,000.
    Maximum number of values in a List vocabulary source

    glide.nlu.static_lookup.value_limit

    		 1,000 NLU Workbench Keep the value under 1,000.
    Enable pre-built vocabulary for software names

    glide.mlpredictor.option.nlu.@LookupSources:software

    		 enabled NLU Workbench Enable pre-built vocabulary so the system can recognize software names.
    Enable pre-built vocabulary for hardware names

    glide.mlpredictor.option.nlu.@LookupSources:hardware

    		 enabled NLU Workbench Enable pre-built vocabulary so the system can recognize hardware names.
    Table 2. Advanced Settings
    Label and Name Default value Plugin Recommended usage
    Maximum number of records for Intent Discovery classification

    sn_nlu_discovery.intent_discovery_max_classification_limit

    		 300,000 Intent Discovery Keep the number of records less than 500,000.
    Minimum number of records for Intent Discovery classification

    sn_nlu_discovery.intent_discovery_min_classification_limit

    		 10,000 Intent Discovery Use at least 10,000 records to get high quality results.
    Minimum number of records for NLU performance analysis

    sn_nlu_workbench.glide.nlu.performance.min_clustering_records

    		 5,000 NLU Workbench - Advanced Features Use at least 5,000 records to get high quality results.
    NLU Conflict Detection - Moderate Threshold

    sn_nlu_workbench.glide.nlu.conflict.moderate_threshold

    		 .85 NLU Workbench - Advanced Features Must be a decimal between 0 and 1. Keep this threshold less than the Critical Threshold.
    NLU Conflict Detection - Critical Threshold

    sn_nlu_workbench.glide.nlu.conflict.critical_threshold

    		 .95 NLU Workbench - Advanced Features Must be a decimal between 0 and 1. Keep this threshold greater than the Moderate Threshold.
    The maximum number of rows in a batch test import file

    sn_nlu_workbench.glide.nlu.batch_test.max_import_rows

    		 10,000 NLU Workbench - Advanced Features Make sure your batch test import file has no more than 10,000 rows.
    The maximum number of utterances to display for feedback in the expert feedback loop

    glide.mlpredictor.option.nlu.activeLearning.label_candidate_table.max_response_size

    		 300 NLU Workbench - Advanced Features Pull no more than 300 utterances from your users' Virtual Agent chat logs to display for feedback in the Expert Feedback Loop application.The minimum umber of utterances a user should review before tuning the model
    The minimum number of utterances a user should review before tuning the model

    sn_nlu_workbench.glide.nlu.optimize.min_labeled_data

    		 100 NLU Workbench - Advanced Features Provide and save feedback for at least 100 utterances from your users' Virtual Agent chat logs so you can execute the Tune Model feature in the Expert Feedback Loop application.
    The maximum number of records to fetch from Virtual Agent chat logs

    glide.mlpredictor.option.nlu.activeLearning.va_chat_logs.max_row_limit - 3000

    		 3,000 NLU Workbench - Advanced Features If there is high NLU usage, increasing the default value to a maximum of 50,000 records will increase the data available for the active learning job to filter up on and display in the Expert Feedback Loop application to give feedback on.
    Size limit on Label Candidate Table (used for pruning the table)

    glide.mlpredictor.option.nlu.activeLearning.label_candidate_table.max_data_size - 10000

    		 10,000 NLU Workbench - Advanced Features The recommended usage for this property is the same as the property above.
    Size limit on Labeled Data Table (used for pruning the table)

    glide.mlpredictor.option.nlu.activeLearning.label_table.max_data_size - 10000

    		 10,000 NLU Workbench - Advanced Features The recommended usage for this property is the same as the property above.
    Enable this property to unblock your instance during NLU model training. The training will be scheduled for an off-peak time, and we will notify you when it's done.

    glide.mlpredictor.scheduled.nlu.model.training

    		 False NLU Workbench - Advanced Features False

    To get more feedback data from Virtual Agent (VA) chat logs, refer to the Procuring additional VA feedback data on demand section in the Expert Feedback Loop documentation.