NLU vocabulary
Summarize
Summary of NLU Vocabulary
NLU vocabulary items in ServiceNow help your system accurately recognize and interpret the diverse ways users express their requests. These vocabulary items improve intent prediction by mapping various words, phrases, or patterns to defined synonyms, enhancing confidence and consistency in recognizing user inputs.
Show less
Vocabulary sources complement this by referencing lists or tables (such as conference rooms or catalog items) to efficiently handle groups of related terms without listing them individually in training utterances. These sources can be reused across multiple NLU models.
Vocabulary Types and Usage
- Regular Vocabulary Items: Words or phrases, including acronyms or industry-specific terms, that are case insensitive and mapped to a single synonym.
- Pattern Vocabulary Items: Use regular expressions (regex) to capture specific formats like email addresses or to enforce case sensitivity, enabling precise identification of terms that might otherwise be ambiguous.
Vocabulary items are best used for individual terms or phrases mapped to one synonym. Vocabulary sources are suitable when referencing sets of objects from ServiceNow tables or static lists, enabling efficient reuse and simplifying training data.
Practical Application in Intent Training
When building intents, adding relevant vocabulary items can resolve recognition issues. For example, defining an acronym like “sfcrm” as a vocabulary item with a synonym “CRM software” raises the model’s confidence in predicting the correct intent. Using pattern vocabulary items can help distinguish common words from domain-specific terms, such as recognizing “Word” as software rather than a generic term.
Use the @ symbol to reference vocabulary sources in training utterances, and annotate entities to extract specific values during Virtual Agent conversations. Mapping multiple vocabulary items to one synonym is supported, but one vocabulary item should not map to multiple synonyms.
Regex and Pattern Vocabulary Tips
Pattern vocabulary items can enforce case sensitivity and specific word boundaries using regex, preventing incorrect matches (e.g., distinguishing the acronym “IT” from the word “it”). You can modify case sensitivity in regex with inline flags like (?i) for case-insensitive matching.
Use NLU vocabulary items to help the system recognize the various ways your users express their requests. Use vocabulary sources to help the system recognize objects in tables or lists, such as names of conference rooms or catalog items.
Vocabulary usage and context
Vocabulary helps your model with the various words and phrases that it may encounter from your users. Since humans are not computers, they may come up with different ways to say the same thing. For example, if one user inputs an acronym rather than the full phrase, the system might be able to predict the correct intent by using the context in the utterance. However, by defining the acronym with a vocabulary item, you raise the model's confidence level and capability to predict intents correctly.
Vocabulary items cover the varying words or phrases that might appear in utterances. The vocabulary items are mapped to the synonyms that you provide, for intent prediction. This helps the system recognize the various ways your users express their requests, while at the same time ensuring the consistency, confidence, and accuracy of predictions.
Vocabulary item types
| Type | Definition |
|---|---|
| Regular | A word or phrase that is not commonly known, such as a business or industry specific term or acronym. Regular vocabulary is case insensitive, so all case variations will be captured by the vocabulary you create. |
| Pattern | A regular expression (regex) that can capture specific formats such as email addresses and phone numbers. |
Vocabulary sources
- Use a vocabulary item for an individual word, phrase, or pattern that can easily be mapped to a single synonym for the model to use.
- Use a vocabulary source to reference a ServiceNow table or list so that the values can all be replaced by the single synonym you define. Vocabulary sources can be reused across all your NLU models.
Use vocabulary sources when your user utterances refer to objects in a set. The sources can be referenced from a list of values from a ServiceNow table or a static list that you define, such as a list of catalog items or conference rooms. Once you create and save these sources, you can use the @ symbol to specify them in training utterances. You can also use these sources as entity values.
For example, say you already have a list of all the conference rooms for your office. You can create a vocabulary source to reference that list rather than typing each conference room name into your intent's training utterances.
To create vocabulary sources, refer to Create a table vocabulary source and Create a list vocabulary source.
Vocabulary usage in relation to an intent
| Utterance | Issue and Solution |
|---|---|
| "I need to access sfcrm" |
|
| "I need to install Word" |
|
Use a single word or short phrase as a synonym for best results. You can map multiple vocabulary items to one synonym. Do not map one vocabulary item to multiple synonyms.
For more context and examples, see Create an NLU intent.
A Regex example for a pattern vocabulary item
Let's say that you want to use a vocabulary item to identify the acronym IT, and map it to the synonym information technology.
Regular vocabulary items are case-insensitive by default. This means that a regular vocabulary item would match both the acronym IT and the common word it. So you decide to use a pattern vocabulary item.
The regex pattern that you can use is \bIT\b. The \b is a word boundary marker. This ensures that the pattern would not match ITSM or JIT, for example. The default case sensitivity of pattern vocabulary items means that \bIT\b would not match the common word it.
Regex details: You can turn off case sensitivity in a pattern by using (?i). You can end that mode by using a minus sign as in (?-i). For example, (?i)te(?-i)st should match both test and TEst, but not teST or TEST.
For more information, see Using regular expressions in entities.