Max Number of Utterances Per Intent

[Shaksham]

Hello,

I am a bit confused regarding the limit on maximum number of utterances per intent. We have two models with the given config:-

• Model 1 - 200 max utterances and 5 min utterances per intent. Total # of observations = ~4800 (70 intents)
• Model 2 - 300 max utterances and 5 min utterances per intent. Total # of observations = ~5300 (70 intents)

Now my questions are:-

• In this document, it states - "The system currently supports up to 300 intents or 4,500 utterances (whichever comes first) per model."
• In my case, as you can see, we have crossed the fore mentioned limit. So, I am wondering while model training, will the model consider ONLY the first 4500 utterances in either case?
• Also, what are some other caveats associated with increasing the number of utterances per intent? I know if we have an intent with more utterances, it will skew the predictions, but is there anything else that will get affected?

Thanks

[Community Alums]

Hi @Shaksham ,

There are few properties which decides the Number of Intents and Utterances.

Maximum number of utterances per intent

glide.nlu.utterances_per_intent.value_limit : value is 200 as default.

Maximum number of records for Intent Discovery classification

sn_nlu_discovery.intent_discovery_max_classification_limit : Default Value is 300,000 

Minimum number of records for Intent Discovery classification

sn_nlu_discovery.intent_discovery_min_classification_limit : Default Value is 10,000.

Refer to this doc:https://docs.servicenow.com/bundle/quebec-now-intelligence/page/administer/natural-language-understa...

Mark my answer correct & Helpful, if Applicable.

Thanks,

Sandeep

[Filipe Cruz]

@Brian Bakker think this is a post for you! 🙂

[Brian Bakker · Accepted solution]

Hello Shaksham,

The number of utterances per NLU model is not an exact figure, as the NLU Model size limitation is in Megabytes and so it depends on the length of your example utterances, as they have a limit of 25 words or 200 characters and the number of utterances in the NLU Model. We have increased the size of the NLU Model with each version of the ServiceNow Platform, as follows -

In Rome,

• The system currently supports up to 300 intents or 4,500 utterances (whichever comes first) per model.

You should have a maximum of 200 example utterances in one intent. If an intent has that many example utterances, I suspect there are many duplicate examples and/or you are not using Vocabulary to optimise the number of example utterances that are needed. Avoid having intents with extreme ranges of example utterances, as this skews predictions towards the intents with the most examples. This can result in incorrectly predicted intents for some test utterances, but it depends on many factors, such as utterance uniqueness between intents. Hence, we have the NLU Workbench - Advanced Features, which is a set of tools that will really help with your NLU implementation.

You are not exceeding the number of intents in your NLU Models, but you are exceeding the total number of utterances for your NLU Models. However, if the NLU Model training is not timing out, then it will have trained all your example utterances in your NLU Model, even if beyond the 4,500 utterances limitation. For further details as to what happens when there are too many utterances in your NLU Model, please review KB0827206: [NLU/VA] Too many utterances in your NLU Model will cause a "Read timed out" and it is un....

Note: The limitation in Quebec is 3,000 utterances 

In San Diego,

• A model now supports 750 intents or 20,000 utterances. However, models with over 300 intents or 4500 utterances, whichever comes first, take longer to train, test, and publish.

We have introduced asynchronous NLU Model training for both larger NLU Models and certain language specific NLU Models and so you will need to ensure the ML configuration is setup correctly to train these NLU Models. For further details, please review KB0996623: [NLU] Training Brazilian Portuguese, Canadian French, Danish, Finnish, Norwegian and Swed....

Therefore, if your NLU Model now exceeds 300 intents or 4500 utterances, give or take, then it will switch to asynchronous NLU Model training, which supports sizes up to 750 intents or 20,000 utterances (whichever comes first) per NLU model. Asynchronous NLU Model training can take 10x longer (10-30 mins depending on NLU Model size) than synchronous NLU Model training, so you need a bit more patience for asynchronous NLU Model training to complete.

Note: To view the above KB articles, you have to be logged into the ServiceNow Support web site.

Please also review the following Community articles for further tips on implementing your custom NLU model -

• NLU FAQ, best practices, and general troubleshooting (as of San Diego release)
• NLU Best Practice - Using Vocabulary & Vocabulary Sources

I hope this helps.

Regards,

Brian