NLU: Extract Word from Utterance

Raj64
Tera Guru

Hello everyone,

If I have an utterance like: What is the temperature in X

How do I make "X" dynamic. In other words, "X" can be anything. Eg: What is the temperature in Toronto; What is the temperature in Bangkok; What is the temperature in Atlanta

When I have the value of "X", I want to use it in a VA topic.

Thank you.

1 ACCEPTED SOLUTION

Brian Bakker
ServiceNow Employee
ServiceNow Employee

Hello Raj,

If you don't want to maintain a list of Xs or "cities" using your example, you can Create an open-ended entity called "Cities" and then annotate the city names as the open-ended entity "Cities" on a few sample utterances within an intent in your NLU Model, such as -

  • What is the temperature in Bangkok
  • How hot is it in London
  • Can you tell me what the temp is in Sydney
  • How many degrees is it in New York
  • Is it cold in Moscow

When we send the utterance to the Prediction server, it essentially removes the city from the utterance before we make a prediction, because the user could type in any city in the world such as "Wellington". However, when it predicts your intent with the "Cities" open-ended entity based on the utterance "What is the temperature in Wellington", it should recognise "Wellington" as the value for the "Cities" entity in the prediction result.

An intent should have at a minimum 5 sample utterances and you should start with 15 sample utterances to improve prediction scores. The key is that each utterance should be unique in the way of saying the same thing, but in a different way. Avoid repeat utterances, such as -

  • What is the temperature in Bangkok
  • What is the temperature in London
  • What is the temperature in Sydney...

I have seen many NLU Models with repeat utterances and it can negatively impact prediction scores. All intents in an NLU Model should have an equal distributed number of 20-50 sample utterances, otherwise if one or two intents have 150+ sample utterances, it can skew predictions towards those intents with more samples.

I hope this helps.

Regards,

Brian

View solution in original post

4 REPLIES 4

Brian Bakker
ServiceNow Employee
ServiceNow Employee

Hello Raj,

If you don't want to maintain a list of Xs or "cities" using your example, you can Create an open-ended entity called "Cities" and then annotate the city names as the open-ended entity "Cities" on a few sample utterances within an intent in your NLU Model, such as -

  • What is the temperature in Bangkok
  • How hot is it in London
  • Can you tell me what the temp is in Sydney
  • How many degrees is it in New York
  • Is it cold in Moscow

When we send the utterance to the Prediction server, it essentially removes the city from the utterance before we make a prediction, because the user could type in any city in the world such as "Wellington". However, when it predicts your intent with the "Cities" open-ended entity based on the utterance "What is the temperature in Wellington", it should recognise "Wellington" as the value for the "Cities" entity in the prediction result.

An intent should have at a minimum 5 sample utterances and you should start with 15 sample utterances to improve prediction scores. The key is that each utterance should be unique in the way of saying the same thing, but in a different way. Avoid repeat utterances, such as -

  • What is the temperature in Bangkok
  • What is the temperature in London
  • What is the temperature in Sydney...

I have seen many NLU Models with repeat utterances and it can negatively impact prediction scores. All intents in an NLU Model should have an equal distributed number of 20-50 sample utterances, otherwise if one or two intents have 150+ sample utterances, it can skew predictions towards those intents with more samples.

I hope this helps.

Regards,

Brian

Raj64
Tera Guru

Thank you Brian. This is exactly what I wanted. I was playing around with patterns, but wasn't getting anywhere.

Thanks again!

jeffrubinoff
ServiceNow Employee
ServiceNow Employee

Brian's link to a topic in Rome documentation gets redirected somewhere useless. Instead, use this link to Create an open-ended entity (washingtondc release) 

Thanks @jeffrubinoff - I updated my original link to Washington docs.