NLU Case Sensitivity

[Susan Britt]

Is it accurate that NLU is case sensitive?  I have been unable to find any documentation or community questions about it.  I am hoping it's a bug based on what I'm seeing or something simple we've missed in configuration, but thought I'd ask here before opening a HI Case.  When actually in VA chat from ESC and within the NLU Model doing tests, we noticed we get different results when typing the same phrase/utterance using all lowercase versus when we use camel case.  For example:  an Utterance for one Intent will be "i need help with beqom".  If we test the probability typing in the exact same case, we get the expected 100% match.  If we test using camel case "I need help with Beqom", the expected Intent isn't even returned.  When trying to add an Utterance with the capitalized word, add a new Vocabulary base word with the capitalize word, or even try to add the capitalized synonym (i.e. Beqom) to existing lower case base word (i.e. beqom), we get error "There was an error reading from or writing to the database".  The actual error on the NLU Vocabularies table during this attempt is "Word already exists". It's like VA and the NLU Model "Test" are case sensitive, but of course, the tables are not so you cannot add duplicate (i.e. Beqom when beqom is already there).  This is not isolated the one system name Beqom, it's just one example.  I cannot imagine it is supposed to be case sensitive, but wondering if anyone else has encountered such.  See attached for more examples with screenshots.

[Asif · Accepted solution]

Hi Sbritt,

Case sensitivity for added vocabulary can be addressed via a simple regex expression.  Hays 3ss should be added in as \b(?i)Hays 3ss\b.  This will not only account for case sensitivity, but also for misspellings, such as "hasy 3ss".  This should also solve your issue with Roodland.  Simply modify the existing vocab word so that the regex expression reads: \b(?i)Roodland\b.

HTH,

Asif

[Susan Britt]

Oh my, I never thought about using Regex in Vocabulary.  Thank you! This worked.

[Rachel38]

We have recently implemented VA in New York and were told the case sensitivity issues with utterances and vocabulary were resolved in Paris (which we were looking forward to upgrading to) but is that not actually the case? 

[Chris D]

We're in the early stages of implementing NLU and as I'm learning that utterances are case-sensitive, I'm even less impressed by it. Keywords aren't case-sensitive, why would utterances be? And why would we expect users to chat with a bot using proper casing? So now we need to either add duplicate utterances or regex for virtually every single utterance??

Thanks at least for posting this and making me aware of this before we got too deep.