Hello all,
We came across something accidentally the other day which raised a bit of concern for us as a ServiceNow customer.
While typing reward into the knowledge search functionality the word 'Reward' is considered misspelled and returns the R-Slur as a suggestion.
When we noticed that we immediately reached out to ServiceNow because the auto-suggestion functionality is a third-party locked solution we don't have access to.
Now before we start our own custom solution and implement (on the client-side) our own filters and custom ban-list of words adding on to Performance degradation, I feel this is something that needs to be addressed by ServiceNow or that 3rd party implementor themselves.
I do not know if any other similar cases can happen, but going forward as a society we moved on from various words / behaviours and mindsets and this does not look like the case for the specific vocabulary database.
Not to mention the fact that why does the plugin consider the word 'Reward' misspelled in order to return results.
The response we got from ServiceNow, among other things was summed up as this:
There are 3 parts to this functionality:
a) Stemming:is responsible for creation of word families, words related to the searched phrase by its meaning, synonyms etc. The [ts_word] table is populated based on this information, and the "Did you mean" functionality uses this data.b) Spell Checker; this 3rd party library is used for suggestion based on incorrect spellings in the search phrase.c) Trends; the system stores popular trends based on the search strings of users and frequency of them. The most popular one based on its mean and logic, are clubbed with the searched phrase and become part of "Did you mean" suggestions.
The suggestions are all present in the [ts_phrase] table and associations are stored in the [ts_chain_summary] table.
The words like "retard zone", "regard zone" are displayed in DYM section because of (b) above - SPELL CHECKER
ServiceNow uses a thrid party spell checker engineer called "com.swabunga.spell.engine" (which is widely used across the globe)
This 3rd party spell checker engine returns a list of Word objects that are the suggestions to any word. If the word is correctly spelled, then this method could return just that one word, or it could still return a list of words with similar spellings.
We cannot provide a list of inappropriate words/remove existing words from appearing in spell checker result o/p as it is owned and manged by this 3rd party library (com.swabunga.spell.engine)
Looking forward to hearing everyone's opinions and if someone encountered any similar event/issue.
Thank you in advance!
