Enable AI Search in portal to support multiple languages

[lambatsaksh]

When a user’s preferred language is set to English, searching for catalog items using another language (e.g., German) does not return relevant results, despite the catalog items being properly translated and available in the translation tables.

The portal search should support cross-language recognition. For instance, if a user searches for “Ein IT-Ticket erstellen,” the system should return the corresponding English catalog item “Raise an IT Ticket,” and vice versa.

Similar to how, in certain informal or coded communication contexts, different phrases may convey the same meaning and still lead to the intended outcome, the portal search should be capable of interpreting multilingual inputs and mapping them to the correct catalog items regardless of the user’s preferred language setting.

This requirement should be fulfilled without relying on meta tags, preferably leveraging AI-enabled portal search capabilities.When a user’s preferred language is set to English, searching for catalog items using another language (e.g., German) does not return relevant results, despite the catalog items being properly translated and available in the translation tables.

[lauri457]

I don't think it is possible to enable language detection for ai search with any simple configuration. It at least looks like the locale is parameterized for the ai search graphql query so might be that you could use dynamic translation or a custom api call to detect language for the search term but it would be a customization this way.

I can imagine if the search term was applied as is to multiple languages, relevance ranking and results would be affected. If a search contains keywords like IT for example, there is no way of knowing if priority should be given to the english or german results. Or a meaningful word for the search in one language could be a stopword in another language so how to combine that?

[lambatsaksh]

Just want to update the description, let's take example., if user is Spanish user and if tries to search a catalog with spanish or english name then he should get a proper search result for that particular catalog, no matter whether user search for a catalog using spanish or english.

[lauri457]

I understand your scenario, my point is that ai search relies on the user’s session language to determine which language model, indexed content and NLP "stuff" to use so if your session is set to english, the search pipeline assumes english semantics and vocabulary. 

This could potentially be overcome by doing language detection on the search term and calling the api with the identified locale but I don't think this is a feature of ai search in any UI and I couldn't say if it is actually viable. Knowledge searching offers more control on how the search works but I don't think it is extended to other tables oob.