Has anyone successfully replaced traditional Knowledge Search with AI Search?

[Akhila1]

Has anyone successfully replaced traditional Knowledge Search with AI Search? What improvements and challenges did you encounter?

• Did you notice a significant improvement in search relevance and user experience?
• Were there any challenges with indexing, search sources, or knowledge article ranking?
• How much configuration/tuning was required before users started seeing meaningful results?
• Are there any limitations or lessons learned that aren't obvious from the documentation?

I'd love to hear about your experiences, especially around ITSM and Knowledge Management use cases.

[k_lutz]

Just some notes:

• Did you notice a significant improvement in search relevance and user experience?
• No but this is because we enabled AI Search using our existing knowledge bases and knowledge articles. They are not setup in the best way for the system to utilize them for the AI search. They work about the same. I would suggest reviewing some of the postings on the community as to the best way to structure knowledge articles for RAG consumption.
• Were there any challenges with indexing, search sources, or knowledge article ranking?
• Other than the one noted above we did run into issues around the setup of the Search Results Action area. A small misalignment here or a missing entry will have weird results like your catalog item opening with “form” in the URL so it looks like the user is trying to access the cat item through the platform side.
• How much configuration/tuning was required before users started seeing meaningful results?
• This is ongoing as we have a team looking at the AI searches being done via the chatbot on the portal to see where / why people are not getting the responses they want. This then helps identify missing knowledge, incorrectly or poorly structured articles.
• Are there any limitations or lessons learned that aren't obvious from the documentation?
• I found more assistance going through the community and testing issues in our environment. I am not a fan of the DOC site as it is very “choppy” and not useful if you are truly trying to go through the setup especially in an instance that is not a “green field”.
• The biggest one is to make sure the knowledge is very well structured for the AI search to consume.
• If your contract has it, I would look into External Content Connectors as I tried those in DEV and worked really well to get knowledge off of websites.