Need Guidance: Chat-like Interface in ServiceNow using ServiceNow AI (RAG & Vector Search)

[GautamK06956547]

Hi Community,

I’m working on a requirement where I need to build a chat-like interface in ServiceNow.

The idea is:

• For a particular record, a user will upload a document into ServiceNow.

• The AI model should be trained only on that document’s data.

• If the user asks anything in natural language, the response should come only from that uploaded document.

• If no relevant answer is found, it should simply say “No result found.”

I have already achieved this using OpenAI’s Assistant API integration, but my client rejected it due to data trust/security concerns.

Since ServiceNow also provides AI capabilities with RAG (Retrieval Augmented Generation) and vector search, I wanted to check:

 Is it possible to implement this use case using ServiceNow AI features only (without external APIs)?

 Has anyone in the community tried something similar or has experience with this setup?

Any guidance, experience sharing, or pointers would be highly appreciated. 

Thanks!

[Laurent5]

Hi @GautamK06956547 ,

In order to achieve your use case, you can use RAG indeed, i.e the model won't be trained on your document data (as models are Pre-trained) but can be augmented with it.

The way to add your document is to attach it to a Knowledge article and then Now Assist will be able to leverage its content for summarization.

Below is an example screenshot (in Portal but same approach will be used with VA, you just need to configure your AI Search profile accordingly):

You can see that the summary is based on 2 docs (a pdf and a .doc)

I know its not what you are after but I wrote an article a while back using external tools (Langchain) but that was before this capability was available on the platform:

https://medium.com/@laurent-roussel/leveraging-external-llms-and-rag-with-servicenow-virtual-agent-l...

Hope it helps.

Laurent