RAG in ServiceNow takes a user query, rewrites and embeds it, retrieves relevant chunks from vectorized knowledge, re-ranks them, and feeds them into Now LLM to generate a grounded, reliable response.
Lets try to understand above flow
-> Search Query
- A user asks a question in Virtual Agent (e.g., “How do I reset my password?”).
- This raw query enters the RAG pipeline.
-> Query Rewrite
- The system refines the query to improve retrieval.
- Example: “reset password” → “password reset procedure ServiceNow”.
- This helps match both semantic and keyword searches.
-> Embedding Model
- The rewritten query is converted into a vector representation (numerical embedding).
- This allows semantic similarity search, not just keyword matching.
-> Hybrid Search (Semantic + Keyword)
- Two searches happen in parallel:
- Semantic search: Finds chunks of text similar in meaning to the query.
- Keyword search: Matches exact words/phrases.
- Results are combined for better coverage.
-> Data Sources (Vectorized)
- Knowledge articles, catalog items, or external docs are pre‑processed:
- Chunking: Large documents are split into smaller pieces.
- Retrieval: Relevant chunks are pulled based on embeddings/keywords.
- Data Store: These chunks live in a vector database inside ServiceNow.
-> Relevancy Re-ranker (Vectorized)
- Retrieved chunks are scored and reordered.
- The most relevant pieces rise to the top, reducing noise.
-> Now LLM (Augmented Generation)
- The reranked chunks are passed into the Now LLM.
- Instead of generating answers from scratch, the LLM uses retrieved context to build a response.
- This is the “augmentation” part of RAG.
-> Grounded Response
- The final answer is returned to the user.
- It’s “grounded” because it’s based on actual ServiceNow knowledge sources, not just free‑form LLM text.
- Example: “To reset your password, go to the Employee Center → My Profile → Security Settings.”
I have created custom skill from scratch which is using ServiceNow knowledge article as source to answer user query.
Refer the attached document containts the high level steps to create custom skill using RAG tool by leveraging NOW LLM Model.
