How to Integrate External LLM Models Into Virtual Agent Without Using Now Assist

[siva44]

Hi Community,

I am working on a requirement to integrate external LLM models (such as OpenAI, Azure OpenAI, Gemini, etc.) directly into ServiceNow Virtual Agent, without using Now Assist

My goals are:

1. Use an external LLM to generate responses for Virtual Agent topics.
2. Avoid Now Assist because the business requirement is to use our own licensed external LLM.
3. Ensure data security, privacy, and prevent any sensitive or PII data from being exposed to the LLM API.
4. Understand the best practices for securely invoking external LLM models from ServiceNow.