Gen AI Controller - Why use it instead of Outbound API Call?

[Chris Yang]

The configuration is much more complicated than a simple REST call, what is the benefit of going through the Gen AI Controller config, especially for custom LLMs.

[SANDEEP DUTTA]

Hi @Chris Yang ,

Generative AI Controller is used to integrate third-party large language models (LLMs) with your workflows.

Using Rest call , can you integrate the external LLM?

[Chris Yang]

Yes, we make the REST call with the prompt and get a response back, which we can use in any area of the platform.

GenAI Controller is doing the same thing at the end, you can see in the Outbound HTTP logs.

[rpriyadarshy · Accepted solution]

Hi Chris

As this Point of time i see 3 flavors.

P2P Integration Using REST APIs

Gen AI Controller

ServiceNow Integrated Model Provider (Latest in the series-Model provider flexibility (ServiceNow-integrated ... - ServiceNow Community)

I see one important point when using ServiceNow Framework Vs Typical REST Based API Calls--> Sensitive Data Handling is part of ServiceNow Gen AI Framework, Also Now Assist Guradian for content moderation and adversarial attack detection guardrails.  ServiceNow provides built-in governance that is native to the platform->AI Control Tower.

These are added as a part of Framework and it makes a complete central package Considering multidimension angles of GEN AI (Data, Control , Guardrails etc etc)

Regards

RP

[Chris Yang]

This makes sense, thanks.