Model Choice Without Lock-In: Keeping Your LLM Options Open on ServiceNow

sachinbhasin11 · 2 hours ago

Model liquidity: the freedom to switch

Model liquidity is your ability to move a use case from one model to another with little or no friction. It sounds like an engineering detail. It is actually one of the most important governance decisions you will make.

In the first post of this series we looked at what happens to your data and how to route workloads by sensitivity. Both depend on this one. If you cannot leave a provider, you cannot really negotiate with them.

Why depending on one model is a risk

The last few years have shown how many ways a single model dependency can hurt you:

Outages. When the only model behind a workflow goes down, the workflow goes with it.
Policy changes. Data retention terms, usage policies and pricing can all change after you have built on them.
Shifting refusals. A model update can start declining requests your process relies on.
Regional and regulatory limits. Some models are not available in every region or regulated environment.

With one provider, each of these is a business continuity problem. With several viable options, each is an inconvenience.

There is an upside too. Different models are good at different things, and new ones arrive constantly. Language models are unlikely to be the last form of AI you adopt. A setup that lets you slot models in and out means you can use the best option for each task, rather than the one you happened to start with.

Switching ability is negotiating power

Recall from post 1 that zero data retention has to be negotiated, provider by provider. Negotiation only works when you have somewhere else to go. If a provider wants to change how your data is handled, the most persuasive answer you can give is that you will move the workload.

That answer is only credible if moving is actually cheap. It also compounds. A strong data protection term won with one provider becomes the benchmark you bring to the next.

There is a natural pull in the other direction. Every extra provider means another contract to review and another set of terms to track. Consolidating on one looks efficient. Resist that pull for anything strategic. Keep at least one credible alternative tested and ready for your most important AI use cases.

Make switching routine with continuous evaluation

Switching only stays cheap if you practise it. The habit that makes it practical is a standing evaluation loop: a fixed set of real examples with known good answers, which every candidate model and prompt change is scored against.

With that loop in place, a new model is a test run, not a project. Cheaper models take over wherever quality holds, and quality improves wherever a better model wins.

What model choice looks like on ServiceNow

ServiceNow has steadily opened up model choice. Based on current documentation and community guidance, the main levers are:

Option	What it gives you	Where it is managed
Now LLM Service	ServiceNow's own models, the default for Now Assist	Default; no setup
Supported third-party providers	Switch out-of-the-box skills to a supported external model	Enable in AI Control Tower, then set in Now Assist Admin; at instance, skill group or skill level
Bring your own key (BYOK)	Requests run through your own provider account and credentials	Model configuration in the Generative AI Controller
Bring your own LLM (BYOLLM)	Connect a model outside the certified providers through a generic connector	Now Assist Skill Kit, for custom skills

A few practical points:

Not everything can switch. Some out-of-the-box skills, agents and agentic workflows support only specific providers. Check ServiceNow's exception list before you plan a migration.
Bringing your own shifts responsibility. With BYOK or BYOLLM, the provider's licence and data terms are yours to manage. That is where the zero-retention questions from post 1 come back.
Know which model answered. Generative AI log metadata records which model and version produced each response. That record is essential when you compare models or investigate an issue.
Design custom skills to be portable. Keep prompts, inputs and outputs well defined in Now Assist Skill Kit, so changing the provider is a configuration change, not a rebuild.

Product names and capabilities move quickly in this area. Confirm what is available on your release and licence before committing to a design.

A quick checklist

List your production AI use cases and the model behind each one.
For each critical use case, identify and test at least one alternative model.
Build a gold set of real examples for every critical skill, so you can compare models objectively.
Re-run those evaluations whenever a model is updated or a new one becomes available.
Confirm data handling terms for every external provider before enabling it.
Note which skills cannot switch providers today, and track them as a known dependency.

Models are becoming interchangeable. What is not interchangeable is the knowledge your organization builds up as it works. In the next post, we look at why that knowledge should live in your platform rather than in any model.