---
sourceDocument: Zurich Enable AI
sourceDocumentLink: https://www.servicenow.com/docs/r/zurich/intelligent-experiences

 Release :

    - zurich

ft:locale :

    - en-US

ft:publication_title :

    - Zurich Enable AI

ft:clusterId :

    - platai

bundleId :

    - platai

workflow :

    - Platform


---

# Understanding LLMs

# Understanding large language models (LLMs) {#ariaid-title1}

Release version: Zurich  
Updated March 12, 2026  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 2 minutes to read
Summarize  
![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI  
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.  

## Summary of Understanding large language models (LLMs)

Large language models (LLMs) like ChatGPT or Copilot are advanced AI systems that generate human-like language dynamically.
Unlike retrieval-based systems, LLMs produce responses probabilistically, which means they do not return identical answers every time a question is asked.
This inherent variability enables flexibility, creativity, and adaptability in their outputs.
Show full answer Show less  

## How LLMs Work

LLMs are trained on vast amounts of text data to develop a statistical understanding of language. They generate answers one word or token at a time by predicting the most likely next word based on learned probabilities rather than retrieving fixed responses. This probabilistic approach leads to non-deterministic results, meaning repeated queries may yield different, yet valid, answers.

## Reasons for Variation in Responses

* **Probabilistic Sampling:** The model samples from several likely word options rather than always choosing the single most probable word, introducing natural variation.
* **Temperature Settings:** This internal parameter controls randomness---higher temperature values increase creativity and variability, while lower values favor more repetitive outputs.
* **Multiple Valid Answers:** Many questions can be answered correctly in various ways, so phrasing or emphasis may differ across responses.
* **Context Sensitivity:** Minor changes in punctuation or conversation history can influence the generated output.
* **System-Level Factors:** Hardware differences, floating-point calculations, and backend updates can cause slight output variations even with identical inputs.

This randomness is purposeful, ensuring the model remains versatile rather than rigid.

## Implications for ServiceNow Customers

When using the ServiceNow AI Platform®, customers should expect that AI-powered search tools and LLM-based features may return different answers for the same or similar queries. These variations are normal and reflect the probabilistic nature of the underlying models. Understanding this behavior helps set appropriate expectations around the flexibility and creativity of AI-driven responses.

For detailed information on supported models and differences among AI search tools, customers can refer to the ServiceNow AI Platform documentation.  
Large language models are generative, not retrieval-based. They create responses dynamically using probability, which means you can't expect identical outputs every time. This variability is a feature, not a bug, because it
allows for flexibility, creativity, and adaptability.

## How LLMs work {#sn-ai-understanding-llms__section_jqc_55j_jhc}

Large language models (LLMs), like ChatGPT or Copilot, are advanced AI systems that are trained on massive amounts of text to understand and generate human-like language. They build a statistical model of language, so they don't store fixed answers like an
encyclopedia. When you ask a question, the model generates an answer one word (or token) at a time, choosing the next most likely word based on probabilities learned during training. This prediction process makes them powerful, but it's
also why they are non-deterministic. This means the system does not always produce the exact same result (output) for the same prompt (input).

## Why results may vary {#sn-ai-understanding-llms__section_s1j_dwj_jhc}

Even if you provide the same question or prompt twice, the response can differ. Here's why:

Probabilistic sampling
:   The model doesn't always pick the single most likely word. It samples from several likely options. This introduces variation.

Temperature settings
:   Temperature controls randomness, and this internal parameter varies among LLM models. A higher temperature delivers more creative responses, while lower temperatures tend to be more repetitive.

Multiple valid answers
:   Many questions have more than one correct way to explain something. The model may choose different phrasing or emphasis each time.

Context sensitivity
:   Tiny changes in punctuation or prior conversation can shift the output.

System-level factors
:   Hardware concurrency, floating-point math, and backend updates can introduce slight variations, even when everything else is fixed.

For example, think of it like rolling dice to pick words. When you ask a question, the model doesn't follow a fixed script. Instead, it looks at many possible next words and picks one based on probabilities---like rolling weighted dice.
The dice are weighted toward the most likely words, but there's still a chance for variation. If you roll again (ask the same question), you might get a slightly different sequence, even though the rules didn't change. This randomness
is intentional. It makes the model flexible and creative, rather than rigid and repetitive.

For more information about supported LLM models, see [Large language models on the ServiceNow AI Platform](https://www.servicenow.com/docs/ubAM4QNCHtjhTpXnMUGGYQ "ServiceNow AI Platform utilizes large language models (LLM) for generative AI and agentic AI functionality. You can choose the ServiceNow LLM or supported third-party models. LLMs bring about the Generative AI application capabilities by understanding and generating human language, and processing vast amount of data.").

## Variations in search {#sn-ai-understanding-llms__section_yvz_nvj_jhc}

The ServiceNow AI Platform® offers a variety of search tools, which may return different answers for the same or similar searches. This disparity in results is expected. For more information, see the following topics:

* [Discrepancies when using different AI search tools](https://www.servicenow.com/docs/Mus7fM9oIU0u91xhKpAuYg "Different AI search tools may return different answers for the same or similar searches. This difference in results is expected. It occurs because each large language model (LLM) uses a different approach to find results and generate answers that match your search.")
* [Search result disparities between AI Search and ServiceNow Otto® search features](https://www.servicenow.com/docs/access?context=search-disparities-ai-search-now-assist&version=zurich&pubname=zurich-platform-administration&ft:locale=en-US)
{#sn-ai-understanding-llms__ul_rjd_dck_jhc}