Now LLM Service updates
Summarize
Summary of Now LLM Service updates
The Now LLM Service, updated on August 1, 2024, provides access to specialized large language models (LLMs) developed by ServiceNow, as well as selected open-source models. These models are designed to perform various language-related tasks, enhancing service delivery across the platform.
Show less
Key Features
- Model Cards: Details on each model's context, intended use, and limitations. This includes models for conversational use cases, code generation, flow recommendations, and high-throughput tasks.
- Multilingual Support: The service now supports eight additional languages, facilitating global usage in native languages.
- JSON Format Support: Outputs in JSON format enhance integration capabilities and workflow automation, while also improving response predictability and reliability.
- Deterministic Responses: Structured JSON outputs reduce format errors and stray characters, minimizing integration issues.
- Lower Token Consumption: The structured response format can lead to reduced token usage, making it cost-effective for frequent applications.
- Improved Instruction Following: The model has been fine-tuned for precise instruction understanding, resulting in quicker and more actionable responses.
Key Outcomes
These updates enhance the performance and quality of the Now LLM Service, allowing ServiceNow customers to benefit from improved accuracy, efficiency, and accessibility in their language-related tasks, ultimately leading to better service delivery and operational effectiveness.
The Now LLM Service provides access to specialized large language models (LLMs) that are developed by ServiceNow. It also provides access to open-source LLMs that are selected, configured, or enhanced by ServiceNow, from the ServiceNow community and partners. Review these reference materials and model cards for additional information about the Now LLM Service and about the models used.
Model cards
Large language models (LLMs) are complex machine-learning models that are trained on large datasets like websites and documentation to perform language-related tasks, such as text generation for case summaries and resolution notes.
Model cards explain the specific model's context, intended use, training data, limitations, and other important information.
These model cards are for skills that use the Now LLM Service. There are certain skills, such as Now Assist Multi-Turn Catalog Ordering, that use Azure OpenAI instead. To see what LLM a skill is using, you can check the skill list in the Now Assist Admin console and review the LLM service column.
- Model card for ServiceNow text-to-text LLM
-
Model used for conversational use cases like Virtual Agent topic execution and conversational catalog and agent assist use cases like alert analysis, AI search, and incident, case, and chat summarization.
- Model card for ServiceNow text-to-code LLM
- Model used for code generation.
- Model card for ServiceNow flow next-best-action LLM
- Model used for flow recommendations.
- Model card for ServiceNow text-to-flow LLM
- Model used for flow generation.
- Model card for ServiceNow text-to-text SLM
- Model used for Now Assist Guardian, text-to-cypher and other use cases that demand rapid inference and high throughput.
- https://downloads.docs.servicenow.com/resource/enus/infocard/third-party-llm.pdf
- Model used for AI-driven solutions for text generation, summarization, and conversational AI.
November 2024
Several key improvements were added to the Now LLM Service that are aimed at enhancing performance and quality.
-
Multilingual support: Now LLM Service supports 8 additional languages, enabling global teams to use the model in their native languages.
The supported languages are: English, German, French, Japanese, Dutch, French Canadian, Spanish, Brazilian Portuguese, and Italian.
- JSON format support: The model now provides output in JSON format, making it easier for developers to integrate with various applications and automate workflows seamlessly.
- Deterministic responses: JSON mode ensures structured, consistent output, which improves predictability and reliability when integrating with applications.
- Error reduction: Unlike free-form text mode, JSON responses are less prone to format errors or stray characters, minimizing integration issues.
- Lower token consumption: The fixed structure of JSON can reduce token usage, making it more efficient and cost-effective for applications with high response frequency.
- Improvements in instruction following: The model has been fine-tuned to understand and follow instructions more precisely. This enables the model to deliver more to-the-point and actionable responses, helping users get the information they need faster and more efficiently.