Generative AI Controller release notes
Summarize
Summary of Generative AI Controller Release Notes
The ServiceNow® Generative AI Controller application enhances user experience by integrating generative AI capabilities with third-party large language models (LLMs) on the ServiceNow AI Platform. The latest updates in the Washington DC release, dated February 1, 2024, focus on reducing complexity and improving content generation, summarization, sentiment analysis, and prompt writing.
Show less
Key Features
- Integration with Custom LLMs: Utilize generative AI in Flow Designer and Virtual Agent Designer.
- Recursive Summarization: Processes large inputs by breaking them down, summarizing, and recombining for accurate LLM responses.
- External LLM Connectivity: Connect with Aleph Alpha and IBM watsonx models using your API credentials.
- Sensitive Content Detection: Semantic filters in Virtual Agent identify and manage discussions on sensitive topics.
- Dynamic Translation Control: Option to disable multi-language support for LLM conversations in Virtual Agent.
- Removal of QnA Capability: This feature has been eliminated from Virtual Agent Designer and Flow Designer.
Key Outcomes
By utilizing the updated Generative AI Controller, ServiceNow customers can enhance their AI-driven workflows, achieve more accurate LLM interactions, and ensure sensitive topics are handled appropriately. This enables improved user satisfaction and operational efficiency across various ServiceNow applications, including Now Assist, Agent Chat, Customer Service Management, and IT Service Management.
The ServiceNow® Generative AI Controller application enables you to use generative AI capabilities with third-party large language module (LLM) providers on the ServiceNow AI Platform to reduce complexity and enhance the user experience by generating and summarizing content, analyzing user sentiment, and writing unique prompts. Generative AI Controller was enhanced and updated in the Washington DC release.
Generative AI Controller highlights for the Washington DC release
- Integrate with custom LLMs in generative AI capabilities in Flow Designer and Virtual Agent Designer
- Add semantic filters in Virtual Agent to detect and handle sensitive topics
See Generative AI Controller for more information.
New in the Washington DC release
- Recursive summarization for large inputs
- Large language models (LLMs) have a maximum number of tokens per request. Recursive summarization breaks down large inputs into smaller pieces, summarizes those pieces, and then recombines them to make the original call. This process prevents important context from being lost when submitting requests to the LLM, which can result in more accurate responses with fewer hallucinations.
- Integrate with Aleph Alpha models
- Bring your own API credentials and connect Aleph Alpha's Luminous LLM with the ServiceNow AI Platform.
- Integrate with IBM watsonx models
- Use your own API credentials to connect to IBM watsonx's Granite LLMs with the ServiceNow AI Platform.
- Integrate with external LLMs
- Connect API credentials with a generic LLM connector and use an external LLM on the ServiceNow AI Platform.
- Semantic filter for sensitive content in generative AI conversations
- Use AI Semantic Search to help determine if sensitive content is being discussed, such as workplace misconduct or employee health concerns, and trigger a switch to a different Virtual Agent topic.
- Disable Dynamic Translation for LLM conversations in Virtual Agent
- Dynamic Translation is enabled by default for ServiceNow AI Platform feature that is available with activation of any Now Assist applications, but you can disable it if you prefer not to have multi-language support for LLM conversations.
Removed in this release
- The QnA capability has been removed and the action has been removed from Virtual Agent Designer and Flow Designer.
Activation information
Generative AI Controller is a ServiceNow AI Platform feature that is available with activation of any Now Assist application. For details, see Installing Generative AI Controller.