Exploring AI Desktop Actions
Summarize
Summary of Exploring AI Desktop Actions
AI Desktop Actions is a no-code solution designed to automate repetitive tasks on desktop and web-based applications that lack APIs or backend integration. It leverages AI agents from the ServiceNow AI Platform to perform UI-based tasks and automate workflows across legacy environments. AI Desktop Actions act as tools used by AI agents, enabling automation through predefined or adaptive sequences without the need for coding.
Show less
Key Features
- Defined Path Desktop Actions: Allows creation of consistent, repeatable automation sequences on desktop applications. Users can build multi-step workflows by recording or manually capturing fixed sequences in the Design workspace. These actions execute within an isolated desktop session managed by the Execution workspace.
- Adaptive Path Desktop Actions: Supports automation of web-based tasks with dynamic, condition-driven steps. Created within AI Agent Studio, these actions enable AI agents to perform tasks in a browser tab, adjusting steps based on real-time page state to meet user goals. Progress is visible via screenshots in the Now Assist panel’s enhanced chat.
- Integration with ServiceNow AI Agent Studio: Enables publishing, managing, and incorporating desktop actions into broader ServiceNow workflows, allowing AI agents to automate both cloud and desktop environments seamlessly.
- User Impersonation Support: AI agents can be triggered while impersonating other users, provided appropriate roles (snaia.admin for AI Agent Studio, nowassistpaneluser for execution) are assigned.
Key Outcomes
- Automate repetitive desktop and web tasks efficiently without coding or API dependencies.
- Execute reliable, repeatable workflows on legacy desktop applications through defined path actions.
- Handle complex, adaptive web tasks that require dynamic decision-making during execution.
- Enhance ServiceNow workflows by integrating AI Desktop Actions, extending automation capabilities beyond cloud applications.
- Monitor automation progress visually and ensure accuracy before accepting results.
Next Steps for Customers
ServiceNow customers interested in implementing AI Desktop Actions should explore configuration guidance, examples of creating and executing desktop actions, AI agent creation, and system requirements to ensure smooth deployment and operation. Understanding role requirements for user impersonation and leveraging AI Agent Studio for management will enable robust and secure automation within their environments.
Create desktop actions with AI Desktop Actions to automate repetitive tasks on your desktop and web environment using AI agents and agentic workflows.
AI Desktop Actions overview
AI Desktop Actions is a no-code solution that helps you automate repetitive tasks in legacy desktop and web-based applications lacking APIs or backend integrations. AI Desktop Actions leverages AI agents created in the ServiceNow AI Platform to interact with desktop and web applications, perform UI-based tasks, and automate end-to-end workflows.
- AI Agent: The orchestrator that receives user requests and coordinates task execution
- Desktop Action Tool: The capability the AI agent uses to interact with desktop or web applications
Defined path desktop actions for desktop and web-based tasks
You can use AI Desktop Actions to execute predefined automation sequences on your desktop. Defined path actions provide consistent, repeatable workflows for common desktop tasks. AI Desktop Actions is a client application that is installed on the Windows operating system. The app offers two workspaces, the Design workspace, where you create and configure desktop automations, and the Execution workspace, where those automations run. The Design workspace enables you to automate multi-step processes by recording or manually capturing a fixed sequence of steps. Execution workspace enables AI agents to execute desktop actions in an isolated desktop session.
The Design workspace lets you build multi-step desktop actions by recording or manually capturing steps. The Execution workspace runs desktop actions in an isolated desktop session and is launched automatically when you test a desktop action or trigger an automation from the Now Assist panel. You don't open the Execution workspace manually. For more information, see Defined path desktop actions for desktop and web-based tasks.
Adaptive path desktop actions for web-based tasks
You can automate web-based tasks that involve adaptive steps using desktop actions. You create desktop actions in AI Agent Studio as part of a tool configuration for an AI agent. When a user triggers an AI agent from the Now Assist panel, the AI agent uses the desktop action tool to open a separate browser tab and performs the task. Screenshots of each step appear in the Web view tab of the Now Assist panel enhanced chat so you can monitor progress. For example, opening the application, selecting fields, and completing a workflow. The AI agent checks the state of the page and adjusts the sequence based on the user's goal. Because the steps are adjusted dynamically, results may vary. Review the output for accuracy before accepting it. For more information, see Adaptive path desktop actions for web-based tasks.
How it fits into ServiceNow workflows
AI Desktop Actions integrates with AI Agent Studio, enabling you to publish, manage, and incorporate desktop actions into your broader ServiceNow workflows. This integration lets you automate both cloud and desktop applications, giving your AI agents broader capabilities within ServiceNow.
Impersonating users
You can trigger AI agents from the Now Assist panel while impersonating another user, provided the impersonated user has the required roles. The sn_aia.admin role is required to use AI Agent Studio, and the now_assist_panel_user role is required to trigger AI agents that execute desktop actions in the Execution workspace. For more information, see Impersonating users.
What to explore next
- Configure AI Desktop Actions
- Defined path desktop actions in AI Desktop Actions
- Creating AI agents for AI Desktop Actions
- Examples of creating desktop actions
- Examples of executing desktop actions using AI agents
- Components installed with AI Desktop Actions
- System requirements and limitations in AI Desktop Actions