Exploring AI Desktop Actions
Summarize
Summary of Exploring AI Desktop Actions
AI Desktop Actions is a no-code automation solution within ServiceNow that enables customers to automate repetitive tasks on desktop and web applications, especially those lacking APIs or backend integrations. It uses AI agents from the ServiceNow AI Platform to interact with user interfaces and perform multi-step workflows across legacy desktop and web-based environments.
Show less
The solution consists of two main components:
- AI Agents: Orchestrators that receive user requests and manage task execution.
- Desktop Action Tools: Capabilities used by AI agents to interact with desktop or web applications through UI-based actions.
Key Features
- Defined Path Desktop Actions: Enable consistent, repeatable automation sequences on desktop applications. These actions are created and configured in the Design workspace by recording or manually capturing fixed steps, and executed in an isolated Execution workspace on Windows.
- Adaptive Path Desktop Actions: Support automation of dynamic, web-based tasks with variable steps. Created in AI Agent Studio, these actions run in a browser tab controlled by AI agents, dynamically adjusting steps based on webpage state. Progress can be monitored via screenshots in the Now Assist panel's enhanced chat.
- Integration with AI Agent Studio: Allows publishing and management of desktop actions within ServiceNow workflows, enabling automation across both cloud and desktop applications for broader AI agent capabilities.
- User Impersonation Support: AI agents can be triggered from the Now Assist panel while impersonating users who have the necessary roles, facilitating flexible automation scenarios.
Practical Benefits for ServiceNow Customers
- Automate repetitive tasks on legacy and web applications without requiring backend integration or APIs.
- Use no-code tools to build, test, and execute multi-step workflows efficiently on Windows desktops.
- Monitor and adjust web-based automation dynamically for adaptive task execution.
- Leverage AI agents within existing ServiceNow workflows to enhance automation reach and effectiveness.
- Enable role-based access and impersonation to maintain security and proper access control during automation.
Next Steps
ServiceNow customers interested in implementing AI Desktop Actions should explore configuration options, learn to create and execute both defined and adaptive path desktop actions, understand AI agent creation, and review system requirements and limitations to optimize deployment and usage.
Create desktop actions with AI Desktop Actions to automate repetitive tasks on your desktop and web environment using AI agents and agentic workflows.
AI Desktop Actions overview
AI Desktop Actions is a no-code solution that helps you automate repetitive tasks in legacy desktop and web-based applications lacking APIs or backend integrations. AI Desktop Actions leverages AI agents created in the ServiceNow AI Platform to interact with desktop and web applications, perform UI-based tasks, and automate end-to-end workflows.
- AI Agent: The orchestrator that receives user requests and coordinates task execution
- Desktop Action Tool: The capability the AI agent uses to interact with desktop or web applications
Defined path desktop actions for desktop and web-based tasks
You can use AI Desktop Actions to execute predefined automation sequences on your desktop. Defined path actions provide consistent, repeatable workflows for common desktop tasks. AI Desktop Actions is a client application that is installed on the Windows operating system. The app offers two workspaces, the Design workspace, where you create and configure desktop automations, and the Execution workspace, where those automations run. The Design workspace enables you to automate multi-step processes by recording or manually capturing a fixed sequence of steps. Execution workspace enables AI agents to execute desktop actions in an isolated desktop session.
The Design workspace lets you build multi-step desktop actions by recording or manually capturing steps. The Execution workspace runs desktop actions in an isolated desktop session and is launched automatically when you test a desktop action or trigger an automation from the Now Assist panel. You don't open the Execution workspace manually. For more information, see Defined path desktop actions for desktop and web-based tasks.
Adaptive path desktop actions for web-based tasks
You can automate web-based tasks that involve adaptive steps using desktop actions. You create desktop actions in AI Agent Studio as part of a tool configuration for an AI agent. When a user triggers an AI agent from the Now Assist panel, the AI agent uses the desktop action tool to open a separate browser tab and performs the task. Screenshots of each step appear in the Web view tab of the Now Assist panel enhanced chat so you can monitor progress. For example, opening the application, selecting fields, and completing a workflow. The AI agent checks the state of the page and adjusts the sequence based on the user's goal. Because the steps are adjusted dynamically, results may vary. Review the output for accuracy before accepting it. For more information, see Adaptive path desktop actions for web-based tasks.
How it fits into ServiceNow workflows
AI Desktop Actions integrates with AI Agent Studio, enabling you to publish, manage, and incorporate desktop actions into your broader ServiceNow workflows. This integration lets you automate both cloud and desktop applications, giving your AI agents broader capabilities within ServiceNow.
Impersonating users
You can trigger AI agents from the Now Assist panel while impersonating another user, provided the impersonated user has the required roles. The sn_aia.admin role is required to use AI Agent Studio, and the now_assist_panel_user role is required to trigger AI agents that execute desktop actions in the Execution workspace. For more information, see Impersonating users.
What to explore next
- Configure AI Desktop Actions
- Defined path desktop actions in AI Desktop Actions
- Creating AI agents for AI Desktop Actions
- Examples of creating desktop actions
- Examples of executing desktop actions using AI agents
- Components installed with AI Desktop Actions
- System requirements and limitations in AI Desktop Actions