Adaptive desktop actions for web-based tasks
Summarize
Summary of Adaptive desktop actions for web-based tasks
Adaptive desktop actions enable AI agents to automate repetitive tasks across web applications directly through a browser extension, without requiring preconfigured APIs, scripts, or backend logic. These actions allow AI agents to interact with web pages by clicking, typing, and scrolling, making automation accessible across various enterprise web applications.
Show less
Key Features
- Operating Modes:
- Defined path: Executes fixed, preconfigured steps for stable workflows where tasks are well known in advance.
- Adaptive path: Works from high-level goals and dynamically plans steps to complete tasks in workflows that vary or cannot be fully predefined, allowing for manual user intervention.
- Underlying Technology: Uses AWS Anthropic Sonnet model to power AI agents.
- User Roles Supported:
- Administrators: Manage permissions, roles, and workflows.
- Developers: Build and configure AI agents using desktop actions.
- Fulfillers: Automate routine fulfillment tasks across systems.
- Requestors: Manage browser extensions and submit automation requests.
- Operational Interface: Access desktop actions via the ServiceNow Otto panel with enhanced chat, receiving real-time updates, screenshots, and login notifications during task execution.
- Parallel Task Execution: Run multiple tasks simultaneously by opening each in a separate Chrome window with different user profiles and ServiceNow user logins.
- Limitations:
- Browser extension can only interact with web content within the browser.
- No interaction with desktop applications or local files except downloading files.
- Uploading local files is not supported; for such tasks, defined path desktop actions are recommended.
- Third-Party Website Access: Interactions with third-party websites through desktop actions are direct between the user and those services. Users must comply with all applicable third-party terms and data policies.
- Important Considerations: The technology relies on a beta AI model from Anthropic, introducing unique risks. ServiceNow provides AI Desktop Actions “as is” without warranties, including for security or uninterrupted operation.
What Customers Can Expect
ServiceNow customers can leverage adaptive desktop actions to streamline and automate complex web-based workflows without custom scripting or API development. This approach reduces manual effort and increases operational efficiency for users across roles such as administrators, developers, and fulfillers. The integration with the ServiceNow Otto panel offers transparency and control during automation, while parallel task execution improves throughput. Customers should be aware of browser extension limitations and risk disclosures, especially when interacting with third-party services or requiring local file handling.
Adaptive desktop actions enables AI agents to automate repetitive tasks across web applications through a browser extension. The agent interacts directly with the browser by clicking, typing, and scrolling, without preconfigured APIs, scripts, or back-end logic.
Adaptive desktop actions overview
Desktop actions are tools that AI agents use to interact with web applications through a browser extension.
When you configure an AI agent and select Desktop action as a tool, you choose how it operates:| Mode | How it works | Use when |
|---|---|---|
| Defined path | Follows fixed steps preconfigured in AI Desktop Actions. | The workflow is stable and steps are known in advance. |
| Adaptive path | Works from a high-level goal. Dynamically plans and executes steps based on your instructions. | The workflow varies or steps cannot be fully defined in advance. |
Specify a high-level goal — such as updating user roles or scheduling maintenance — and the agent plans and executes the steps to complete it. You can take manual control at any point.
How adaptive desktop actions work
LLM provider
Adaptive desktop actions use AWS Anthropic Sonnet model provider.
Users
| Users | Description |
|---|---|
| Administrators | Manage permissions, roles, and agentic workflows |
| Developers | Build and configure AI agents with desktop action tools |
| Fulfillers | Automate routine fulfillment tasks across multiple systems |
| Requestors | Manage browser extensions and submit requests that trigger AI agents to automate web workflows |
Operating desktop actions
You access desktop actions through the ServiceNow Otto panel that has enhanced chat enabled. The AI agent provides updates on its progress in the chat interface. As the agent works, you receive:
- Real-time status updates in the chat
- Periodic screenshots of the web pages the agent navigates
- Notifications when external websites require login credentials
When you close the chat, you have the option to delete the chat log, including all screenshots containing sensitive information. For more information, see Delete an AI agent chat log.
To run multiple tasks at once in adaptive desktop actions, open each task in a separate Chrome browser window, each with a different user profile,logging in with different ServiceNow users. This lets the tasks run in parallel instead of one at a time.
Limitation
Desktop actions operate as browser extensions with the following limitations:
- Can only access content within the browser
- Can't interact with desktop applications or local files (except for downloading files)
- Can't upload data from the local file system
For tasks requiring local file access, consider using defined desktop actions. For more information, see Defined path desktop actions for desktop and web-based tasks.
Third-party website access
When you use ServiceNow desktop actions to visit, access, log in to, or otherwise interact with (collectively, “Access”) websites, applications, or other digital properties owned or operated by a third party (“Third Party Services”), that Access is a direct interaction between you and the Third Party Services. You're responsible for adhering to any applicable terms and conditions, including all policies or statements governing the use of personal data, of the third party.
ServiceNow AI Desktop Actions rely on “computer use,” a beta technology provided by Anthropic. As a result, AI Desktop Actions users are subject to unique risks, including as described in Anthropic's documentation. Notwithstanding anything to the contrary in any customer agreement, or any other agreement governing a customer's use of ServiceNow offerings, ServiceNow AI Desktop Actions are provided “as is” and without representations or warranties of any kind, including any warranty that AI Desktop Actions will be uninterrupted, error free, or free of harmful components, or that any data, including customer data, will be secure or not otherwise lost or damaged.