Amazon Bedrock AgentCore Browser adds OS-level interaction capabilities

Amazon Bedrock AgentCore Browser has introduced new OS-level interaction capabilities, significantly enhancing its ability to automate browser workflows that require direct control over the operating system. This update extends beyond the capabilities offered by the Chrome DevTools Protocol (CDP), addressing scenarios where CDP falls short. Such scenarios include operations involving mouse movements, handling print dialogs, responding to native system alerts, and executing keyboard shortcuts.

These advanced features are particularly beneficial for AI agent developers, test automation engineers, and organizations focused on building web interaction tools powered by large language models (LLMs). With the new capabilities, users can automate a range of actions including mouse operations like clicking, moving, dragging, and scrolling, as well as keyboard operations such as typing, pressing keys, and using shortcuts like ctrl+a and ctrl+p. Additionally, users can capture full desktop screenshots, all based on OS-level coordinates that extend beyond the confines of the browser viewport.

The key applications of these features include automated testing that involves managing system dialogs, document management workflows, handling complex user interface interactions with right-click menus, and supporting vision-based AI agents that need comprehensive visibility of the browser environment.

This feature is now available by default across all browser instances in the 14 AWS Regions where the Amazon Bedrock AgentCore Browser is operational. These regions include US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Europe (Stockholm), Asia Pacific (Mumbai), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Asia Pacific (Seoul), and Canada (Central).

For more information, users are encouraged to refer to the AgentCore Browser documentation.