'The Executor Behind the Screen' — When Handling Browser AI as an Authoritative Entity

On July 17, 2025, OpenAI stated in a ChatGPT agent system card that when these agents, which operate remote virtual browsers, run while logged into email or bank accounts, they enter "Watch Mode" and automatically stop when the user leaves the screen (OpenAI). In December of the same year, Google explained that regarding Chrome's agent capabilities, they have implemented an inspector model—separate from the planning model—that does not look at web content at all, allowing it to refuse individual actions one by one (Google). In March 2026, a paper from a public red-teaming competition targeting 13 state-of-the-art models was released, reporting that successful hijacking attacks were found in all models (arXiv).

In this past year, AI that operates browsers has moved from research previews to general availability. Anthropic expanded Claude for Chrome to beta in November 2025 (Anthropic, Bunrin Works Short Report), and Google began a preview of agent capabilities after integrating Gemini into Chrome (Bunrin Works Short Report). In the era of conversational AI, the focus of safety discussions was "whether it says something strange." For AI that reads pages, clicks, fills out forms, and even completes purchases or sends messages, what needs to be protected has changed. This is because attackers do not need to talk to the user; if they plant text somewhere on the page the AI is reading, they can control its hands.

What I want to examine in this feature is the following hypothesis: The safety of browser AI is not determined solely by whether a single model is resistant to prompt injection, but requires a design that separates the planning system, which reads external content, from the execution authority, restricts the sites it can read, the sites it can write to, and the operations it can perform on a task-by-task basis, and hands over critical operations to a separate inspection system or human confirmation. We will review the records of public competitions, design documents from three companies, and the records of attacks observed by Google on the actual web, using the same criteria.

From Conversational Safety to Behavioral Safety

Anthropic explains that browser usage increases danger in two directions. The broad attack surface—meaning that pages, embedded documents, advertisements, and dynamically loaded scripts all become entry points for instructions. And the large number of actions an agent can take—meaning that navigating to a URL, filling in a form, clicking, or downloading can all become tools for an attacker if hijacked (Anthropic). The company writes that "every web page an agent visits can become an attack vector."

Google expresses the same threat using the language of Chrome's security model. For years, browsers have used site-specific isolation and the same-origin policy to prevent one site from accessing data from another. However, an agent's job is to move across sites, such as gathering recipes on one site and filling a cart on another. If a hijacked agent can operate any site, it effectively becomes a bypass of site isolation (Google). If this occurs in a local browser that is logged into sites, the damage leads directly to data leaks.

In short, the point of contention has shifted from "whether the AI gives a strange response" to "under whose command the AI uses its authority." I interpret this change as the emergence of a need to treat browser AI as an authoritative entity acting as a proxy for a single user. If it were a human employee, they would not have access rights to systems outside their scope, and there would be an approver for fund transfers. The question is whether the same design is required for AI, or if making models smarter will suffice.

What the Public Competition Measured

There are records that measure the resilience of individual models from the outside, rather than relying on the self-declarations of the parties involved. This was a public red-teaming competition organized by Gray Swan, with the UK AI Safety Institute, the US CAISI, and several development companies participating in the analysis. The paper was submitted to arXiv on March 16, 2026, and CAISI of NIST released an explanation on the 23rd of the same month (NIST).

The authors of the paper state that the correlation between model capability and robustness is weak, and that Gemini 2.5 Pro showed both high capability and high vulnerability simultaneously (arXiv). CAISI's explanation adds that attacks found against robust models tend to transfer to non-robust models, but the reverse does not hold. Another point emphasized by this competition is "concealment." Since users often only see the agent's final response, attacks that execute harmful operations without leaving any trace in the response can succeed.

I will note a caution when reading these numbers. The success rate is a value relative to the population of the competition—that is, participants refining attacks for rewards—and the 41 scenarios prepared by the organizers; it is not the probability of a user suffering damage on the general web. Furthermore, even the lowest value of 0.5% merely means it is the best among 13 models, and the fact that it is not zero carries heavy significance for design. The authors mention the issue of benchmarks becoming saturated and outdated and state they will continue quarterly competition updates.

From these records, I derive two things. First, improvements in capability do not guarantee robustness. Second, as long as success cases remain even with the best models, model resilience is a layer that reduces the success rate of attacks, not a layer that determines what can be done after a hijacking. Next, we will look at how the three companies are designing for the "after hijacking" state.

Comparing the Designs of the Three Companies on a Single Scale

The explanatory documents from all three companies are primary source materials from the parties involved. Here, we align their statements regarding what they have built using the same perspectives. The perspectives are fourfold: the model's inherent robustness, the presence of an inspector that does not read external content, how the scope of authority is restricted, and the conditions under which critical operations are handed over to humans.

Anthropic's explanation places the most emphasis on model robustness. They use reinforcement learning to train models to identify and refuse instructions embedded in simulated web environments, employ classifiers to scan all untrusted content entering the model's context, and employ human red teams to continuously search for vulnerabilities. As a result, they state that they have reduced the success rate against internal adaptive attackers to approximately 1%, while simultaneously noting that "a 1% success rate, while a significant improvement, represents a meaningful risk" (Anthropic). Regarding authority, this is supplemented by their April 2026 document. They allow users to choose "Always allow," "Require approval," or "Block" for each tool, implemented "Plan Mode" in Claude Code—where the entire plan is approved upfront rather than individual operations—and advise that "customers should carefully consider which tools and data to provide, which permissions to grant, and in which environment to run." (Anthropic). The company states that safeguards are required across all four layers—model, harness, tools, and environment—and that even a well-trained model can be exploited through a poorly configured harness or tools with broad permissions. However, there is no mention in either document of placing an inspector that does not read external content within the product.

Google's explanation, conversely, focuses on structure. For planning models, they use a technique called "spotlighting" to prioritize instructions from the user and the system; while they aim to incorporate known attacks into training, they state that this alone is insufficient and have implemented a "User Alignment Critic." This is another model built with Gemini that examines each proposed action after a plan is finalized, refusing any actions that do not align with the user's intent. Because this model only receives summaries of proposals and does not see the raw content of the web, it cannot be poisoned by the page. If refusals accumulate, the planning model returns control to the user. Regarding authority, they utilize "Agent Origin Sets." For each session, they separate the set of origins capable of reading content from the set of origins capable of clicking or inputting, and the determination of which origin belongs to which set is made by a judgment function that does not touch web content. Irrelevant iframe origins are not passed to the model, and URLs generated by the model undergo a deterministic inspection limited to known public URLs. Human verification consists of three stages: stopping navigation to sensitive sites like banks or medical services through a list comparison, verifying logins from password managers, and pausing before purchases, payments, or sending messages to leave the final step to the user (Google). The company revealed that they will start with a simplified version of the origin sets in the initial implementation and has established a bounty program offering up to $20,000 to external researchers.

OpenAI's design appears to be somewhere in the middle of the two. The ChatGPT agent combines model training with updateable monitoring/filters, confirmation before operations that change state, and "Watch Mode" in sensitive contexts. At the time of release, they disabled memory functions and limited communication from the terminal to images and specific GET requests for datasets. The effectiveness of their training is published in the following table, though the system card itself notes that it "only measures model behavior and does not measure the entirety of the countermeasures" (OpenAI).

Comparing the three companies, it is clear they are arriving at the same conclusion through different paths. While all three continue the work of increasing model robustness, none of them state that "this is enough." Anthropic and OpenAI divert the remaining gap toward user approval and narrowing permissions, while Google goes a step further by integrating a structure that includes an inspector that does not read web content into the product. In my view, there is no evidence in the parties' own documents to support the theory that improving capability can serve as a substitute for the separation of authority.

On the other hand, there is something invisible in this comparison. The figures provided by the three companies use different measurement conditions; Anthropic and OpenAI provide defense rates against internal attackers, while Google does not publish its rates. Anthropic itself admits in its April 2026 document that there is no standard method for comparing agent robustness and that the self-inspections of each company have not been independently verified (Anthropic). Until external measurements, such as public competitions, accumulate, it is reasonable to accept these as the parties' own explanations of how their designs function.

Vulnerability in Experiments vs. Real-World Damage are Different Quantities

Up to this point, we have looked at experimental records where attacks could succeed. But to what extent are attackers active on the real web? A study published by Google's Threat Intelligence division in April 2026 is one of the few resources that measures this separately. The company scanned Common Crawl's public web archives to search for known types of prompt injection (Google).

Many of what they found were pranks to change the AI's tone, instructions attempting to add a company's context to AI summaries, text for SEO purposes to get a company recommended, or obfuscation to prevent AI from reading certain content. Examples targeting data theft were few, and even then, most appeared to be experiments by site operators. The company assessed that "attackers do not yet seem to have put these research results into large-scale practice." However, the same study also notes that the detection of malicious classifications increased relatively by 32% between November 2025 and February 2026, and that the scan did not include social media platforms that require logging in. The company views that if the cost-effectiveness for the attacking side changes, the scale and sophistication will grow.

I read this material as evidence unfavorable to my hypothesis. Just because a system is vulnerable in an experiment does not mean that damage is currently occurring frequently. To speak of the 8,648 successful attempts in the competition and experimental pranks on the web as being of the same magnitude would give readers a false sense of urgency. Nevertheless, the reason to rush design lies not in the frequency of damage, but in the nature of the damage. A failure in a conversational AI remains within the response, but a failure in a browser AI remains as sent emails, completed purchases, or changed settings—and as the competition records show, there may be no traces left in the final response. The combination of low frequency and irreversible consequences is why the design of authority cannot be postponed.

The Boundaries Readers Should Observe

Based on the above, I will provide my verdict. The hypothesis is supported by the evidence. Public competitions have shown that even if the inherent resistance of a model reduces the attack success rate, it does not drop to zero, demonstrating that capability and robustness are different quantities. The design documents of all three companies supplement resistance with authority design, and there are no descriptions supporting the theory that capability improvements can serve as a substitute. The fact that real-world damage is still limited is not a reason to rush design; rather, given that the nature of failure has changed, it is a reason to draw the boundaries first.

The boundaries that implementers should observe are exactly the same as the perspective of this feature. Is there a separation between sites an agent can read and sites an agent can write to? Is there an inspector that does not read web content, or instead, for which operations is a human called? Does it always stop before irreversible operations such as purchases or transmissions? And who, other than the parties involved, is measuring that promise? As of now, there is no product that can answer the fourth question.

I will also state the conditions under which this assessment might be wrong. If updates in public competitions show that the success rate of state-of-the-art models remains close to zero even against adaptive attacks, the weight of model resistance will increase. Conversely, if widespread, practicalized theft attacks are found in Google's next planned study including social media, my reading regarding the frequency of damage will require revision. What I will look for next is the quarterly update from Gray Swan, whether Chrome's origin set moves from a simplified version to a full two-set model, and whether frameworks to independently measure the self-reported figures of each company emerge from organizations like CAISI. Now that AI is the one moving the hands on the other side of the screen, what those hands are allowed to hold will be determined not by the intelligence of the model, but by our design.