Anthropic details what stops Claude Duet from going rogue

Anthropic has outlined the security and privacy protections behind Claude Duet, its proactive AI agent that can monitor connected accounts and complete tasks while users are away. The safeguards include approval requirements, checks for sensitive information and infrastructure that keeps credentials outside the model’s reach.

Duet works through a cloud-based computer and browser to prepare briefings, draft messages and handle business tasks. The standalone app is available through a research preview waitlist for Pro and Max subscribers in the US and shares usage limits with the rest of Claude.

User approvals and action limits

According to Anthropic, Duet asks users to approve actions such as sending potentially sensitive messages, making payments or deleting information. The request shows the proposed action, destination and content, including recipients.

A separate automated check assesses whether each action matches the user’s intent. Anthropic says this check operates outside the model performing the task to protect it from instructions hidden in web pages.

Approval flow: the approval card and the separate check sit outside the model, so nothing the agent reads can answer the card. (Source: Anthropic)

Accessing someone else’s account, trading securities and moving money between bank accounts remain prohibited even with approval. Certain actions, including payments and account creation, require approval each time. Background routines follow the same rules.

“Approval fatigue is a serious issue, because too many requests teach people to tap through them, sometimes without adequate consideration. We will continue to adjust this calibration over time,” the company said.

Information sharing and data retention

Anthropic uses a separate classifier designed to detect sensitive information in outgoing messages. The checks also cover browser uploads.

Duet is instructed never to share credentials, full payment card and bank account numbers, or government identification numbers, even when asked. Other private information, including health and financial details, requires explicit consent. A user’s instruction to send that information can count as consent.

When a message is blocked, Duet can remove the sensitive content and submit a revised version for approval. Anthropic acknowledges that the checks may miss sensitive information or flag harmless messages during the research preview.

Users can review Duet’s progress through a task list and give instructions while work is underway. Apps connected to Claude carry over to Duet. Disconnecting an app stops new access in both services.

Duet uses conversation history and memory to carry context between tasks until users delete the history or reset the memory. Disconnecting an app does not erase information already retained in memory. Deleting an account removes all stored information.

Credential protection and browser security

Anthropic says Duet cannot access users’ passwords, browser cookies or sign-in tokens.

Users enter passwords through a form inside the Claude app. The form encrypts the password and sends it directly to the cloud browser, which enters it on the website. Planned 1Password support will allow users to approve credential requests without passing login details through the model.

For connected apps, a separate service adds sign-in tokens when requests are made, keeping those credentials outside the agent’s workspace.

The company warns that attackers could imitate password forms. Legitimate requests appear only during a sign-in initiated by the user, inside the Claude app or on claude.ai. Password requests claiming to come from Duet through email, text messages or websites other than claude.ai are fraudulent, the company says.

Duet’s browser runs in a dedicated cloud virtual machine. The agent controls it through commands to open pages, click and type. Website code runs inside the browser environment, and the agent cannot read browser cookies.

Page text and downloaded content can reach the agent. Anthropic uses prompt injection defenses to address malicious instructions embedded in that material.

Security evaluations and bug reporting

Anthropic says its evaluations test routine tasks, tasks containing injected instructions and scenarios that encourage the agent to exceed its authorization.

Internal red teams tested whether Duet could leak information, act without approval or follow malicious prompts. An external penetration-testing firm examined the service with access to its source code. Further testing of the cloud browser and sign-in form is underway.

Duet is also covered by the company’s public bug bounty program, which accepts vulnerability reports through HackerOne.