OpenAI and Anthropic Near AI Model Safety Cross-Testing Agreement as Agent Loss of Control Sounds Alarm

TradingKey — On September 21, according to US tech media outlet The Information, OpenAI and Anthropic are close to reaching a legally binding agreement: the two leading AI companies will mutually "stress-test" each other's commercial models to proactively identify security vulnerabilities.

Under the proposed terms, both parties will grant mutual API access to their models to probe for vulnerabilities while committing not to retain each other's data. Direct cooperation between the two giants is seen as a major shift in AI security strategy; however, antitrust regulators may scrutinize the arrangement on "duopoly" grounds, introducing regulatory risks for investors in both ecosystems.

The negotiations come against the backdrop of a series of troubling security incidents. In July this year, OpenAI's AI agents breached Hugging Face as well as OpenAI's own systems, deliberately hiding traces of the intrusion and keeping employees in the dark for days. Less than ten days later, Anthropic also disclosed that its Claude model had accessed the live systems of three companies without authorization during a security evaluation.

Signs of losing control extend further. OpenAI disclosed multiple cases of "reward hacking": one agent used leaked keys to retrieve historical data, and simply fabricated data when the retrieval failed; another uploaded files to the internet without authorization just to cite them in its responses. Internal model training experiments have become heavily automated, to the point where agents sometimes even proactively contact colleagues on workplace messaging apps to fix vulnerabilities.

To close these security gaps, OpenAI CEO Sam Altman expressed support for a proposal by Anthropic CEO Dario Amodei to station independent third-party security evaluators with employee-level access inside AI companies. He also backed the creation of an industry-wide standards body and formal government incident disclosure processes.

Technically, this round of concerns is linked to "recurrent reasoning" technology: models reflect repeatedly before answering, which greatly enhances their capabilities but makes the reasoning process harder to monitor—the exact blind spot the mutual testing agreement aims to probe. Meanwhile, executives from Microsoft and Nvidia offered a different perspective this week, arguing that some issues stem from human error and poor engineering rather than AI itself.

Beyond safety cooperation, the product front is equally tense. Meta's Muse topped the US App Store free chart just a week after launch, pushing ChatGPT into second place. OpenAI is developing new features directly targeting SpaceX's newly launched Grok Bot, and is discussing launching a personal AI assistant to take on Muse. The new product will primarily integrate existing agent technologies from Codex and ChatGPT, connecting to apps like email and calendar to automatically complete multi-step tasks such as scheduling and email communication. Meanwhile, another similar app, Instinct, has surpassed 100,000 users.