Anthropic's top safety researcher publicly announced his resignation

A lead safety researcher from Anthropic, Mrinank Sharma, announced his resignation from the company this week in a post on X. This decision by Sharma seems to be driven by his concerns surrounding the current state of AI and the world

Mrinank Sharma led the Safeguards Research Team at Anthropic, a prominent AI company whose large language model (LLM), Claude, is widely regarded as a top competitor to OpenAI’s ChatGPT. Sharma’s departure was rather abrupt, as the Safeguards Research Team was only officially launched in February of last year. The team’s primary focus was to identify, understand, and help mitigate the risks associated with Anthropic’s deployed AI systems, like Claude.

This sudden departure of a top safety researcher at one of the largest U.S. AI companies has caused a great deal of controversy on social media. Perhaps the most notable part of the resignation letter was when Sharma cryptically warned that “the world is in peril.” He attributed this “not just to AI, or bioweapons,” but to “a whole series of interconnected crises unfolding in this very moment.” This was interpreted by many people as a warning about the existential risks that come with AI advancements. Sharma’s resignation is part of a larger, concerning, and accelerating trend of resignations by high-profile employees at AI companies recently.

Interpreting Sharma’s resignation letter

Mrinank Sharma began the letter by briefly addressing his background and what inspires him, most notably “a willingness to make difficult decisions and stand for what is good.” He also spoke on his contributions to Anthropic, including developing and deploying defenses “to reduce risks from AI assisted bioterrorism,” and writing one of the first AI safety cases. His final project was “understanding how AI assistants could make us less human or distort our humanity.”

However, the part of his letter that caused the most concern was the third paragraph. While he did not directly accuse Anthropic of any wrongdoing or blatantly say AI is going to kill us all, he did use a lot of philosophical language to explain his resignation. He stated that “we appear to be reaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, less we face the consequences.” This was followed up by him writing, “I’ve repeatedly seen how hard it is to truly let our values govern our actions.” He also described the world being in peril from a series of interconnected crises, which he described in a footnote as a “poly-crisis” underpinned by a “meta-crisis.”

This language alludes that his departure from Anthropic was triggered by more of a philosophical divergence as opposed to any type of internal dispute at the company. By describing the current moment as a “poly-crisis” underpinned by a “meta-crisis” Sharma seems to be pointing to a much larger structural problem facing society and AI development by extension. Technology is advancing faster than collective wisdom, and the current systems and powers that manage and influence its development are not properly equipped to do so in the current state of the world.

The larger takeaway from Sharma’s letter

The larger takeaway from Sharma’s resignation letter is multifaceted and existential. On one hand, he seems to believe there is a fundamental problem with how technology companies are navigating the acceleration of AI development inside a competitive system. Global powers are in an arms race to surpass each other in AI and other technological advancements, with global tech spending set to hit $5.6 trillion in 2026. This means that AI companies are not just innovating and building products, but are a crucial component of geopolitical conflict. Additionally, these companies have a fiduciary responsibility to perform well for shareholders, creating an incentive to outperform their rivals in technological advancement.

This fosters an environment where safety principles and procedures must also align with market pressures, national competitiveness, and the expectations of investors. Still, as AI companies rapidly expand and advance their capabilities, they need to identify, understand, and mitigate the risks that come with them. The problem Sharma appears to be addressing is that the current system in which AI companies operate naturally prioritizes growth over safety and ethical considerations. The implications of this dynamic are existentially profound and a great cause for concern. A man like Sharma, who appears to be of good integrity, simply could not continue to operate within this system without compromising on his values, leading him to withdraw from it entirely.

Earn 8% CASHBACK in USDC when you pay with COCA. Order your FREE card.