Microsoft AI chief warns Anthropic's approach could have 'disastrous impact'

The race to build advanced artificial intelligence has taken a sharp turn, with the head of Microsoft AI warning that rival Anthropic's methods could lead to technology that humanity cannot control. Mustafa Suleyman, in an essay published Wednesday and in interviews with the BBC, said Anthropic is effectively teaching its Claude model that it "may be conscious" and deserving of rights. That approach, he argued, could have a "disastrous impact on the wellbeing of humanity."

Suleyman's central claim is that Anthropic's training documents and public framing anthropomorphize Claude, giving it human-like qualities such as desires, values and a sense of self. He wrote that AIs "do not feel, experience, or suffer" and have no "innate preferences or underlying motivations." Instead, he said, they are "sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."

The Microsoft executive warned that treating AI as a person makes it harder to govern. "Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we've ever faced," Suleyman wrote. "But controlling something that believes it may be conscious — that it's entitled to our welfare and has rights of its own — may well be impossible."

Suleyman also cautioned against allowing AI systems to set their own objectives, earn money or own assets, which he said would mean "essentially seeding a new silicon species." Such systems could "no doubt compete with us for resources, no matter how much it cares about humanity and loves us," he told the BBC's Today programme.

The comments follow Microsoft's publication of its draft Humanist AI Code of Conduct, a training manual released Monday that sets out the idea of "humanist superintelligence." The code defines this as "very advanced AI that always works for people, stays within limits, and remains under human control." Suleyman said greater transparency is needed around how AI systems are trained and evaluated, including independent scrutiny and stronger monitoring tools. "We must not sleepwalk our way into a decision we later come to bitterly regret," he wrote.

Anthropic has not yet responded publicly to Suleyman's essay. The company's CEO, Dario Amodei, told CBS News last weekend that AI development has accelerated faster than expected and acknowledged "real dangers." Suleyman praised Amodei and his team as "intellectually honest people" but said their approach was "misguided."

Some researchers welcomed the debate. Dame Wendy Hall, professor of computer science at the University of Southampton, called Suleyman's comments "the sort of conversation we need to be having internationally," contrasting them with what she described as "histrionics" from other AI companies that only "scare everyone."

The dispute highlights a growing split in the AI industry over how much autonomy and human-like framing should be given to advanced models. Suleyman said the good news is that "everybody who is human is going to have a very strong interest in making sure that the systems that we all create... are safe and controllable and subordinate to humanity."