‘Not worth pursuing’: Nadella says AI must help humanity and remain under human control

Microsoft CEO Satya Nadella has backed calls for a more measured pace of artificial intelligence development, days after Anthropic chief Dario Amodei urged companies and governments to slow down the race to increasingly powerful AI models.

However, Nadella’s position is less about putting the brakes on AI development and more about ensuring that safety, alignment, and human oversight keep pace with increasingly capable systems.

In a post on X, Nadella said the pursuit of superintelligence should be guided by a basic principle: “if the AI we build is not helping humanity and under human control, it’s not worth pursuing.”

He also endorsed the need for “deliberate pacing needed to get alignment right” and backed proposals for embedded evaluators that could continuously monitor AI systems and translate safety principles into mechanisms that companies can enforce.

Nadella’s focus on who controls AI

For Nadella, the question of AI safety extends beyond how quickly models are developed. It also concerns who controls the technology. “The key is that this cannot be controlled by a handful of entities,” Nadella said, arguing for a broader AI ecosystem involving countries, industries and academia.

He also called for both closed and open-source AI models to have room to develop. The objective, he said, should be to prevent businesses and users from becoming permanently dependent on a small number of AI providers.

The comments come amid growing debate among AI companies and researchers over how to balance rapid development with the risks posed by increasingly powerful models.

Why Nadella is concerned about AI dependence

Nadella has previously warned that businesses could effectively end up paying for AI twice.

Companies pay providers to access AI tools, but they can also give away valuable internal knowledge through their interactions with those systems. Prompts, corrections, workflows and feedback can reveal how an organisation operates and, over time, become a valuable competitive asset.

His proposed answer is greater control over the AI systems and knowledge businesses use.

Nadella has argued that organisations should own their own “learning loop”—the process through which their data, workflows and feedback improve AI systems. He has also advocated giving companies greater control over the model weights they use and allowing them to embed their proprietary knowledge into AI models.

The approach is aimed at allowing businesses to benefit from increasingly capable AI without becoming locked into a single technology provider.

Amodei calls for a pause in AI development

Nadella’s comments follow a recent call from Anthropic CEO Dario Amodei for a slowdown in the development of advanced AI models.

In an essay published recently, Amodei said the development of AI itself was not in question but warned that the risks associated with increasingly capable models were “serious”. He argued that companies and governments need time to understand and address those risks as AI capabilities advance.

Amodei’s position has received support from several prominent technology figures, including Tesla CEO Elon Musk and OpenAI CEO Sam Altman.

The debate reflects a growing divide within the AI industry—not necessarily over whether development should continue, but over how quickly it should proceed and what safeguards need to be in place before more powerful systems are deployed.

Nadella’s latest intervention adds another dimension to that debate: AI development may need deliberate pacing, but control over the technology should also remain distributed rather than concentrated among a handful of companies.