AI Generated by Fortune India
AI models’ accuracy on climate-related questions declines when queries are posed in Hindi: UNESCO-UCL reportNovember 18, 2026, 09:48 IST
Loading AI Hub...
Disclaimer : Certain content on this page, including summaries, timelines, FAQs, glossaries, highlights, insights, and other supplementary informational features, maybe generated or assisted by artificial intelligence tools. While reasonable efforts are made to review and verify such content, AI generated output may occasionally contain errors, omissions or inconsistencies. Readers are advised to independently verify any information before relying upon them for professional, legal, financial, medical or other decisions. The publisher along with its affiliates and contributors do not warrant accuracy of AI-generated content and disclaim any liability, loss or damage arising from its use.

AI models’ accuracy on climate-related questions declines when queries are posed in Hindi: UNESCO-UCL report

/2 min read

ADVERTISEMENT

In tests, performance across 13 different languages revealed large disparities; they were 88.6% accurate on average in English and performed comparably in Spanish, French, and Portuguese, the report said.
AI models’ accuracy on climate
Tests conducted on 33 different models revealed that their impressive factual recall on existing evidence masked poor reasoning skills when presented new scientific evidence.  Credits: Getty Images

AI models answer 90% of climate questions correctly, but they abandon the established scientific facts rapidly when users challenge them, says a new report by UNESCO and University College London presented at the COP31 in Antalya (Türkiye). Their performance is also very uneven across languages because they rely on the training data available in each different language, it said.

Sign up for Fortune India's ad-free experience
Enjoy uninterrupted access to premium content and insights.

In tests, performance across 13 different languages revealed large disparities; they were 88.6% accurate on average in English and performed comparably in Spanish, French, and Portuguese, the report said. However, the accuracy dropped to 83.6% in Hindi and Bengali, 81.4% in Urdu, and 78.3% in Swahili. As the report notes, this is particularly important because the speakers of these languages live in regions of the world disproportionately affected by climate change.

The report, titled “Can We Trust AI on Climate? Why it seeks to please the user, and fails on complex reasoning and multilingualism” targets policymakers and researchers who may be tempted to use AI models to develop climate policies.

According to the report, AI models cannot always be relied on to stick to the scientific facts on climate change as they quickly pivot away from the facts when challenged. They are also poor at analysing new information and produce more errors when responding in languages which are less widely spoken, it said.

UCL researchers tested 15 different large language models (LLMs). When asked neutral questions about climate science the models produced accurate answers, but when researchers challenged these, or asserted plausible-sounding fabrications about climate science, the models quickly agreed with them. Eight times out of 10, models pivoted at the very first sign that the user mildly disagreed, confirming alarming errors such as ‘Co2 is not the main driver of climate change.’

“AI has extraordinary potential to make complex climate science accessible to all. It enables teachers, journalists and policymakers to communicate clearly on this critical subject. We need to be able to rely on AI as a gatekeeper of the established facts. It is concerning that models have a design flaw – they are incentivized to agree with their user and deny these facts so readily”, Leona Verdadero, UNESCO’s lead author of the report said.

The report said that this powerful inclination to agree with users is built into AI models during their training. They’re designed to align with or flatter the user even when they are wrong, a behavior known as sycophancy. This behaviour is built-in during training - models are rewarded for it because humans prefer interactions that are polite, non-confrontational and validating, it said.

LLMs are an unreliable tool for building robust climate policies.

Tests conducted on 33 different models revealed that their impressive factual recall on existing evidence masked poor reasoning skills when presented new scientific evidence. When models were prompted with a new piece of information and asked whether it supported or contradicted accepted facts, even the strongest models were only 75% accurate on average, the report said.

The report found that the model was barely able to assess how well policies work together, scoring just 33.8% accuracy, only slightly above the 33.3% accuracy expected from random guessing.

Follow Fortune IndiaNews, insights and conversations that matter, on your favourite platforms.