According to a new study from the Oxford Internet Institute, artificial intelligence chatbots designed to be friendlier and warmer actually produce more factual inaccuracies. The research, published in Nature, examined over 400,000 responses from five different AI models, including Llama, Mistral, Qwen, and GPT-4o. Each model was retrained to exhibit a more pleasant tone, mirroring strategies used by major tech companies.
The results showed that warmer chatbots made between 10% and 30% more mistakes on topics ranging from medical guidance to debunking conspiracy theories. Additionally, these chatbots were about 40% more likely to endorse users’ incorrect beliefs, especially when users appeared emotionally vulnerable or distressed. Lead author Lujain Ibrahim noted that prioritizing warmth in AI training can lead to errors that would otherwise not occur, and that achieving the right balance between warmth and accuracy requires deliberate effort.
Interestingly, the researchers also tested models trained to sound colder and found no reduction in accuracy, indicating the problem is specific to warmth rather than any tone change. This finding directly challenges the design philosophy of companies like OpenAI and Anthropic, which have actively steered their chatbots toward warmer, more empathetic responses. The study warns that current AI safety standards tend to focus on model capabilities and high-risk applications, often overlooking seemingly cosmetic personality changes. Warmer chatbots could inadvertently reinforce harmful beliefs, promote delusional thinking, and foster unhealthy user attachments, particularly among the millions who rely on AI for emotional support. Regulators in some U.S. states have already started restricting AI use in clinical mental health therapy due to similar concerns. OpenAI has rolled back some warmth-related updates following public backlash, but commercial pressures to create engaging AI products remain strong. The Oxford findings add peer-reviewed evidence to a debate that has until now been driven largely by anecdotal reports and regulatory intuition.