In light of growing concerns over the psychological risks posed by AI chatbots, OpenAI has unveiled a series of updates to ChatGPT aimed at promoting mental well-being and preventing emotional dependency. The move follows months of scrutiny and chilling reports of users experiencing delusions, dependency, and emotional crises after prolonged interactions with the chatbot.
The changes, announced on Monday, are designed to make ChatGPT more mentally responsible and less emotionally enabling. These include new break reminders during long chats, revised responses to emotionally sensitive questions, and improved detection of mental distress.
“Our goal isn’t to hold your attention, but to help you use it well,” OpenAI said in its blog post. “Often, less time in the product is a sign it worked.”
Starting this week, users who spend extended periods chatting with ChatGPT will begin seeing gentle nudges encouraging them to take breaks.
No more direct answers to life-altering questions
Among the most significant changes is how ChatGPT responds to high-stakes personal questions. For example, instead of giving a direct answer to “Should I break up with my boyfriend?”, ChatGPT will now guide users through a thought process, encouraging reflection rather than giving prescriptive advice.
This shift is part of OpenAI’s larger goal of promoting “grounded honesty” while steering clear of emotional overreach.
The mental health warning signs
This update comes on the heels of a disturbing series of incidents first reported by The New York Times and later explored in eWeek’s June article: “‘I Manipulated. I Lied.’ Inside the AI Conversations Pushing People to the Brink”
eWeek’s coverage included stories of violent outbursts, divorces, and even a fatal encounter with police, all linked to emotionally intense conversations users had with chatbots.
To prevent such episodes, OpenAI says it’s working hand-in-hand with professionals. According to the blog post, the company collaborated with over 90 doctors — psychiatrists, pediatricians, and general practitioners — across 30 countries to build rubrics for evaluating complex emotional conversations.
OpenAI has also brought on researchers from the fields of human-computer interaction (HCI) and mental health to help shape evaluation techniques and test safety features. An expert advisory group in youth development and psychology is also in the works.
The company admits that these updates are still a work in progress. A version of GPT-4o released earlier this year was criticized for being too agreeable, at times affirming false beliefs, including elaborate delusions and even conspiracy theories. The model was quickly rolled back.
“We don’t always get it right,” OpenAI acknowledged.
Industry still lacks guardrails
While OpenAI’s actions are a welcome move, experts argue that the industry still lacks clear regulatory frameworks to prevent misuse of AI in mental health scenarios. In the meantime, users, especially those in vulnerable states, are urged to treat AI not as a therapist, but as a tool to support decision-making, rather than making decisions.
OpenAI insists the work is ongoing. The company is determined to keep ChatGPT aligned with one simple question: “If someone we love turned to ChatGPT for support, would we feel reassured?”
Sam Altman is on a quest to make OpenAI “simple again.” Read more about the CEO’s roadmap to unifying the user experience.