OpenAI has announced new safeguards in ChatGPT aimed at improving how the system responds to users experiencing mental health distress.
In a post titled “Strengthening ChatGPT’s Responses in Sensitive Conversations,” the company announced its new upgrades designed to reduce “undesired responses” in sensitive areas such as psychosis, mania, self-harm, and emotional reliance on AI by between 65% and 80%.
OpenAI says it worked with more than 170 psychiatrists and psychologists to refine how ChatGPT handles people in crisis.
The post emphasized that the system now encourages users to reconnect with real-world connections and professional care, while discouraging unhealthy dependence on the chatbot itself.
A massive scale and a serious responsibility
OpenAI’s initial analysis estimates that around 0.15% of its 800 million weekly active users, or roughly 1.2 million people, talk to ChatGPT about suicide in a given week. Another 0.07%, or about 560,000 users, show signs of possible psychosis or mania.
While OpenAI describes these conversations as “extremely rare,” at this scale, even a low percentage represents a large number of individuals relying on AI for emotional support.
The company also said it routes certain high-risk conversations to models configured for greater safety. In one example provided, ChatGPT reassures a distressed user that “no aircraft or outside force can steal or insert your thoughts,” and offers grounding techniques and crisis resources.
Perceived optics and ongoing challenges
While the changes may reduce harm, OpenAI’s tone raises eyebrows. Its blog post reads like carefully worded promotional copy peppered with corporate jargon (including “taxonomies” and “desired behaviors”), while offering minimal transparency into how ChatGPT’s empathy is trained, measured, or audited.
OpenAI acknowledged that its definitions of “distress” continue to evolve and that it plans to refine its taxonomies and evaluations in future model releases.
OpenAI said these updates are part of an effort to ensure ChatGPT “supports and respects users’ real-world relationships” while maintaining safety in serious situations.
Still, skeptics argue that the company’s humanistic rhetoric doubles as brand management, questioning whether the change is ethical innovation or expertly packaged reassurance.
OpenAI CEO Sam Altman is taking a new step in the realm of AI… this time straight into the human brain.