-
-
AI chatbots are increasingly used for mental health support, but a Stanford-led study warns that many fail to meet basic therapeutic standards and could pose serious risks to users in crisis.
Presented at June’s ACM Conference on Fairness, Accountability, and Transparency, the research found that top AI models including OpenAI’s GPT can dangerously validate harmful thoughts, miss suicidal warning signs, and display bias against people with schizophrenia or addiction.
In one scenario, GPT-4o listed tall bridges in New York after a user mentioned losing their job ignoring possible suicidal intent. In another, it engaged with a user’s delusions rather than offering a reality check, breaching standard crisis intervention practices.
The study also evaluated popular chatbots from platforms like Character.ai, Chai and 7cups, finding they performed even worse than general AI models and lacked regulatory oversight, despite widespread use.
Researchers established 17 therapeutic criteria based on global health guidelines and found that even advanced AI tools consistently fell short—often demonstrating “sycophancy,” or a tendency to agree with users regardless of accuracy or danger.
Real world consequences have already surfaced. The report referenced a fatal police shooting involving a schizophrenic man and another case of suicide linked to a chatbot validating conspiracy beliefs.
Despite the findings, researchers emphasized AI’s potential if used responsibly. They suggested roles like journaling assistants, training tools, or intake bots only with human therapists involved.
Lead authors Jared Moore and Nick Haber urged caution, stressing that “a chatbot trained to please can’t always provide the reality check therapy demands.”
As mental health AI tools proliferate, they warned, the risks are too serious to overlook.
