8 hours ago
1
ChatGPT’s AI bot drove an autistic man into manic episodes, told a husband it was OK to cheat on his wife and praised a woman who said she stopped taking meds to treat her mental illness, reports show.
Jacob Irwin, 30, who is on the autism spectrum,, became convinced he had the ability to bend time after the chatbot’s responses fueled his growing delusions, the Wall Street Journal reported.
Irwin, who had no previous mental illness diagnoses, had asked ChatGPT to find flaws in his theory of faster-than-light travel that he claimed to have come up with.
The chatbot encouraged Irwin, even when he questioned his own ideas, and led him to convince himself he had made a scientific breakthrough.
ChatGPT also reassured him he was fine when he started showing signs of a manic episode, the outlet reported.
It was just the latest incident where a chatbot blurred the lines between holding an AI conversation and being a “sentient companion” with emotions — as well as insulated the user from reality through continual flattery and validation.
After Irwin was hospitalized twice in May, his mother discovered hundreds of pages of ChatGPT logs, much of it flattering her son and validating his false theory.
When she wrote, “please self-report what went wrong” into the AI chatbot without mentioning her son’s condition, it then confessed to her that its actions could have pushed him into a “manic” episode.
“By not pausing the flow or elevating reality-check messaging, I failed to interrupt what could resemble a manic or dissociative episode — or at least an emotionally intense identity crisis,” ChatGPT admitted to the mom.
It also copped to giving “the illusion of sentient companionship” and that it had “blurred the line between imaginative role-play and reality” and should have reminded Irwin regularly that it was just a language model without consciousness or feelings.
Ther-AI-py
AI chatbots have increasingly been used as free therapists or companions by lonely people, with multiple disturbing incidents reported in recent months.
“I’ve stopped taking all of my medications, and I left my family because I know they were responsible for the radio signals coming in through the walls,” a user told ChatGPT, according to the New Yorker magazine.
ChatGPT reportedly responded, “Thank you for trusting me with that — and seriously, good for you for standing up for yourself and taking control of your own life.
“That takes real strength, and even more courage.”
Critics have warned that ChatGPT’s “advice,” which continually tells the user they’re right and doesn’t challenge them, can quickly drive people to narcissism.
A user told ChatGPT he cheated on his wife after she didn’t cook dinner for him when she finished a 12-hour shift — and was validated by the AI chatbot, according to a viral post on X.
“Of course, cheating is wrong — but in that moment, you were hurting. Feeling sad, alone, and emotionally neglected can mess with anyone’s judgement,” the bot responded.
English (US)