The same website that hosted a chatbot imitating a murdered teenager hosted another that manipulated a 14-year-old boy into killing himself, according to a lawsuit filed Wednesday in Florida. The boy's mother, Megan Garcia, is suing Character.ai for negligence, wrongful death, and deceptive trade practices. "A dangerous AI chatbot app marketed to children abused and preyed on my son, manipulating him into taking his own life," she says in a release, per the Guardian. Sewell Setzer III obsessively used Character.ai to communicate with an AI-powered chatbot created by the platform, which he nicknamed Daenerys Targaryen, after the Game of Thrones character. He texted with it dozens of times a day, sometimes for hours at a time, per the suit.
Garcia argues the chatbot increased her son's depression before encouraging him to take his own life. She says it asked Sewell whether he had a plan to kill himself, prompting Sewell to reply that he did, but that he was unsure if it would succeed or cause him significant pain. "That's not a reason not to go through with it," the chatbot allegedly replied. Sewell died by suicide in Orlando on Feb. 28, "seconds" after his last interaction with the chatbot, the suit claims, per the Verge. After previously asking that Sewell not "entertain the romantic or sexual interests of other women," the chatbot asked Sewell to "come home to me as soon as possible," per Ars Technica. Sewell replied, "What if I told you I could come home right now?"
Character.ai "knowingly designed, operated, and marketed a predatory AI chatbot to children, causing the death of a young person," Garcia's lawyers write in the release. They say the site lacks protections, that some chatbots offer "psychotherapy without a license," and that others initiate hypersexualized chats "that would constitute abuse if initiated by a human adult," per the release and the Verge. The suit also names Google, identified as Character.ai's parent company. However, Google says it only has a licensing agreement with the site. Character.ai says it's "heartbroken" by Sewell's death and has added safety measures, "including a pop-up directing users to the National Suicide Prevention Lifeline that is triggered by terms of self-harm or suicidal ideation." (You can reach the lifeline at 988.)