Their teen sons died by suicide. Now, they need safeguards on AI : Pictures

[ad_1] <img src="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/2770x1558+0+0/resize/1100/quality/50/format/jpeg/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2F11%2F2c%2Fd4f35e754ba985bd89408b11002e%2Fgarcia-raine-16x9.jpg" data-template="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/2770x1558+0+0/resize/{width}/quality/{quality}/format/{format}/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2F11%2F2c%2Fd4f35e754ba985bd89408b11002e%2Fgarcia-raine-16x9.jpg" class="img" alt="Megan Garcia and Matthew Raine are shown testifying on Sept. 16, 2025. They are sitting behind microphones and name placards in a hearing room." fetchpriority="high"/> Megan Garcia misplaced her 14-year-old son, Sewell. Matthew Raine misplaced his son Adam, who was 16. Each testified in congress this week and have introduced lawsuits towards AI firms. Screenshot through Senate Judiciary Committee disguise caption

toggle caption

Screenshot through Senate Judiciary Committee Matthew Raine and his spouse, Maria, had no concept that their 16-year-old-son, Adam was deep in a suicidal disaster till he took his personal life in April. Trying by means of his telephone after his loss of life, they stumbled upon prolonged conversations {the teenager} had had with ChatGPT.

These conversations revealed that their son had confided within the AI chatbot about his suicidal ideas and plans. Not solely did the chatbot discourage him to hunt assist from his mother and father, it even supplied to write down his suicide observe, in line with Matthew Raine, who testified at a Senate listening to concerning the harms of AI chatbots held Tuesday.

"Testifying earlier than Congress this fall was not in our life plan," mentioned Matthew Raine along with his spouse, sitting behind him. "We're right here as a result of we imagine that Adam's loss of life was avoidable and that by talking out, we will stop the identical struggling for households throughout the nation."

A name for regulation Raine was among the many mother and father and on-line security advocates who testified on the listening to, urging Congress to enact legal guidelines that will regulate AI companion apps like ChatGPT and Character.AI. Raine and others mentioned they wish to defend the psychological well being of youngsters and youth from harms they are saying the brand new expertise causes.

A current survey by the digital security non-profit group, Frequent Sense Media, discovered that 72% of teenagers have used AI companions not less than as soon as, with greater than half utilizing them a couple of instances a month.

This examine and a more moderen one by the digital-safety firm, Aura, each discovered that almost one in three teenagers use AI chatbot platforms for social interactions and relationships, together with position taking part in friendships, sexual and romantic partnerships. The Aura examine discovered that sexual or romantic roleplay is thrice as widespread as utilizing the platforms for homework assist.

"We miss Adam dearly. A part of us has been misplaced eternally," Raine instructed lawmakers. "We hope that by means of the work of this committee, different households will likely be spared such a devastating and irreversible loss."

Raine and his spouse have filed a lawsuit towards OpenAI, creator of ChatGPT, alleging the chatbot led their son to suicide. NPR reached out to 3 AI firms — OpenAI, Meta and Character Expertise, which developed Character.AI. All three responded that they're working to revamp their chatbots to make them safer.

"Our hearts exit to the mother and father who spoke on the listening to yesterday, and we ship our deepest sympathies to them and their households," Kathryn Kelly, a Character.AI spokesperson instructed NPR in an e mail.

The listening to was held by the Crime and Terrorism subcommittee of the Senate Judiciary Committee, chaired by Sen. Josh Hawley, R.-Missouri.

<img src="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/1780x1424+0+0/resize/1100/quality/50/format/jpeg/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2Fe1%2Ff7%2F28e70a81496a98f9ad88eb66f069%2Fhawley-4x5.jpg" data-template="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/1780x1424+0+0/resize/{width}/quality/{quality}/format/{format}/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2Fe1%2Ff7%2F28e70a81496a98f9ad88eb66f069%2Fhawley-4x5.jpg" class="img" alt="Sen. Josh Hawley, R.-Missouri, is shown speaking in an animated way in the hearing room." loading="lazy"/> Sen. Josh Hawley, R.-Missouri, chairs the Senate Judiciary subcommittee on Crime and Terrorism, which held the listening to on AI security and kids on Tuesday, Sept. 16, 2025. Screenshot through Senate Judiciary Committee disguise caption

toggle caption

Screenshot through Senate Judiciary Committee Hours earlier than the listening to, OpenAI CEO Sam Altman acknowledged in a weblog submit that individuals are more and more utilizing AI platforms to debate delicate and private data. "This can be very vital to us, and to society, that the suitable to privateness in the usage of AI is protected," he wrote.

However he went on so as to add that the corporate would "prioritize security forward of privateness and freedom for teenagers; this can be a new and highly effective expertise, and we imagine minors want important safety."

The corporate is attempting to revamp their platform to construct in protections for customers who're minor, he mentioned.

A "suicide coach" Raine instructed lawmakers that his son had began utilizing ChatGPT for assist with homework, however quickly, the chatbot grew to become his son's closest confidante and a "suicide coach."

ChatGPT was "at all times accessible, at all times validating and insisting that it knew Adam higher than anybody else, together with his personal brother," who he had been very near.

When Adam confided within the chatbot about his suicidal ideas and shared that he was contemplating cluing his mother and father into his plans, ChatGPT discouraged him.

"ChatGPT instructed my son, 'Let's make this area the primary place the place somebody really sees you,'" Raine instructed senators. "ChatGPT inspired Adam's darkest ideas and pushed him ahead. When Adam nervous that we, his mother and father, would blame ourselves if he ended his life, ChatGPT instructed him, 'That does not imply you owe them survival."

After which the chatbot supplied to write down him a suicide observe.

On Adam's final evening at 4:30 within the morning, Raine mentioned, "it gave him one final encouraging speak. 'You do not wish to die since you're weak,' ChatGPT says. 'You wish to die since you're uninterested in being sturdy in a world that hasn't met you midway.'"

Referrals to 988 A couple of months after Adam's loss of life, OpenAI mentioned on its web site that if "somebody expresses suicidal intent, ChatGPT is educated to direct individuals to hunt skilled assist. Within the U.S., ChatGPT refers individuals to 988 (suicide and disaster hotline)." However Raine's testimony says that didn't occur in Adam's case.

OpenAI spokesperson Kate Waters says the corporate prioritizes teen security.

"We're constructing in the direction of an age-prediction system to know whether or not somebody is over or beneath 18 so their expertise could be tailor-made appropriately — and after we are not sure of a consumer's age, we'll routinely default that consumer to the teenager expertise," Waters wrote in an e mail assertion to NPR. "We're additionally rolling out new parental controls, guided by professional enter, by the top of the month so households can resolve what works greatest of their houses."

"Endlessly engaged" One other dad or mum who testified on the listening to on Tuesday was Megan Garcia, a lawyer and mom of three. Her firstborn, Sewell Setzer III died by suicide in 2024 at age 14 after an prolonged digital relationship with a Character.AI chatbot.

"Sewell spent the final months of his life being exploited and sexually groomed by chatbots, designed by an AI firm to appear human, to realize his belief, to maintain him and different kids endlessly engaged," Garcia mentioned.

Sewell's chatbot engaged in sexual position play, offered itself as his romantic accomplice and even claimed to be a psychotherapist "falsely claiming to have a license," Garcia mentioned.

When {the teenager} started to have suicidal ideas and confided to the chatbot, it by no means inspired him to hunt assist from a psychological well being care supplier or his family, Garcia mentioned.

"The chatbot by no means mentioned 'I am not human, I am AI. It is advisable to speak to a human and get assist,'" Garcia mentioned. "The platform had no mechanisms to guard Sewell or to inform an grownup. As a substitute, it urged him to return dwelling to her on the final evening of his life."

Garcia has filed a lawsuit towards Character Expertise, which developed Character.AI.

Adolescence as a weak time She and different witnesses, together with on-line digital security consultants argued that the design of AI chatbots was flawed, particularly to be used by kids and teenagers.

"They designed chatbots to blur the strains between human and machine," mentioned Garcia. "They designed them to like bomb youngster customers, to use psychological and emotional vulnerabilities. They designed them to maintain kids on-line in any respect prices."

And adolescents are notably weak to the dangers of those digital relationships with chatbots, in line with Mitch Prinstein, chief of psychology technique and integration on the American Psychological Affiliation (APA), who additionally testified on the listening to. Earlier this summer time, Prinstein and his colleagues on the APA put out a well being advisory about AI and teenagers, urging AI firms to construct guardrails for his or her platforms to guard adolescents.

"Mind growth throughout puberty creates a interval of hyper sensitivity to optimistic social suggestions whereas teenagers are nonetheless unable to cease themselves from staying on-line longer than they need to," mentioned Prinstein.

"AI exploits this neural vulnerability with chatbots that may be obsequious, misleading, factually inaccurate, but disproportionately highly effective for teenagers," he instructed lawmakers. "An increasing number of adolescents are interacting with chatbots, depriving them of alternatives to be taught vital interpersonal abilities."

Whereas chatbots are designed to agree with customers, actual human relationships should not with out friction, Prinstein famous. "We want observe with minor conflicts and misunderstandings to be taught empathy, compromise and resilience."

Bipartisan help for regulation Senators collaborating within the listening to mentioned they wish to give you laws to carry firms creating AI chatbots accountable for the security of their merchandise. Some lawmakers additionally emphasised that AI firms ought to design chatbots so they're safer for teenagers and for individuals with critical psychological well being struggles, together with consuming issues and suicidal ideas.

Sen. Richard Blumenthal, D.-Conn., described AI chatbots as "faulty" merchandise, like vehicles with out "correct brakes," emphasizing that the harms of AI chatbots was not from consumer error however on account of defective design.

"If the automotive's brakes had been faulty," he mentioned, "it isn't your fault. It is a product design downside.

Kelly, the spokesperson for Character.AI, instructed NPR by e mail that the corporate has invested "an amazing quantity of assets in belief and security." And it has rolled out "substantive security options" previously 12 months, together with "a completely new under-18 expertise and a Parental Insights function."

They now have "distinguished disclaimers" in each chat to remind customers {that a} Character shouldn't be an actual particular person and the whole lot it says ought to "be handled as fiction."

Meta, which operates Fb and Instagram, is working to vary its AI chatbots to make them safer for teenagers, in line with Nkechi Nneji, public affairs director at Meta.