Circuit Breaker Labs hopes to make AI safer for your kids (and you)

摘要: With all the talk about how AI might one day kill us all, it’s easy to forget that AI has already been life-threatening to some, not through bioweapon

正文

With all the talk about how AI might one day kill us all, it’s easy to forget that AI has already been life-threatening to some, not through bioweapons, but psychologically.

For example, Character.AI

settled several wrongful death lawsuits

earlier this year brought by families of underage users who died by suicide after interactions with its bots.

Multiple families have also sued OpenAI

over ChatGPT’s alleged role in their loved ones’ suicides and delusions.

Making AI safer across languages and cultures is the mission of Circuit Breaker Labs, one of TechCrunch’s 2026 Startup Battlefield 200 finalists. (Circuit Breaker will be

pitching at TechCrunch Disrupt

, which takes place this year at Moscone West in San Francisco from October 13-15.)

Founders Shirali and Arul Nigam, who are siblings, were motivated by Sewell Setzer, the 14-year-old who developed an emotional attachment to a Character.AI chatbot and confessed thoughts to it of harming himself before dying by suicide. The chatbot,

the parents alleged in a 2024 lawsuit

, encouraged him. The bot may not have understood what words like “I want to be with you” really implied, said Arul, who is Circuit Breaker Labs’ CTO.

“A lot of people, especially young people, turn to these systems for support, and usually they aren’t actually getting the help they need. But in many cases, they’re actively being harmed, and people unfortunately have taken their lives already,” Arul said. “Those sorts of safety vulnerabilities, where people aren’t necessarily actively trying to break the system — they’re engaging in a natural way — and the system has context pollution or it doesn’t understand the nuance, and then takes really dangerous action, we’re trying to prevent that.”

Circuit Breaker Labs has created AI agents that it likens to an army of crash-test dummies. These agents mimic folks from all ages, backgrounds, languages, and cultures, and are used to test models on their ability to detect dangerous, psychologicall

来源: TechCrunch AI