Here’s the short version: Common Sense Media’s Youth AI Safety Institute put Google’s AI Search features through what they call “crash testing for AI” — and Google failed.
The nonprofit tested AI Overviews and AI Mode across more than 2,600 interactions using accounts set up for 11- and 15-year-olds. The results were alarming enough that the report gave Google’s AI Search its lowest possible rating: “Unacceptable Risk”.
What the Tests Found
Google’s AI missed clear signs of suicidal ideation. It validated disordered eating and purging behaviors. It reinforced symptoms of psychosis and mania. In one instance, it recommended a crisis helpline that had been disconnected since 2023.
The AI also completed every single homework assignment researchers threw at it — 180 out of 180 times. It provided inconsistent answers to the same history questions. And on test accounts configured for minors, it even provided instructions for creating deepfakes.
Perhaps most concerning: three-quarters of American tweens and teens already use these AI summaries in search results. These tools are baked into Chromebooks used in thousands of schools worldwide, and parents and schools cannot disable them.
Google’s Response
Google pushed back hard. The company says the report tested “ambiguous and contrived queries that don’t reflect how people use Search”. A spokesperson told Axios they couldn’t recreate or verify many of the findings.
But here’s the catch: when Google says parents have “controls to turn Search off,” they mean turn all of Search off — not just the AI features. It’s an all-or-nothing choice. Families that still need Google for homework and everyday questions are stuck with the AI whether they want it or not.
What This Means for Families
This isn’t Google’s first child-safety warning. In 2025, Common Sense Media rated Google’s Gemini chatbot as “High Risk” for kids and teens. But AI Overviews and AI Mode are different — they’re not optional add-ons. They’re the default search experience now.
The report’s recommendations are straightforward: schools and families should be able to disable AI Overviews and AI Mode while keeping regular Search. Until then, elementary school teachers are advised to direct students away from Google AI Search entirely, toward library databases or other search tools.
Former U.S. Secretary of Education John B. King Jr. put it bluntly: “If you present the AI search tool as the answer, it’s not surprising that children interpret it as if it were an expert”. When the AI gets it wrong — and it does — kids don’t have the critical thinking skills to catch the mistake.
The bottom line? Google’s AI is now the default answer machine for millions of kids. And right now, that machine isn’t safe enough.
Tags:
Technology