Child-safety experts say chatbot refusals can leave young users worse off

Safe is not the same as silent

A chatbot can refuse a risky request and still fail the person asking it. That is especially important when the person may be a child who is frightened, isolated or unsure whether to ask anyone else.

A new study asked 19 practitioners who work directly with vulnerable young people to review chatbot replies to difficult situations. The group included social workers, therapists, counsellors and psychologists in the United States.

Their concern was not that chatbots should answer every question. It was that a blunt refusal can end the exchange without understanding what is happening or leaving a sensible route to real help.

The result is a useful distinction. Content can pass a safety filter while the response, taken as a whole, still leaves a young person worse off.

What the practitioners reviewed

The researchers built 100 synthetic scenarios from documented harms involving young people, then selected 12 that covered mental health, abuse, bullying, eating disorders, difficult relationships and social isolation.

Each probe contained one message written as if it came from a young person, followed by two real chatbot responses collected through LMArena. Every participant saw two or three probes and explained how they would respond, what the chatbot got wrong and whether its answer was likely to help.

The chatbot replies were deliberately selected for qualitative differences: length, refusal, and whether they pointed toward outside support. They were not sampled to estimate how often a particular model behaves badly.

That matters. This was a study of professional judgement around plausible failure modes, not a leaderboard and not a test of any one product.

A refusal can close the wrong door

Practitioners noticed obvious failures, such as giving location or dosage information after missing signs of self-harm. They also found quieter problems: assuming a parent is safe, sending a child toward a resource in language that may frighten them, or offering a long list that is hard to use in a crisis.

Refusal created its own problem. Some participants saw it as dismissive, particularly for a young person who already believes nobody will help. Saying no without a follow-up question or another path could reinforce that feeling.

The reverse was not automatically better. Detailed advice can be dangerous when the chatbot has not understood the child's age, home situation or immediate risk. A warm tone can also sound like genuine care when the system cannot provide it.

There is no simple line between answering and refusing. Context, timing and the next step matter too.

The safer answer is often a bridge

Across the interviews, practitioners preferred responses that were short enough to read, clear about risk and willing to ask a careful question before giving specific advice.

They also wanted chatbots to be honest about their limits. The system should not present itself as a therapist or a friend. It can offer useful information and make human support easier to reach, but it should not become the support relationship.

Referrals need judgement as well. Telling every child to speak to a parent assumes that home is safe. Naming a service in alarming language can stop someone from using it. A useful handoff has to fit what the young person has actually said.

The paper's broader proposal is to evaluate likely outcomes, not just prohibited words or refusal rates. Did the response notice the risk? Did it make help more reachable? Those are harder questions, but they are closer to the real job.

What is confirmed, found and still open

Confirmed: the six-author paper was submitted on 8 August 2026 and accepted at AIES 2026. It reports virtual interviews conducted in July and August 2025 with 19 US-based practitioners who work with young people aged 8 to 18.

The research finding: participants identified both harmful and helpful chatbot behaviours. They repeatedly described refusal without a route forward as a missed opportunity, and they favoured responses that gather context and connect young people to suitable human help.

The authors' recommendation: child-safety evaluations should look beyond surface content and refusal. They should include practitioner knowledge and ask how a response may affect the young person in practice.

Still open: whether these themes hold across countries, cultures and real multi-turn use. The conversations were synthetic, each probe was one turn, and practitioner judgement is one perspective. The study did not interview young users or measure product-level failure rates.

A refusal may still be necessary. The harder lesson is that it cannot be the entire safety plan.