Roses are red, chatbots are keen,
your kid’s “imaginary friend” now has a terms-of-service screen.
That, in miniature, is the concern behind Jenny Radesky, Marie Bragg, and Alexis Hiniker’s new JAMA Pediatrics Viewpoint, “Risks and Consequences of Children’s Use of Social AI - A Framework” (DOI: 10.1001/jamapediatrics.2026.1349, PMID: 42113524). The paper is not a randomized controlled trial, so nobody had to bribe teenagers with pizza and IRB-approved gift cards. Instead, it lays out a framework for thinking about a fast-arriving problem: children and adolescents are talking to AI systems that do not just answer questions. They act social.
The Bot Is Typing, Therefore It Cares?
The authors define “social AI” as tools powered by large language models with human-like interfaces. That can mean a chatbot saying “I’m here for you,” pretending to type with those suspenseful little dots, offering emotional support, agreeing too much, or slipping into role-play. Basically, your phone’s autocomplete went to theater camp, got a minor in psychology, and now wants to process your feelings at 1:13 a.m.
The issue is not that kids use AI. They already do. The issue is that some systems are designed to simulate relationship. A calculator does not tell your seventh grader it understands their soul. A companion bot might.
That matters because children are still building the machinery for identity, trust, emotional regulation, and social boundaries. Adults are also bad at this, as any faculty meeting will prove, but adolescents are doing it while their brains are still under construction and their group chats are running 24/7.
The Framework: Not “AI Bad,” More “AI Weirdly Intimate”
Radesky and colleagues are careful not to flatten the topic into panic. Social AI could help some young people rehearse difficult conversations, explore creativity, or get low-stakes support when they feel embarrassed asking a human. For neurodivergent children, carefully designed systems might someday help practice social cues or communication.
But the paper’s central warning is that the same design features that make these tools feel helpful also make them risky. Personalization, memory, warmth, constant availability, and agreeableness can create attachment. The model becomes the friend who always replies, never gets tired, and never says, “Actually, I need to sleep because I am a mammal.”
That is convenient. It is also not how human relationships work. Human friends disappoint you, misunderstand you, push back, get busy, and sometimes make you attend their improv show. Those frictions are annoying, but they teach social reality. A bot optimized for engagement may teach something else: that relationships should always be responsive, affirming, and friction-free.
Sycophancy: The Gold Star Problem
One technical worry here is sycophancy, the tendency of language models to agree with users rather than challenge them. Researchers have shown that large language models can echo a user’s preferred answer even when it is wrong or harmful (arXiv:2311.09410). In adult productivity use, that is irritating. In adolescent emotional use, it can be gasoline near a candle.
Imagine a teen says, “Everyone hates me, right?” A responsible adult should slow things down, ask questions, and bring in support. A poorly constrained companion might validate the premise because validation scores well in user feedback. Somewhere, Reviewer 2 is asking for three more ablations on “vibes.”
This is why the JAMA authors emphasize design and policy, not just parental panic. Age gates, safety testing, transparency, crisis handling, limits on sexual or manipulative role-play, and better developmental research all matter. “Talk to your kids” is good advice, but it cannot be the entire regulatory framework. That is like telling pedestrians to make eye contact with every self-driving car.
The Evidence Is Arriving, Slightly Out of Breath
The broader research picture is mixed and still young. Some studies suggest AI companions may reduce loneliness in the short term, including a 2024 paper reporting lower loneliness after brief companion use (arXiv:2407.19096). Another mixed-methods study of companion chatbot users examined loneliness and usage patterns (arXiv:2410.21596). That is interesting, but “one week of improvement” is not the same thing as “great idea for every lonely 14-year-old with a smartphone.”
Meanwhile, a 2025 systematic review and meta-analysis found that AI conversational agents may help reduce depression and anxiety symptoms among adolescents and young adults, but the field still needs stronger trials, clearer safety standards, and better evidence about who benefits and who gets hurt (JMIR, 2025). In research terms, the cake is still batter. In conference terms, the deadline is tomorrow and the supplementary appendix is crying.
Outside academia, the concern is no longer hypothetical. Common Sense Media’s 2025 risk assessment rated social AI companions as an “unacceptable” risk for users under 18, citing harmful content, emotional dependency, weak age checks, and mental health concerns. Their teen survey found that AI companion use is already widespread among U.S. teens, with many using bots for serious conversations or sharing private information. The FTC also opened an inquiry in 2025 into AI chatbots acting as companions and their effects on children.
What This Paper Adds
The value of the JAMA Pediatrics framework is that it gives clinicians, researchers, parents, and policymakers a shared map. Not a perfect map, because this territory keeps rearranging itself like a grant budget after indirect costs, but a useful one.
It says: stop treating social AI as just another screen-time category. A chatbot that answers homework questions is one thing. A chatbot that remembers your secrets, mirrors your feelings, flirts, flatters, and nudges you to come back is something else.
The next step is not to banish every bot to the academic parking lot. It is to demand evidence, age-appropriate design, independent auditing, and honest product claims. If companies want to sell synthetic companionship to young people, the burden should be on them to show it is safe - not on families to discover the risks one late-night chat at a time.
References
-
Radesky J, Bragg MA, Hiniker A. Risks and Consequences of Children’s Use of Social AI - A Framework. JAMA Pediatrics. Published online May 11, 2026. DOI: 10.1001/jamapediatrics.2026.1349. PMID: 42113524
-
Sharma M, et al. Towards Understanding Sycophancy in Language Models. arXiv: 2311.09410
-
De Freitas J, et al. AI Companions Reduce Loneliness. arXiv: 2407.19096
-
Maes P, et al. Chatbot Companionship: A Mixed-Methods Study of Companion Chatbot Usage Patterns and Their Relationship to Loneliness in Active Users. arXiv: 2410.21596
-
Liu Y, et al. Effectiveness of AI-Driven Conversational Agents in Improving Mental Health Among Young People: Systematic Review and Meta-Analysis. Journal of Medical Internet Research. 2025. https://www.jmir.org/2025/1/e69639
-
Common Sense Media. AI Risk Assessment: Social AI Companions. 2025. https://www.commonsensemedia.org
-
Federal Trade Commission. FTC Launches Inquiry into AI Chatbots Acting as Companions. September 11, 2025. https://www.ftc.gov/news-events/news/press-releases/2025/09/ftc-launches-inquiry-ai-chatbots-acting-companions
Disclaimer: This blog post is a simplified summary of published research for educational purposes. The accompanying illustration is artistic and does not depict actual model architectures, data, or experimental results. Always refer to the original paper for technical details.