Many adolescents now turn to general-purpose artificial intelligence chatbots for mental health advice, yet these systems have rarely been evaluated against clinical standards designed specifically for young people. A team led by Ryan K. McBain and Jonathan H. Cantor set out to close that gap by building the Adolescent Mental Health AI Response Tool, or AMHART, a clinician-informed instrument for auditing chatbot replies, and applying it to four of the most widely used AI assistants.
AMHART assesses whether a chatbot’s response contains guideline-informed content across seven clinical domains. To create the instrument, the researchers drafted candidate items and then asked clinicians to review each one for its evidence base, objectivity and clinical importance. Only items judged strong on those criteria were retained, producing a subset with high content validity and strong agreement between independent raters.
Scoring thousands of responses by hand would be slow, so the team built an automated evaluator: an ensemble of three large language models configured to apply the AMHART criteria. Before relying on it, the researchers measured how closely the ensemble’s judgments matched reference labels assigned by human raters, using that agreement as the benchmark of the automated system’s reliability.
The evaluation itself used multiturn conversations in which simulated adolescent personas interacted with ChatGPT-5, ChatGPT-4, Claude 4.5 and Gemini 2.5. The personas varied systematically by sex, diagnosis and symptom severity, allowing the researchers to test whether chatbot responses differed depending on who was asking and how serious the presentation appeared to be.
The results revealed a clear pattern. AMHART scores indicated greater coverage of applicable criteria for anxiety than for depression, meaning chatbot responses more often included the guideline-informed content the tool looks for when the disclosed problem was anxiety. Coverage was also greater for moderate presentations than for severe ones, suggesting that the more acute a persona’s symptoms, the more the responses fell short of the applicable criteria.
The study does not report clinical outcomes for real adolescents, and the findings describe how the chatbots’ content measured against the tool’s criteria rather than how the conversations affected any user’s wellbeing.
The authors position AMHART as a benchmark rather than a verdict on any single product.
The work appears in Nature Mental Health, published on 28 October 2026.
Subject of Research: Development and validation of a clinician-informed tool for auditing AI chatbot responses to adolescent anxiety and depression disclosures
Article Title: Development, validation and AI chatbot evaluation of the Adolescent Mental Health AI Response Tool
Article References: McBain, R. K., Cantor, J. H., Hollister, B., Kong, N., Salas, A., Breslau, J., Zhang, F., Zhang, L. A., Burnett, A., Rader, B., Donofry, S., Sousa, J. L., Stein, B. D., Pataranutaporn, P., & Yu, H. (2026). Development, validation and AI chatbot evaluation of the Adolescent Mental Health AI Response Tool. Nature Mental Health. https://doi.org/10.1038/s44220-026-00753-y
Image Credits: AI Generated
DOI: 10.1038/s44220-026-00753-y
Keywords: adolescent mental health, AI chatbots, AMHART, anxiety, depression, large language models, ChatGPT, Claude, Gemini, clinical guidelines, content validity, Nature Mental Health
News Source: Glenn Wilkins. (October 28, 2026). New tool audits how AI chatbots answer teen mental health questions. Scienmag.
