An evaluation of an artificial intelligence-enabled prototype of the Participation and Environment Measure designed for family-centered goal setting in early intervention pediatric rehabilitation demonstrated that the comprehensive software platform maintained an acceptable level of overall system usability, even while its integrated conversational agent received very poor usability ratings from participating caregivers. The single-arm usability study, conducted with families enrolled in an urban early intervention program, examined whether adding interactive computational components into established pediatric goal-planning workflows could preserve software feasibility without overwhelming end users. According to study findings published in BMC Medical Informatics and Decision Making, caregivers reported an acceptable overall System Usability Scale score of 67.0 out of 100 for the entire digital application. In contrast, evaluation of the embedded conversational agent using the BOT Usability Scale revealed a median score of 53.4 percent, indicating substantial performance discrepancies across software layers during pediatric rehabilitation service planning.
Family-centered care coordination serves as a recognized best practice standard across pediatric rehabilitation services, prioritizing structured partnerships between rehabilitation practitioners and family members of young children with disabilities. Within conventional early intervention structures, care planning frequently encounters operational limitations, such as relying on a single synchronous interview between providers and family members to define developmental priorities. These constrained formats can impede caregivers from contributing their firsthand insights into their children’s daily participation barriers, support requirements, and environmental contexts. Digital health platforms have emerged as a potential remedy to provide families with adaptable timing, centralized recordkeeping, and continuous collaboration with clinical teams. The Participation and Environment Measure framework operationalizes this approach through two interconnected components: the Young Children’s Participation and Environment Measure, an electronic patient-reported outcome measure, and PEM-Plus, an intervention platform guiding parents through prioritizing participation needs, formulating specific developmental goals, and choosing actionable intervention strategies to attain meaningful family outcomes.
To enhance how diverse families interact with complex goal-setting steps, investigators incorporated artificial intelligence techniques into the Participation and Environment Measure architecture. Prior research demonstrated that natural language processing could classify caregiver-generated strategies into standardized participation constructs, supported by human and synthetic data augmentation within pipelined classification structures. Building upon these algorithmic capabilities, the research team engineered a prototypic conversational agent designed to provide dynamic navigation support throughout the PEM-Plus workflow. This software design directly addressed qualitative feedback from historically minoritized families who had identified reading comprehension challenges, difficulties navigating lengthy written instructions, and an absence of tailored guidance in earlier iterations. Unlike static instructional videos previously deployed to assist families, conversational agents offer dynamic interaction capabilities that can assist caregivers with low health literacy, provide neutral computational interfaces that mitigate provider-related communication barriers, and reinforce caregiver autonomy during early intervention shared decision-making processes across diverse clinical care environments.
The technical development of the conversational agent prototype utilized Dialogflow, a cloud computing platform supporting intent matching, entity extraction, and conversational context management. Rather than relying on data-heavy machine learning models that require continuous real-time model retraining, the conversational agent employed a rule-based slot-filling architecture governed by a static dialogue policy. The developmental process unfolded across three deliberate stages: curating training input, configuring intent matching algorithms, and establishing structured automated responses. The team seeded the software with thirty-five distinct intents and two hundred three associated training phrases derived from common frequently asked questions documented during prior PEM-Plus testing phases. These intents were categorized into three functional groups: activity and strategy intents providing context-specific guidance, step-specific intents answering operational questions about particular application screens, and general intents handling basic greetings and user prompts. Responses ranged from brief static text notifications to complex interactive menus featuring selectable visual categories and clickable hyperlinks.
The visual interface of the conversational agent featured Pemba, an animated persona whose name originates from an African contextual term signifying the force of the present. Depicted as a cisgendered and racially minoritized animated character in earlier project recruitment and instructional media, Pemba was embedded directly into the client-facing user interface to foster consistent rapport during user onboarding, assessment review, and intervention completion. Caregivers could initiate automated dialogues by selecting a dedicated profile icon at any stage of account generation, following completion of the electronic patient-reported outcome report, and throughout the PEM-Plus intervention. When caregivers requested assistance with home-based participation strategies, Pemba responded with interactive prompts differentiating child-focused accommodations from environment-focused modifications. This branching dialogue structure aimed to support parents in navigating complex clinical exercises, helping users systematically locate and explore peer-tested strategies that aligned with their specific domestic routines, community activities, and developmental priorities.
The usability trial enrolled eleven female caregivers of young children receiving early intervention services through the second largest urban broker in Colorado between February 2024 and March 2026. Enrolled caregivers met five strict eligibility criteria: holding legal guardianship of an enrolled child, being at least eighteen years of age, demonstrating English literacy, possessing internet and telephone access, and having a child aged zero to three years who had received services for at least three months under Part C of the Individuals with Disabilities Education Act. The participating children had a mean age of 24.5 months with a standard deviation of 6.2 months. The sample was characterized by high socioeconomic stability, with 45.5 percent holding graduate degrees, 63.6 percent reporting household incomes above sixty thousand dollars, and 72.7 percent identifying as non-Hispanic, Latinx, or Spanish. Most enrolled children were male (72.7 percent) and received developmental delay services (72.7 percent).
The evaluation framework paired two standardized usability instruments to isolate perceptions of the overarching system from perceptions of the conversational agent prototype. Global platform usability was evaluated using the ten-item System Usability Scale, which yields scores from zero to one hundred, where scores under fifty indicate severe usability deficits. Usability of the conversational agent was measured using the eleven-item BOT Usability Scale, which evaluates four discrete domains: accessibility of chatbot functions, functional interactive conversation, privacy and security, and timely response. Total BOT Usability Scale scores are converted into percentages, with values under sixty-seven percent classified as very poor. Most caregivers (91 percent) endorsed technology as important for accessing early intervention services, and 66.7 percent actively engaged with Pemba during one or more intervention iterations. Approximately thirty percent noted that they did not observe or use Pemba, while active users frequently consulted the agent during priority activity selection, peer strategy searches, and final report generation.
Usability metrics demonstrated a sharp divergence between total platform utility and conversational agent performance. The complete artificial intelligence-enabled Participation and Environment Measure application achieved an acceptable System Usability Scale score of 67.0, with participant evaluations spanning from 47.5 to 90.0. Conversely, the Pemba conversational agent received an overall BOT Usability Scale median score of 53.4 percent, with an interquartile range from 50.0 to 55.4 percent, placing it in the very poor category. Domain-level analyses revealed nuanced perceptions: caregivers rated privacy and security as good, generating a mean score of 3.1 across a two-to-five range, while accessibility of functions achieved an average median rating of 3.8. Open-ended caregiver feedback confirmed that while the agent was easily located and functioned smoothly, response latency was noticeable. Statistical analyses via univariate logistic regression models showed that higher usability ratings in three chatbot domains corresponded with higher odds of completing multiple care plans, but these associations did not achieve statistical or clinical significance.
Discussing these findings, the study authors highlighted multiple technical and contextual factors influencing the split usability scores. While the overarching platform maintained usability comparable to earlier non-computational versions, the System Usability Scale may reflect broader parental impressions of early intervention goal formulation rather than isolated digital interface mechanics. Furthermore, the presence of mixed positive and negative item wordings within the scale could introduce measurement noise and outlier effects. For the conversational agent, shortcomings in conversational depth and processing speed highlighted critical areas for iterative technical optimization. The authors proposed integrating expanded natural language classification mechanisms into Pemba to automatically parse parent-reported participation barriers, such as adverse community racial climates or negative interpersonal experiences at neighborhood playgrounds. Equipping future agents with context-aligned, empathic language models could deliver personalized, psychologically safe navigation support, empowering families to identify environmental accommodations that directly counter structural participation obstacles encountered during daily routines.
Several limitations contextualize the trial findings, foremost being a small sample size concentrated among Caucasian caregivers with higher educational attainment and household incomes, which restricts generalizability across broader, socially disadvantaged early intervention cohorts. In addition, measurement timing varied across the cohort: while most participants submitted evaluation instruments immediately following plan finalization, four caregivers delayed questionnaire completion by at least one month until compensation processing, introducing potential recall bias. Furthermore, sparse responses to optional open-ended items limited qualitative insights into specific conversational friction points. Future stages of the overarching trial, registered under trial identifier NCT06458959, will integrate qualitative interviews and focus groups within a mixed-methods design to refine chatbot response speed, expand personalized strategy recommendations, and evaluate whether conversational agents can alleviate provider burdens during pediatric rehabilitation care coordination transitions between specialized hospital environments and early educational settings.
Subject of Research: Usability evaluation of an artificial intelligence-assisted pediatric rehabilitation goal-setting application
Article Title: Usability of an artificial intelligence enabled Participation and Environment Measure prototype for family-centered goal setting support in pediatric rehabilitation
Article References: Lee, G., Villegas, V., Patton, L., Han, Y., Khetani, M., & Parde, N. (2026). Usability of an artificial intelligence enabled Participation and Environment Measure prototype for family-centered goal setting support in pediatric rehabilitation. BMC Medical Informatics and Decision Making, 26(1), Article 478. https://doi.org/10.1186/s12911-026-03890-7
Image Credits: AI Generated
DOI: 10.1186/s12911-026-03890-7
Keywords: pediatric rehabilitation, early intervention, Participation and Environment Measure, PEM-Plus, Young Children's Participation and Environment Measure, conversational agents, healthcare chatbots, System Usability Scale, BOT Usability Scale, family-centered care coordination, Dialogflow, natural language processing, goal setting, patient-reported outcomes, caregiver engagement, developmental delay, CHAT framework
News Source: Harold Sullivan. (November 18, 2026). AI-Enabled Pediatric Rehabilitation Prototype Retains Acceptable Usability Despite Lower Chatbot Ratings. Scienmag.



