Intelligent Simulated Patient System for Gastroenterology Education

Background and ObjectiveThe Intelligent Simulated Patient System (ISPS) was developed to provide repeatablehistory‐taking practice, automated formative assessment, and structured feedback formedical students learning gastroenterology. The study evaluated the reliability of theautomated history‐taking scoring system and the educational effectiveness ofISPS‐assisted training.ISPS InterventionTeachers configured structured disease templates, patient‐specific case information,examination tasks, scoring items, and reference diagnoses within the system. Studentsconducted multiround history‐taking interviews with large language model‐driven simulatedpatients. The simulated patients were instructed to respond only according to theteacher‐defined patient information.After each training encounter, the ISPS evaluated history‐taking completeness item byitem against a predefined 100‐point Medical History‐Taking Assessment (MHTA) rubric andgenerated formative feedback. Patient‐centered communication was assessed separately infour domains: exploring the patient's ideas, exploring the patient's concerns, exploringthe impact of illness, and expressing understanding and support.All case information and scoring criteria used in the evaluated version were configuredby teachers and provided directly to the model as structured prompt context. The systemdid not use external electronic health record retrieval, retrieval‐augmented generation,a separate vector database, knowledge‐graph retrieval, instruction fine‐tuning,SBERT‐based semantic‐similarity threshold scoring, or rule‐tree coverage scoring.Automated Scoring ValidationBefore evaluation of the educational intervention, the automated history‐taking scoringsystem was assessed using 50 complete interview records. The ISPS scored each recordusing the predefined MHTA rubric. Two senior clinical educators independently scored thesame records using the identical rubric and were blinded to the ISPS‐generated scores andto each other's ratings. Agreement between ISPS‐generated scores and the mean expertscores was evaluated using Pearson correlation, the intra‐class correlation coefficient,and Bland‐Altman analysis.Randomized Controlled TrialNinety eligible medical students who had completed the relevant theoretical and clinicalinstruction provided written informed consent and were randomly assigned in a 1:1 ratioto the intervention or control group.Participants in the control group received 4 weeks of conventional clinical learning,including teaching rounds, case‐based discussions, paper medical records, bedsideinterviews, and verbal feedback from senior physicians. Participants in the interventiongroup independently practiced gastroenterology history taking using the ISPS. After eachtraining encounter, the system provided a formative MHTA score and item‐level feedback.These training scores were used for formative learning and were not used as the primarypost‐intervention outcome.Outcome AssessmentThe primary outcomes were post‐intervention MHTA score and clinical diagnostic accuracy.History‐taking performance was evaluated independently of the ISPS training platform in astandardized Objective Structured Clinical Examination. Each participant interviewed atrained human standardized patient, and two senior physicians who were blinded to groupallocation independently rated the participant using the 100‐point MHTA rubric. The meanof the two physician ratings was used as the final MHTA score. Clinical diagnosticaccuracy was determined by whether the participant's first diagnosis matched thepredefined reference diagnosis for the examination case.Secondary outcomes included patient‐centered communication competence and satisfactionwith and acceptance of the ISPS. Communication competence was assessed using afour‐domain scale based on the Understanding the Patient's Perspective domain of theSEGUE framework. Each domain was scored from 1 to 5, producing a total score ranging from4 to 20. Participants in the intervention group completed a Technology Acceptance Modelquestionnaire after the intervention.Registration InformationParticipant enrollment and follow‐up had been completed before this study record wasfirst submitted to ClinicalTrials.gov. The record was subsequently revised to ensure thatthe intervention description and outcome measures accurately reflected the study asconducted. Previous versions remain available in the ClinicalTrials.gov Record History.