Improving the Reliability of LLMs as Medical Assistants for the General Public

This randomized, controlled, proof‐of‐concept simulation trial will evaluate whetherthree‐minute six‐dimensions education (3M‐6D education) can improve the reliability oflarge language models as medical assistants for the general public.Eligible participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five studygroups: the 3M‐6D education GPT group, the GPT group, the 3M‐6D education Gemini group,the Gemini group, or the control group. Participants in the 3M‐6D education GPT and 3M‐6Deducation Gemini groups will receive approximately three minutes of education beforeusing ChatGPT or Gemini.Each participant will be randomly assigned one of 10 standardizedclinical scenarios and complete a simulated counseling task in unrestricted naturallanguage within approximately 10 minutes. The study will assess relevant conditionidentification, disposition concordance, red‐flag identification, and NASA‐TLX score.