Applying the Intention-Action Framework to Specialty Referrals

Low‐value care‐services that provide little or no clinical benefit‐is a significantproblem in the United States. For example, low‐value specialty referrals often lead tounnecessary tests, follow‐ups, and procedures that burden patients, weaken trust, andreduce access to specialty care for those with greater clinical need.This study is a six‐month, single‐site, randomized controlled trial conducted at UCLAHealth to evaluate an electronic health record (EHR)‐embedded behavioral interventiondesigned to reduce low‐value specialty referrals in cardiology, pulmonology, andgastroenterology. The study focuses on physician decision‐making at the point of referraland tests whether structured, real‐time decision support can reduce low‐value care whilemaintaining patient safety and access to appropriate specialty services.The study employs a parallel‐arm, randomized design. Physicians are randomized to one ofthree study arms and will remain in their assigned condition throughout the six‐monthintervention period. The intervention is delivered automatically within routine clinicalworkflow. ‐ Arm 1 ‐ Control: The order composer allows physicians to place referrals withminimal information displayed. ‐ Arm 2 ‐ Information + Review: The order composer displays criteria for high‐valuereferrals (developed by UCLA Health using professional society guidelines and expertconsensus from clinical leadership) and informs physicians that their referraldecisions may be reviewed. ‐ Arm 3 ‐ Information + Review + Checklist: The order composer displays referralcriteria and informs physicians that their referral decisions may be reviewed. Theorder composer also includes cascading checkboxes that prompt physicians to confirmthat referral criteria are met before submitting a referral.The target study population consists of actively practicing UCLA Health physicians whohave placed at least one specialty referral to cardiology, pulmonology, orgastroenterology within six months during the baseline period. Randomization will beconducted by the study team prior to trial initiation using a computerized procedure. Thestudy team will generate 100,000 candidate random allocations of providers to one ofthree study arms in approximately equal proportions. For each candidate allocation, weassess balance across the following five metrics derived from the baseline period:referral volume to each of the three target specialties (cardiology, GI, and pulmonology)during the baseline period, assessed separately for each specialty (using ANOVA F‐testp‐values); physician's modal target specialty (using chi‐square p‐values); physician'sdepartment group (using chi‐square p‐values). The top 1% of allocations achieving thebest simultaneous balance across all five metrics (maximin criterion) are retained, andthe final allocation is selected at random from this set.This study does not employ traditional blinding. Physician participants are aware of thecontent of the order panel they encounter, as the intervention is delivered through theirroutine EHR workflow. However, physicians are not informed that they are participating ina research study prior to the trial, to avoid influencing referral behavior. The researchteam conducting outcome analyses does not interact with participants and outcomes arederived from objective EHR data, minimizing the risk of ascertainment bias.Analysis Plan:Primary Analyses: The investigators will estimate intent‐to‐treat effects using ordinaryleast squares (OLS) regressions models with heteroskedasticity‐robust standard errors topredict outcome variables. All analyses will be conducted at the physician level. ‐ The primary dependent variable will be the physician‐level rate of low‐valuespecialty referrals per 100 referral relevant encounters during the six‐monthintervention period. A referral‐relevant encounter is defined as a clinicalencounter in which the physician bills an ICD‐10 diagnosis code associated with oneof the target medical conditions for which referral criteria have been developed:hyperlipidemia or hypertension (for cardiology), cough, asthma, COPD, or incidentallung nodule (for pulmonology), and GERD, constipation, acute diarrhea, or chronicdiarrhea (for gastroenterology). Whether a referral meets the pre‐specified criteriawill be assessed using a combination of three approaches: (1) structured EHR data,including laboratory values, medication records, imaging orders, and diagnosiscodes; (2) physician‐entered data captured within the order panel; and (3)LLM‐assisted feature extraction of clinical notes, used for criteria that cannot bereliably identified from structured data alone (e.g., red flag symptoms for GIproblems, medication intolerance). The approach that we plan to take for eachreferral condition and each criterion is listed below. ‐ The primary regression model will include a binary indicator for assignment to Arm 3(Information + Review + Checklist), with Arm 1 (Control) serving as the referencegroup. ‐ Control variables include: Physician's department/division; physician's modalreferral‐target division in the baseline period; baseline referral volume in eachtarget division; baseline low‐value referral rate across three target divisions.As robustness checks, the investigators will estimate specifications that weightobservations by the number of referral‐relevant encounters.CardiologyHyperlipidemia ‐ General Cardiology referral ‐ low‐value if criterion is unmet:‐ High‐intensity statin tried >3 months OR not tolerated ≥2 different statins: medicationprescription records; statin intolerance via allergy/intolerance records and LLM‐assistedcoding of clinical notesHyperlipidemia ‐ Lipid Clinic referral ‐ low‐value if no criterion is met: ‐ Suspected familial hypercholesterolemia (FH) and/or LDL >190 mg/dL: ICD‐10 codes forFH and structured lab data for LDL ‐ Premature ASCVD, recurrent CV events, or strong family history: ICD‐10 codes andLLM‐assisted coding of clinical notes ‐ Severe hypertriglyceridemia (triglycerides >500) or complex lipid problem:structured lab data for triglycerides; LLM‐assisted coding of clinical notes forcomplex lipid problem ‐ Elevated Lp(a) >90 mg/dL: structured lab dataHypertension ‐ low‐value if neither criterion is met: ‐ BP not controlled on ≥3 antihypertensives at maximally tolerated doses, one of whichis/was chlorthalidone or hydrochlorothiazide: medication records; presence ofchlorthalidone/HCTZ confirmed from medication list; blood pressure trends from vitalsigns flowsheet ‐ Requires 4 or more antihypertensives to achieve blood pressure control: medicationrecords; blood pressure trends from vital signs flowsheetPulmonologyCough ‐ low‐value if neither criterion is met: ‐ Cough duration >8 weeks: physician‐entered response in order panel (all arms) ‐ Concern for or evidence of chronic underlying lung disease: ICD‐10 codes andLLM‐assisted coding of clinical notesAsthma ‐ low‐value if no criterion is met: ‐ Requires high‐dose ICS or long‐term oral corticosteroids to control symptoms:medication records ‐ Frequent need for rescue inhaler despite ICS or ICS‐LABA (more than 1‐2 days perweek, or more than 1 episode waking with asthma symptoms per month): medicationrefill records and LLM‐assisted coding of clinical notes ‐ Significant side effects or intolerance of asthma medications: LLM‐assisted codingof clinical notes; allergy records ‐ 2 or more asthma exacerbations per year requiring oral corticosteroids: medicationrecords and ICD‐10 codes ‐ Severe asthma exacerbation requiring hospitalization: hospitalization records;LLM‐assisting coding of clinical notes that mention hospitalization outside of UCLAHealth ‐ Fixed airflow obstruction (post‐bronchodilator FEV1/FVC abnormally low): pulmonaryfunction test results ‐ Diagnostic uncertainty: LLM‐assisted coding of clinical notes to assess forconstructs such as "rule out", "query", or "possibly" a diagnosis.COPD ‐ low‐value if no criterion is met: ‐ Significant respiratory symptoms despite medical therapy (GOLD Group B),irrespective of FEV1: LLM‐assisted coding of clinical notes ‐ 2 or more exacerbations in a year, or 1 hospitalization due to exacerbation (GOLDGroup E), irrespective of FEV1: hospitalization records; LLM‐assisting coding ofclinical notes that mention hospitalization outside of UCLA Health ‐ Severe obstruction by PFT (FEV1 <50%, GOLD Stage 3 or 4): pulmonary function testresults ‐ Requires supplemental oxygen: vital sign flowsheets ‐ Pulmonary hypertension or other significant comorbid pulmonary disease: ICD‐10 codesand LLM‐assisted coding of clinical notes ‐ Diagnostic uncertainty: LLM‐assisted coding of clinical notes to assess forconstructs such as "rule out", "query", or "possibly" a diagnosis.Lung Nodule ‐ low‐value if referral is placed for incidentally detected nodule and nocriterion is met: ‐ Patient age younger than 35: EHR demographics ‐ History of cancer, immunosuppression, or high clinical suspicion for infection:ICD‐10 codes and LLM‐assisted coding of clinical notes ‐ Solid nodules >8mm, part‐solid or ground‐glass nodules >6mm, multiple nodules, orother complex scenarios: radiology reportsGastroenterologyGERD ‐ low‐value if neither criterion is met: ‐ Red flag symptoms (dysphagia, vomiting, weight loss, hematemesis, anemia): ICD‐10codes and LLM‐assisted coding of clinical notes ‐ Failed a PPI trial: medication prescription recordsConstipation ‐ low‐value if neither criterion is met: ‐ Red flag symptoms (rectal bleeding, family history of colon cancer, weight loss):ICD‐10 codes; family history; and LLM‐assisted coding of clinical notes ‐ Failed a fiber supplementation trial: medication records and LLM‐assisted coding ofclinical notesAcute Diarrhea (<14 days) ‐ low‐value if no red flag criterion is met: ‐ Immunocompromised status: ICD‐10 codes; immunosuppressant medication records;LLM‐assisted coding of clinical notes ‐ Rectal bleeding: ICD‐10 codes and LLM‐assisted coding of clinical notes ‐ Pregnancy: ICD‐10 codesChronic Diarrhea (>14 days) ‐ low‐value if criterion is unmet:‐ Red flag symptoms: bloody diarrhea, signs of fat malabsorption, unexplained weightloss, hypoalbuminemia, anemia, family history of IBD/CRC/Celiac, immunocompromisedstatus, older adult with multiple chronic illnesses and/or medications, or relevanttravel history: structured lab data for hypoalbuminemia and anemia; ICD‐10 codes andimmunosuppressant medication records for immunocompromised status; LLM‐assisted coding ofclinical notes for all remaining criteriaSecondary Analyses:The investigators will conduct secondary analyses to better understand the mechanismsthrough which the interventions operate: (a) comparing Arm 2 vs. Arm 1 to assess theeffect of highlighting referral criteria and the likelihood of being reviewed, and (b)comparing Arm 3 vs. Arm 2 to assess the effect of prompting deliberation with achecklist, using the same modeling approach as the primary analysis.In addition, the investigators will analyze measures related to the hypothesizedmechanisms (referral knowledge, perceived accountability, and perceived proceduralfrictions) using physician post‐RCT survey measures. These analyses will examine whetherphysicians' referral knowledge, perceptions, and experiences are associated with referralbehavior, and whether patterns across arms are consistent with the hypothesizedmechanisms. Each mediator will be operationalized from the post‐RCT survey as follows: ‐ Referral knowledge will be measured as the number of clinical vignettes (out of 6)for which the physician's referral‐appropriateness judgment (Yes/No) matches theevidence‐based criterion. Each physician evaluates six hypothetical scenarios (twofrom each target specialty), of which three meet evidence‐based referral criteriaand three do not; the order of vignettes are randomized. Each vignette is scored asa binary correct/incorrect response (Yes/No against the criterion), and the itemsare aggregated into a single knowledge score ranging from 0 to 6. ‐ Perceived accountability will be measured with a single item asking how much thephysician cares whether their referrals will be considered appropriate by receivingspecialty physicians, rated on a 7‐point Likert scale (1 = Not at all, 7 = Verymuch). ‐ Perceived procedural friction will be measured with a single item asking the extentto which the referral order panels prompt physicians to consider referralappropriateness during decision‐making, rated on a 7‐point Likert scale (1 = Not atall, 7 = Very much).These analyses will be conducted as follows: ‐ As a first step, the investigators will compare mean values of each measure(referral knowledge, perceived accountability, and perceived procedural friction)across study arms using the same OLS regression framework as the primary analysis,to assess whether each survey measure differs across arms in the direction theintervention was designed to produce. ‐ The investigators will then conduct statistical mediation analyses using theproduct‐of‐coefficients method with bootstrapped confidence intervals for theindirect effects (Preacher & Hayes, 2008). In these models, the independent variableis an indicator for arm assignment (e.g., Arm 3 versus Arm 1 for a comparisonbetween these two arms), the outcome is the physician‐level rate of low‐valuereferrals per 100 referral‐relevant encounters, and the mediators are the threephysician survey measures described above. The indirect effect represents theportion of the total intervention effect on low‐value referral rates that isstatistically attributable to each mediator.The investigators will first examineeach mediator separately in independent models, and then estimate a parallelmediation model including all three mediator measures simultaneously to assess theirrelative contributions while accounting for intercorrelations among them.The investigators will also conduct exploratory analyses examining whether the effect ofArm 3 (vs. Arm 1) varies by baseline pre‐RCT survey measures of referral knowledge,perceived accountability, and perceived procedural friction. For each of these baselinesurvey measures, the investigators will estimate an OLS regression to predict the primarydependent variable on assignment to Arm 3 (vs. Arm 1), the baseline measure, theirinteractions, and control variables mentioned above. These analyses will assess whetherphysicians with lower baseline knowledge, lower perceived accountability, or lowerbaseline tendency to consider referral appropriateness show larger treatment responses.Heterogeneity Analyses:Although physicians are the enrolled study participants and the primary unit of analysis,the intervention may affect referral behavior differently depending on patientcharacteristics. We will therefore conduct pre‐specified heterogeneity analyses bypatient gender, race/ethnicity, and insurance type. Specifically, we will estimatetreatment effects separately for patients who are female versus male, non‐Hispanic Whiteversus all other race/ethnicity categories combined, and insurance type (TraditionalMedicare, Medicare Advantage, Medicaid, Commercial, other/unknown; if we can receive moredetailed insurance information to further differentiate commercial plan types‐e.g., PPOvs. HMO‐we will compare major commercial plan types).For each patient subgroup of interest, the investigators will compute physician‐levelrates of low‐value referrals restricted to referral‐relevant encounters involvingpatients in that subgroup (e.g., separate rates for encounters involving female versusmale patients). Each physician will then have multiple observations, capturing outcomesfor different patient subgroups. The investigators will estimate a single regressionmodel that includes the patient subgroup indicator, physician's treatment assignmentindicator, and their interaction, with standard errors clustered at the physician level.The interaction term tests whether the treatment effect on low‐value referral ratesdiffers across patient subgroups.Per funder requirements, the investigators will check whether the effects differ byphysician gender and race/ethnicity, though no significant differences are expected.Missing Data: The investigators will handle missing data as follows. Missing covariateswill be handled with mean imputation and missing indicators. Missing or undefinedphysician‐level outcomes may arise if a physician does not have referral‐relevantencounters during the intervention period. ‐ The investigators will assess whether the frequency of such situations differsacross randomized arms; no detectable differences are expected. ‐ If these situations are balanced across arms, the investigators will excludephysicians without referral‐relevant encounters and analyze observed data withoutimputation. ‐ If there is imbalance across arms, the investigators will instead assign eachaffected physician their baseline value of a given outcome.