Comparison of mRNA Vaccine Effectiveness Against COVID-19-Associated Hospitalization by Vaccination Source: Immunization Information Systems, Electronic Medical Records, And Self-Report — Ivy Network, February 1–August 31, 2022

Background: Accurate determination of COVID-19 vaccination status is necessary to produce reliable COVID-19 vaccine effectiveness (VE) estimates. Data comparing differences in COVID-19 VE by vaccination sources (i.e., immunization information systems [IIS], electronic medical records [EMR], and self-report) are limited. We compared the number of mRNA COVID-19 vaccine doses identified by each of these sources to assess agreement as well as differences in VE estimates using vaccination data from each individual source and vaccination data adjudicated from all sources combined. Methods: Adults aged ≥18 years who were hospitalized with COVID-like illness at 21 hospitals in 18 U.S. states participating in the IVY Network during February 1–August 31, 2022, were enrolled. Numbers of COVID-19 vaccine doses identified by IIS, EMR, and self-report were compared in kappa agreement analyses. Effectiveness of mRNA COVID-19 vaccines against COVID-19-associated hospitalization was estimated using multivariable logistic regression models to compare the odds of COVID-19 vaccination between SARS-CoV-2-positive case-patients and SARS-CoV-2-negative control-patients. VE was estimated using each source of vaccination data separately and all sources combined. Results: A total of 4,499 patients were included. Patients with ≥1 mRNA COVID-19 vaccine dose were identified most frequently by self-report (n=3,570, 79%), followed by IIS (n=3,272, 73%) and EMR (n=3,057, 68%). Agreement was highest between IIS and self-report for 4 doses with a kappa of 0.77 (95% CI = 0.73–0.81). VE point estimates of 3 doses against COVID-19 hospitalization were substantially lower when using vaccination data from EMR only (VE = 31%, 95% CI = 16%–43%) than when using all sources combined (VE = 53%, 95% CI = 41%–62%). Conclusion: Vaccination data from EMR only may substantially underestimate COVID-19 VE. Declaration of Interests (continued): Funding for this work was provided to all participating sites by the United States Centers for Disease Control and Prevention. Samuel Brown reports institutional funds from Janssen for influenza research. Jonathan Casey reports a travel grant from Fischer and Paykel, outside the submitted work. Steven Chang reports consulting fees from PureTech 75 Health and Kiniksa Pharmaceuticals and participating as a DSMB member for a study at UCLA, outside the submitted work. Manjusha Gaglani reports grants from CDC, CDC-Abt Associates, CDC-Westat, and Janssen, and served as co-chair of the Infectious Diseases and Immunization Committee for the Texas Pediatric Society, outside the submitted work. Kevin Gibbs reports grants from NIH and DoD, as well as support for MHSRS 2022 travel from the DoD, outside the submitted work. Adit Ginde reports receiving grants from NIH, DoD, AbbVie, and Faron Pharmaceuticals, outside the submitted work. Michelle N. Gong reports grants from NHLBI, CDC, AHRQ, speaking at medicine grand rounds at New York Medical College, travel support for the ATS executive meeting and serving as ATS Chair Critical Care Assembly, DSMB membership fees from Regeneron, and participating on the scientific advisory panel for Endpoint, outside the submitted work. Carlos Grijalva reports grants from NIH, CDC, AHRQ, FDA, Campbell Alliance/Syneos Health; receiving consulting fees from and participating on a DSMB for Merck, outside the submitted work. David Hager reports receiving grants from NIH, outside the submitted work. Natasha Halasa reports receiving grants from Sanofi and Quidel, outside the submitted work. Akram Kahn reports receiving grants from United Therapeutics, Johnson & Johnson, 4D Medical, Eli Lily, Dompe Pharmaceuticals, and GlaxoSmithKline; and serves on the guidelines committee for Chest, outside the submitted work. Adam Lauring reports receiving grants from CDC, FluLab, NIAID, and Burroughs Wellcome Fund, and consulting fees from Sanofi and Roche for consulting on oseltamivir and baloxavir respectively, outside the submitted work. Emily Martin reports grants from Merck and NIH, outside the submitted work. Tresa McNeal reports receiving a grant from CDC, receiving a one-time payment for participating as a virtual webinar panelist for Clinical Updates in Heart Failure, and being a Practice Management Committee member for Society of Hospital Medicine, outside the submitted work. Ithan D. Peltan reports grants from NIH, Janssen Pharmaceuticals and institutional support from Asahi Kasei Pharma and Regeneron, outside the submitted work. Jennifer Wilson reports grants from NHLBI, outside the submitted work. No other potential conflicts of interest were disclosed.