Which Test Is Best: Evaluation of Traditional and Contemporary Statistical Tests for Analysis of Spherical Equivalent Prediction Error

PURPOSE: To characterize the performance of traditional and contemporary statistics tests for analysis of spherical equivalent prediction error (SEQ-PE) after cataract surgery, with regard to test significance and self-consistency. DESIGN: Comparison of the utility of statistical tests. SUBJECTS: Eyes from 5 academic centers and 2 private practices that had cataract surgery and postoperative manifest refraction between March, 2011 and December, 2022. METHODS: SEQ-PE data were randomly divided into sub-sets with sample sizes of 100, 300, 500, 700, and 2600 eyes. Mean absolute error (MAE), median absolute error (MedAE), standard deviation (SD), root mean squared absolute error (RMSAE), and the proportion of eyes within 0.50 D of predicted were calculated for six power prediction formulas and analyzed using Friedman post hoc Dunn, Cochran Q post hoc McNemar, Eyetemis, and Wilcox-Holladay-Wang-Koch (WHWK) statistical tests. All tests were corrected for multiple comparisons using the Holm correction. MAIN OUTCOME MEASURES: The percentage of significant relationships (Percent Significance), proportion of inconsistencies (Inconsistency Ratio), and proportion of self-consistent significant relationships (Significance Index) for each statistical test. RESULTS: Analysis was performed on 7,839 eyes of 7,839 patients. WHWK.MAE (42%), WHWK.SD (41%), Eyetemis.MAE (40%), WHWK.RMSAE (39%), and Dunn.MAE (34%) were more robust, respectively, than the remaining three tests by Percent Significance (all p < 0.001). Dunn.MAE had the best Inconsistency Ratio (0.11) in the 100-eye subsets. The same top five tests were most robust by Significance Index (0.39, 0.35, 0.35, 0.34, and 0.31, respectively; all p < 0.02). WHWK.SD and WHWK.RMSAE had the best Significance Indices (both 0.77) in the 2600-eye subsets. McNemar had the poorest Significance Index overall (0.09). CONCLUSIONS: The five high-performing tests produced significant results more often and were also self-consistent. WHWK.MAE and McNemar were highest and lowest performing overall, respectively. Dunn.MAE may be useful in sample sizes < 150 eyes.