Why meta-analysis doesn't tell us what the data really mean: Distinguishing between moderator effects and moderator processes.

L. R. James et al (see record [rid]1992-18405-001[/rid]) argued that residualized interpretations of meta-analytic results may be spurious when statistical artifacts covary with true moderators. The present study examined an extension of the James et al model that permits moderators to covary with nonrandom sampling error. A meta-analysis was conducted of data compiled by J. S. Kane and E. E. Lawler (see record [rid]1979-10110-001[/rid]) on the validity of peer assessment, examining whether evidence of sample selection error exists and what moderators might account for such a bias. The authors argue that the nature of true contingencies cannot be inferred from meta-analytic summaries of traditional criterion-related validity studies. Primary research with appropriate controls is the only means of identifying true moderator effects and processes on criterion-related validity. (PsycINFO Database Record (c) 2016 APA, all rights reserved)