Academic Commons

Reports

Why we (usually) don't have to worry about multiple comparison

Gelman, Andrew E.; Hill, Jennifer; Yajima, Masanao

Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p-values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.

Subjects

Files

  • thumnail for Gelman_CPRC_WPS_09-12_0.pdf Gelman_CPRC_WPS_09-12_0.pdf application/pdf 292 KB Download File

More About This Work

Academic Units
Columbia Population Research Center
Publisher
Columbia Population Research Center
Series
Columbia Population Research Center Working Papers, 09-12
Published Here
January 12, 2011

Notes

September 2009.

Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.