Academic Commons

Articles

Another Look at Norris and Ortega (2000)

Shin, Hye Won

Since Norris and Ortega’s (2000) seminal work on the effectiveness of second language instruction, there has been a proliferation of meta-analyses in the field of applied linguistics. Subsequent meta-analysts, however, have uncritically followed the methodological choices made by Norris and Ortega. This paper suggests a critical reevaluation of the methodological procedures underlying the Norris and Ortega (2000) meta-analysis. I reexamined their procedures, and reassessed the 49 unique samples they used in their meta-analysis. In doing so, I identified three key methodological limitations with the study, pertaining, respectively, to (a) the data collection procedure, (b) the coding system, and (c) the statistical analysis. I argue that the lack of data quality inherent in the primary studies, the oversimplified coding scheme, and the inappropriate use of effect size statistics combine to compromise the validity of the conclusions Norris and Ortega have drawn from their meta-analysis. I subsequently provide alternative procedures which may yield a more empirically sound research synthesis, recommending, for future meta-analysts, the “best evidence synthesis‟ approach where conclusions are drawn from combining quantitative and qualitative analyses.

Files

Also Published In

Title
Working Papers in TESOL & Applied Linguistics

More About This Work

Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.