Abstract

The methodological controversy surrounding ordinal outcome data has posed a distinct challenge to the conceptualization, design, and conduct of research in the social and behavioral sciences for more than 75 years. Accordingly, this study sought to supply a comprehensive and multidisciplinary perspective of the debate and in so doing lay the necessary foundation for proposing practical guidance to education researchers. A systematic review was undertaken with selection criteria aimed at assembling sources comprising a substantial discussion of both the measurement and analysis of ordinal outcome data as well as the controversy surrounding ordinal outcome data. A total of 77 peer-reviewed and published journal articles were obtained by this method. An additional 11 sources covering broad statistical topics were also referenced to provide an encompassing description of the intellectual landscape of ordinal methodology. Results showed that methodological purism, methodological pragmatism, and latent variable modeling all have merit when making decisions about how to analyze ordinal outcome data. This suggests that the intended use and stakes tied to research studies can be relied upon to direct analytic choices looking to balance rigor with realism. Irrespective of the approach, sensitivity analyses and more conservative significance levels are recommended in the absence of a clear-cut direction.

Key words: Quantitative methodology, data analysis, statistics, ordinal data, education.