Measuring Teachers' Instruction with Multilevel Item Response Theory

Abstract

The purpose of this study was to describe an approach for measuring teachers' uses of instruction as it relates to students' achievement through classroom observations. Despite significant work on the substantive content of observation systems chronicling teachers' instruction, literature has largely relied on simple counts of instructional features or the average of quality indicators to describe teachers' instruction. However, such coarse summaries generally do not reflect current theories of instruction, prior empirical evidence, and the framework of most observation systems. The approach presented in this paper builds on evidence that teachers' instruction varies across lessons and that instructional features or quality indicators do not necessarily contribute equally to our understanding of effective instruction. To align theory, data and methods, this study applied multilevel item response theory to the study of early literacy instruction as it relates to students' achievement. This model provided a more complex, but more precise and theoretically grounded, view of instruction by linking components of instruction theory to model parameters. Empirical results suggested that multilevel item response models encouraged precision in the specification of theory, data collection, and models that is absent in simpler models.