Conferences

One important, but almost neglected source of bias and/or DIF (differential item functioning) is the order of the items employed to measure the latent construct. Effects on the response for a particular item may result from learning or tiring processes and/or other sources of DIF. This is particularly true for item-sets frequently administered in population surveys, since respondents may be not familiar with the domain under investigation, but rather learn about the very topic in the course of responding to the questions. Order/position effects on the probability of an item may be both item-specific and depend on characteristics of the respondents.

Like other forms of item bias, the effect of the order of items can only be investigated by means of a model, which parsimoniously represents the data at hand. Only one-dimensional models will be considered here. To estimate the order effects a generalized exploratory IRT model is employed (De Boeck & Wilson, 2004) which is formulated as random intercept model with a logit link and a binomial error (Skrondal & Rabe-Hesketh, 2004). Items are considered to be nested within respondents. The effect of the item position (general, or item-specific) on both the item-difficulty and the item-discrimination is investigated employing 1- and 2-parameter IRT models.

A survey in the republic of Germany was conducted to estimate the effects of interest for 5 dichotomous items designed to measure health related quality of live. Each of the possible 120 (5!) orders were administered in a fully balanced design on 1016 respondents. Order effects with respect to the item-difficulties were estimated and evaluated with respect to the latent dimension under consideration. Interaction between item-specific order effects and respondent-specific characteristics like gender are estimated as cross-level interactions. Computations were carried out by the module gllamm within STATA 10.1 (StataCorp, 2007).

References:

De Boeck, P. & M. Wilson. (2004). Explanatory Item Response Models: A General Linear and Nonlinear Approach. Edited by Paul De Boeck and Mark Wilson . Statistics for Social Science and Public Policy. Ed.: Fienberg, Stephen and Van der Linden, Wim.New York, Berlin: Springer.

Skrondal, A. & S. Rabe-Hesketh. (2004). Generalized Latent Variable Modeling; Multilevel, Longitudinal, and Structural Equation Models. Interdisciplinary Statistics Series. Ed.: Keiding, N., Morgan, B., Speed, T., and Van der Heijden, Peter G. M.London, New York: Chapman & Hall/CRC.

StataCorp. (2007). Stata Statistical Software: Release 10. Edited by StataCorp LP . College Station, TX.