Conferences

Latent constructs measured by multi-item scales have a central position in theory and empirical research in the social sciences. A valid analysis of models with latent constructs requires a two-step approach: 1.Assessing the relations between a construct and its observed indicators (measurement model testing). 2. Analyzing the relations between the construct and other constructs (structural model testing). Yet, most methodological studies testing mode effect on measurement error in social surveys, lack an appropriate attention to measurement model testing. This lack is especially evident in mode effect studies dealing with sensitive subjects and among special populations, in which data quality is especially vulnerable to measurement invariance problems between the modes. Since mean comparison is a central goal in mode effect studies, measurement invariance should be assessed in order to insure that potential means differences can be interpreted correctly.

Surveying minorities on sensitive topics has recently become the focus of interest in survey methodology. The present study illustrates the use of measurement invariance procedure, as an integral part of testing mode effect on response error, in sensitive latent constructs measured among minorities. The study tested the effect of mode of data collection on measurement error, both random and non-random, using multigroup confirmatory factor analysis (MGCFA), which is a common method for measurement invariance testing. Representative face-to-face (N=701) and telephone (N=511) surveys, conducted in 2003 among the Arab minority in Israel, were used to test measurement and scalar invariance of three highly sensitive attitude scales concerning coexistence.

We tested configural invariance (same factor structure in the two modes) and metric invariance (same metric in the two modes), which is a necessary condition to ensure equivalence of the meaning of the construct. Scalar invariance (same systematic response bias in the two modes) was also tested since it is a precondition for latent mean comparison. Configural, metric and scalar invariances of two scales, social integration and threats, have been found and therefore latent mean comparison was conducted. However, for the prejudice scale only metric invariance holds. We conclude that for prejudice there is a mode effect on the measurement model and therefore also on the latent means. This lack in measurement invariance prevents a valid testing of structural invariance, due to confounding results.

The interpretation of results refers to possible causes of measurement error in face-to-face and telephone surveys in general and in the context of measuring sensitive topics among minorities. Conclusions of the specific study refer also to the advantages and limitations inherent in the techniques used, for mode effect studies.