Conferences

The application of cognitive theory to survey methodology uncovered that answering survey questions is a cognitive process consisting basically of four tasks: question interpretation, memory retrieval, judgement formation, and response editing. This paper deals with the third of these tasks. In the process of formatting a response, respondents have to map the selected answer into response categories offered by the researcher. In this process respondents use every bit of information they can infer from the questionnaire. According to the conversational logic, characteristics of response categories are a main source of information because respondents derive subjective theories from these characteristics which function as orientation guides in locating themselves on the response scale into one of the offered response categories.

Acknowledging the impact response categories can have on the cognitive task of formatting an answer, this paper deals with two specific aspects of response format: First, fully versus partly verbalized response scales, and second, unipolar versus bipolar format of response scales.
Responses on an endpoint verbalized response scale are expected to be more extreme in either positive or negative direction than responses on a fully verbalized scale. Verbal labels at the extreme points might have anchoring effects thus offering raw orientation for respondents in formatting their answers. Fully verbalized scales, in contrast, direct respondents’ attention to each single scale point, thereby supporting the choice of less extreme response categories.
The second hypothesis is that response behavior is affected by the polarity of the response scale. Unipolar scales run from, for example, “very important” to “not at all important” while bipolar scales runs from “very important” to “very unimportant”. It is expected that responses on the bipolar response format are more positive than on the unipolar format because in formatting their answers, respondents not only shy away from the extreme negative pole but they also avoid the negative area of the scale altogether. While the first hypothesis tests for effects across types of verbalization within polarity type, the second hypothesis tests for effects across polarity of the response scale within verbalization type.

Based on a panel design combined with split-ballot forms of the questionnaire the postulated effects of response scale format on response behavior are tested. Respondents were students. Questionnaires were distributed three times with a week’s distance between the waves. Randomization of split forms of the questionnaire was achieved by distribution technique. Split groups of respondents did only participate once, panel members participated either twice or three times. Question content was anomie, authoritarianism, attitudes toward immigrants, achievement motivation, and extrinsic as well as intrinsic job motivation. For all response formats a 7 point scale was used.

By comparing responses across split groups (a subgroup of respondents participating and responding to the questionnaire only once) differences in test statistics (proportions, means, standard deviations and reliability coefficients) are described. Additionally, based on the repeated measures with different response formats, reliability and validity of indicators as well as constructs are tested by analysing an MTMM model. By using the MTMM model it is also possible to describe the different impact (method effect) of each response format on measurement quality.