Conferences

A well-known advice on survey design is that balance between positive and negative wordings should be obtained. The assumption behind this advice is that respondents should give the same answers to positive and negative questions. However, a large body of research shows that this assumption is incorrect; people express their opinions more positively when the question is worded negatively.

By relating wording effects to the cognitive process underlying question answering, it can be understood why the asymmetry between positive and negative questions arises. Tourangeau, Rips & Rasinksi (2000) describe four stages in answering attitudinal questions. With respect to the validity of survey questions, it is fruitful to cut down their model into two stages: (1) interpretation of the question and retrieving an evaluation (2) mapping the evaluation onto the response options. If the question wording affects the interpretation stage, positive and negative questions measure different attitudes and differ in their validity. If question wording affects the mapping stage, contrastive questions are equally valid, but the same attitudes are expressed differently onto the response options.

In previous research, Holleman (1999) observed a high correlation between answers to positive and negative questions. This implies that question wording does not affect the interpretation stage. Because the answers to positive and negative questions do differ, the question wording must have become relevant in the mapping stage. More direct evidence comes from Chessa & Holleman (2007). In their experiment, question and answering options were offered separately to respondents, and the times for reading the question (interpretation) and giving an answer (mapping) were recorded. The results show that there is no difference in interpretation times for positive and negative questions, but that the mapping stage takes longer for negative questions.

Based on the evidence available we would conclude that wording effect arise in the mapping stage. Yet, the correlational study gives only indirect evidence for this conclusion, and a drawback of the reaction time study is that the natural process of question answering may have been distorted by separating question and answering options. The aim this research is to explore whether eye-tracking can be used as a non-distorting method to measure cognitive processes.

In an experiment, respondents (N =28¹) answered survey questions about smoking policies, while their eye movements were being measured. Two versions of the survey were compared: questions that were phrased positively in version 1 were worded negatively in version 2. The answering data show that negative questions are answered more positively. Preliminary analyses of the eye-tracking data show that respondents spend more time looking at the question (and not at the answering options) when it is worded negatively. This seems to contradict previous results, because time spent at the question can be interpreted as an interpretation process. However, respondents also switch more often between question and answering options when the question is worded negatively, so the larger time spent at the question might also reflect a mapping process. Various measures that can be distillated from the rich eye-tracking data will be presented and related to the cognitive process of question answering.