Conferences

Cognitive interviewing has gained importance as a pretesting practice for surveys but it is still unclear how to evaluate this practice. How do we know that pretesting makes survey questions better? What are the criteria to be used to determine the effectiveness of specific qualitative pretesting approaches? In this paper we would like to discuss qualitative questionnaire pretesting from the viewpoint of how qualitative research has been done outside this field. Here we focus on the data collection part of pretesting.

It is acknowledged that cognitive interviewing is a kind of qualitative research, but it brings some specifics to it in at least three ways. 1) Within the interpretive tradition, qualitative researchers strive to find out how people give meaning to everyday life, while in cognitive interviewing relevance is strictly related to the questionnaire. 2) In qualitative research emerging, flexible methods are used, in order to get to know the social worlds of the researched. In cognitive interviewing data collection is in any case structured by the survey questions. 3) The findings of qualitative research consist of detailed descriptions and should lead up to a better theoretical understanding of the phenomenon under study. The aim of cognitive interviewing is clearly to improve the questionnaire.

In pretesting practice quality is addressed from two different sides. One side pleads for further standardization of the interview protocol and the other side seeks the solution in training of interviewers so that they are competent for the job and in the transparency of the interviewing process. In current qualitative research practice, quality criteria crystallize even in the form of checklists to assess all the steps taken in the research process. What can we learn from these efforts to preserve quality in qualitative data collection more generally? We will have a critical look at the available criteria and suggest possible adjustments to fit the specific character of cognitive interviewing as a form of qualitative research.