Conferences

Cognitive interviewing has been an accepted part of the standard practice of pretesting questionnaires in government, non-profit, and commercial settings for almost two decades. Examples of its use (Lessler, Tourangeau, and Salter, 1989; DeMaio and Rothgeb, 1996; Pan et al, 2007) and guidelines for its use (Willis, 2005) appear in the survey literature. Yet the lack of formal evaluation of the method has subjected it to healthy doses of skepticism by members of the academic community (Tourangeau, Rips, and Rasinski, 2000; Conrad and Blair, 2004; Blair, Conrad, Ackermann, and Claxton, 2006). The time and resource constraints of the applied setting do not lend themselves to conducting the type of systematic evaluations that provide
advances in the methodological literature. Perhaps, however, other methods of evaluation, such as using benchmarking of cognitively-improved questionnaires against known distributions, can be used to validate the results of cognitive interviews. In this paper, two examples of pretesting
conducted by the U.S. Census Bureau are presented along with tentative validation measures. The examples include testing the Identity Theft Supplement for the National Crime Victimization Survey and testing materials that will be mailed to respondents in the 2010 Census.