Conferences

Cognitive pre-test methods began to be used in the USA and in Europe in the 80’s by survey methodologist to reduce measurement error sources in survey questionnaires. On other hand, few studies have examined the usefulness of the cognitive pre-test methods, more specifically cognitive interviewing, for gathering validity evidence which support the intended use of psychological questionnaires and educational tests.

Since 90’s Validity Theory has acquired great importance within the field of psychological and educational testing. Not only has validity a central position in the processes of evaluating tests and questionnaires, but also in the designing those (Standards for Educational and Psychological Testing, AERA, APA y NCME, 1999). “Evidence based on the response processes” is one of the sources of validity evidence. The Standards suggests resorting to the empirical and theoretical analysis of the response processes of the respondents in order to obtain evidence about the fit between the construct which test or questionnaire intend to measure and the response process actually put in to practice by examinees.

The proposal on which the present study is based is that cognitive interviewing can provide evidence “based on the response processes” of the respondents, when responding to the items of the psychological questionnaires and tests. The aim of this paper was to illustrate how cognitive interviewing can be used to get validity evidence in psychological testing. Preliminary results obtained by using cognitive interviewing to get response process evidence for a “family functioning” questionnaire will be presented as an empirical example. The pre-tested questionnaire intends to measure five components of a “family function construct”: adaptability, partnership, growth, affection and resolving capabilities. The questionnaire consists of 5 items in a rating scale format with 3 response options. The function family questionnaire is included in a health survey questionnaire. 21 cognitive interviews were performed during the cognitive pre-test. The function family questionnaire is included in a national health survey. Validity evidence from cognitive interviewing will be compared with empirical psychometric evidence of dimensionality of “family function” questionnaire. Finally, the conditions under which quantitative psychometric evidence and cognitive interviewing results could be complementary will be discussed.