Conferences

In this paper and based on the experience of comparative analysis with three rounds of the European Social Survey we propose the need for an ethnographic look at survey questions. We define this ethnographic look at three levels.

First, and a part from a special attention to the way questions are formulated – we all know how hard it is to find the “perfect” question, the good indicator for measuring variables – we propose an insightful and descriptive look before starting multivariate analysis and procedures. We will give several examples of answers to questions with Likert scales from European Social Survey, but also from other surveys, where very often the modal answer concentrates on the position “nor agree, nor disagree”. How far can we go in interpretation when it is this “special” position that attracts the large majority of the respondents? And is it not true that when analysing data, producing images by charting the results, or with multivariate analysis, we often forget the first descriptive information?

Secondly, and insisting on the way questions are phrased, we propose focusing our attention on a mixed way of making questions. It is very common the use of questions representing clear stereotypes. Using also neutral questions may be also more insightful. We will give some examples related to gender stereotypes showing how conclusions can be different when questions are phrased in different ways.

Finally, and based on the great advantage that represents the replication of the same questions in ESS’ three rounds, we will give some examples showing how careful we must be when concluding only from one application when comparing countries. A difference showing up in 2002, can disappear in 2004 and reappear in 2006…

In our view a mixing methods stance can also be adopting an ethnographic look at questionnaires. Mixing methods is, at least partially, the adoption of a sociological prudent standpoint, avoiding misconceptions and misinterpretations of data, on one side, and avoiding enchantment with the graphical representation that can be misleading, on the other. The output of combining a quantitative approach with ethnographic sensitivity leads to interpretations more faithful to the data and fulfilling the heuristic possibilities of mixing methods.