Conferences

The strengths and weaknesses of web surveys have been widely described in the literature. Of particular interest is the question of the quality and reliability of web surveys for scientific use – more concretely, to which degree can the obtained results be generalised for the whole population? As respondents are not selected at random and the target population rather forms a convenience than a probability sample, particularly volunteer web surveys are subject to selection bias.

To deal with this problem, weighting adjustment, like post-stratification and propensity score weighing, have been seen as a possible solution to reduce the biases in web surveys. However, particularly the post-stratification weighting, aiming to adjust for demographic differences between the sample and the population under consideration, seems to be limited. As some variables of interest often do not show a sufficiently strong relationship with the demographic weighting variables, this method can correct for proportionality but not necessarily for representativeness. As a consequence, another weighting technique called Propensity Score Adjustment (PSA) has been suggested as an alternative for statistically surmounting inherent problems in web survey data. It aims to correct for differences caused by the varying inclinations of individuals to participate in web surveys. It adjusts for selection bias due to observed covariates which are demographic as well as ‘webographic’(lifestyle) variables. These variables measure general attitudes or behaviour that are hypothesised to differ between the online and the general population. In the scientific community, however, this method has traditionally not been applied in the field of surveys, and there has been a minimal amount of evidence for its applicability and performance, and the implications are not conclusive. Moreover, the statistical theory behind this approach is not well developed and the effectiveness and implications, particularly for survey methodology, still need to be better studied

Against this background, the present paper attempts to explore various statistical weighting procedures for volunteer web surveys and evaluate their effectiveness in adjusting biases arising from non-randomised sample selection. In order to achieve the goal, three methods are compared in more detail: post-stratification, correlations and finally PSA. A first essential step for exploring the existing selection bias within the existing data-sets will be a detailed bias description. It is particularly needed for the application of PSA searching for the variables that are going to be included in the calculation of the Propensity Score weights. The efficiency of different weights will then be tested by comparing unweighted and weighted results from the German, Dutch and Spanish sample of the Wage-indicator Survey 2006 with those that could be found using data from the German Socio-economic panel, the OSA Labour Supply Panel and the Spanish Structure Earnings Survey for the same year. In the framework of these examinations, analytical graphics and formal tests of significance will be used. Furthermore, the sensitivity of the results, particularly to changes in the specification of the propensity score, will also be addressed.