To Know You is to Love You: The Effect of Survey Length and Personally Knowing the Experimenter on Response Rate and Response Quality

This research investigated how the length of a survey and personally knowing the experimenter influence the response rate and response quality of voluntary web-based surveys. An experiment was designed in which the respondents were randomly allocated to either a short or a long version of a similar survey. In addition, all respondents were categorised as personally knowing the experimenter or not. By means of Fisher’s exact tests, Pearson’s chi-squared tests, Mann-Whitney U tests and a logistic regression, the effects of survey length and familiarity were analysed. Results show that, as expected, there are significant differences in response rates as a result of varying survey length and varying familiarity conditions. In addition, dropout rate, a measure of response, quality was also found to be significantly affected. A logistic regression of dropout rate has shown that in this sample, participating in a short survey and personally knowing the experimenter make it less likely to drop out of the survey. Furthermore, respondents who started this survey three days after the initial invitation are also less likely to drop out of the survey than respondents who started the survey on the day of the invitation. For the other two measures of response quality, uncertain responses and variability to questions in grids, no significant differences were observed as a result of varying survey length and varying familiarity conditions.