Predicitions in the Surprisingly Popular Method

Several tested methods that aggregate crowd wisdom in a way superior to majority voting require that respondents make predictions about the answers of other respondents. These predictions are required to be accurate on average, but no research has yet been done into how people tend to form them. This thesis researches whether respondents base their predictions on different possible worlds or not, one world in which the answer to a binary question is correct and one where it is false. It also tests whether researchers can influence this process by having respondents report their confidence in advance. In an experiment with three treatments and two stages, respondents, mostly students, give their main predictions in one stage and their separate predictions for each possible world in the other. A comparison between the two stages then reveals which worlds respondents base their predictions on. Roughly one-third of respondents seem to generally base their predictions on both possible worlds, another third tends to base their predictions on the most likely world, and the last third does not form different predictions for the different worlds. Respondents who had to report their confidence in advance were not more likely than other respondents to fall into any of these three categories. Several factors contributed to respondents giving internally inconsistent predictions, difficulty of the tasks and low incentives being the most important.