Book awards as indicators of literary quality: A quantitative investigation of the level of consensus between expert juries, between expert juries and consumers, and between experts over time

This study reports three distinct analyses of consensus on literary quality: consensus between expert juries, between expert juries and consumers (non-experts), and between experts over time. The study looks into the verdicts of three American juries (Pulitzer Prize for Fiction, National Book Award for Fiction and the PEN/Faulkner Award for Fiction) and three Dutch/Flemish juries (AKO Literatuurprijs, Libris Literatuurprijs and the Gouden Boekenuil) that annually award books and investigates whether they nominate/award the same books in a given year. This research recognizes the argument raised by Dekker and Popik (2014) which states that this type of interjury consensus cannot be appropriately measured by the means of reliability analysis and the Cronbach’s alpha composite. Following Dekker and Popik, different aspects of consensus are discussed, after which the measure (β) is used to assess the level of agreement. Deviations from the overall consensus by particular juries are also indicated. This is followed by an examination of the accordance between the juries’ judgments and consumer preference. The latter is indicated by bestseller lists on the one hand and online ratings on the other hand. Finally, this research employs a test-of-time analysis. It examines whether books that are honoured by the expert juries appear in anthologies too. The analyses reveal an interjury tendency towards dissensus rather than consensus, which contrasts with findings for the film industry reporting moderate to high levels of agreement. Theoretical reasons for this inter-industry difference are put forward. Moreover, results on the level of consensus between expert juries and consumers suggest that consumers also find that the nominated/awarded books are superior to general books. This does however not persuade them to buy these honoured books in great numbers. A final finding is that experts over time agree moderately with each other on literary quality