Measuring polarization using newspaper data

This thesis develops a new daily metric to measure the level of polarization in the American public, by exploiting the discourse used in political news articles. Due to frequency discrepancies, formal comparison of the invented method to previous measurements of polarization is not possible. The metric shows that, in contrary to popular belief, the trend in polarization seems to decrease till 2009 after which the trend evolves in a cyclical pattern. The volatility of the metric declines, indicating an increasing stability over time. Having daily estimates for polarization, a vast array of potential uses to determine the causes and effects of changing polarization is unlocked. Demonstrated as an application is the “election cycle effect”, seeking to determine if the year after a presidential election yields different changes in polarization compared to the other years. Formal testing shows that for the total sample, this is the case.