Model Comparison Based On Alpha

Alpha is used as a measure of model comparison as model mispricing, usually by modeling test-asset returns. However, Barillas and Shanken (2017) give an idea that test assets are irrelevant in model comparison and excluded factors are more reliable evidence. In this paper, we replicate the work of Barillas and Shanken (2017), with illustrations of non-nested model comparison. Most of the replication is quite successful, except that the result of model comparison based on the likelihood metric is different, which is acceptable because likelihood and model alpha have different norms in model comparison. As extensions, we further confirm the power of different methods of comparison for nested models, as well as with a nonsensical factor. It turns out that both excluded factors and test assets perform well for nested models, while the likelihood always gives inaccurate results. With a nonsensical factor, excluded factors fail to make a correct conclusion, but the test-asset evidence is still reliable, which gives an opposite opinion with Barillas and Shanken (2017). Another extension is about model comparison with nontraded factors, where we find that the testasset evidence turns out to be inaccurate due to the effect of mimicking portfolios.