Causal Machine Learning methods have lately added valuable contributions to the field of applied econometric research. Due to the wide amount of methods proposed, there is still lacking guidance on which methods to use in practise. Thus, in this study we revisit the 2016 Atlantic Causal Inference Competition by Dorie et al. (2019) and analyze the performance of methods which have not been directly compared, when estimating treatment effects of different aggregation levels in various complex simulation settings, where traditional methods would fail. Specifically, we employ Double Machine Learning (DML) to estimate the Average Treatment Effect (ATE), Generic Machine Learning for Heterogeneous Treatment Effects to estimate Group Average Treatment Effects (GATE) and Double Robust Modified Outcome Methods, Causal Forest as well as Bayesian Additive Regression Trees (BART) to estimate Individual Treatment Effects (ITE). Along the way, we retrieve all the higher-level estimates that can be derived from the more granular ones, for instance the ATE and GATE via the Causal Forest. Furthermore, we propose a new method by combining the predictive strengths of BART with the asymptotic results of the DML and the Generic framework. Consistently across all aggregation levels, the methods which model the response function most flexibly perform best. Additionally, we find that most of the Causal ML methods estimate the ATE well, however estimates become unstable with limited overlap. Furthermore, the methods are able to detect heterogeneity via automated algorithms, which we additionally demonstrate by analyzing a word experiment in the latest version of the General Social Survey (GSS).

Causal Inference, Machine Learning, Causal Forest, Bayesian Additive Regression Trees, Double Robustness, Average/Heterogeneous Treatment Effect(s)
Naghi, A.A.
Erasmus School of Economics

Wirths, C.P. (2020, May 28). Evaluating Causal Machine Learning Methods. Econometrie. Retrieved from