Essays on public policy evaluation with the causal forest

Abstract

This thesis focuses on the use of the causal forest method to estimate individual-level treatment effects in policy evaluation applications. In particular, it focuses in on how the causal forest can be used to draw out high-level insights about treatment effect heterogeneity in a data-driven way (i.e. these do not have to be patterns for which evaluators pre-specified tests). It consists of four papers around this central theme. Paper 1 is a comprehensive review of papers in the peer-reviewed literature which have applied the causal forest. It lays out how the causal forest is used in practice and shows that the approaches to drawing out high-level insights vary and many are either relatively basic or not statistically sound. Paper 2 discusses the transparency issue both from the perspective of accountability (the traditional critical AI angle) and usability (i.e. the ability to draw out insights). It argues that the latter is much more of a concern than the former in policy evaluation settings and that new tools are needed that are better tailored to the usability problem. Paper 3 proposes an approach to solving this usability problem based on knowledge distillation which can fit powerful, yet interpretable models by leveraging the statistical characteristics of the causal forest which serendipitously makes it very well suited to being used as a 'teacher' (i.e. base model) in knowledge distillation. Chapter 4 demonstrates all these contributions by applying the causal forest and knowledge distillation to a real-world policy evaluation of a conditional cash transfer scheme in Morocco.