The failure of the balanced condition in the natural experiment design

Abstract

An important task in economics and finance studies involves establishing causal inference. Given the obstacles in obtaining reliable instrumental variables, such methodologies as the ordinary least squares (OLS),which is considered natural experimentation, as it takes advantage of exogenous events, have become prominent without questioning the balanced condition hypothesis. One empirical problem with such methodologies is that the treatment assignment is not random and is characterized by non-balanced covariates across the treatment and control groups. This problem is often not obvious to researchers, and they may infer causality where none may exist. By employing examples from influential journals, we demonstrate that natural experiment studies' causal inferences may change after we address all problematic imbalanced conditions. We argue that data quality will affect estimates of the treatment effect: specifically, a well-balanced dataset will require fewer matching activities, and in balancing the dataset, the estimation results after addressing the problematic imbalance become less volatile compared with those from a low-quality dataset. The nature experiment technique's popularity notwithstanding our knowledge reveals results unavailable in previous literature. Show more