Verdicts or Inventions? Interpreting Results from Randomized Controlled Experiments in Criminology

Abstract

The social benefits of randomized controlled trials (RCTs) would be enhanced by general recognition of three problems of their interpretation and a redefinition of their mission in relation to program development and evaluation. One problem is that of "forest versus trees," or the sampling relationship between each test of a hypothesis and the conclusions drawn from all such tests taken together. A second problem is interpreting RCTs as testing theory or policy when they cannot achieve a high correlation between the treatments assigned and treatments actually applied in each case. The third problem is what works for whom, or whether identical treatments cause different effects, on average, for different kinds of people, groups, situations, or other units of analysis that were different at the point of random assignment. Confronting these three problems suggests that RCTs should not only seek verdicts about what works but also should seek better inventions of crime prevention programs for further testing.