'Testing for Analysts' Bias in Crime Prevention Experiments: Can We Accept Eisner's One-Tailed Test?
Date
2009
Authors
Sherman, Lawrence
Strang, Heather
Journal Title
Journal ISSN
Volume Title
Publisher
Springer
Abstract
Eisner (Journal of Experimental Criminology, this issue, 2009) suggests that developer-led evaluations often make programs look better than independent evaluations do because the former suffer systematic biases in favor of prevention success. Yet, his proposed remedies suffer their own systematic bias, constituting a 'one-tailed' test of bias in only one direction. In this response we suggest that a more objective assessment of 'analysts' effects' requires a 'two-tailed' test of bias, in which reviewers would measure indications of bias both for and against success in evaluations reported by both developers and independent evaluators. After exploring the full complexity of the distinction between developers and evaluators, we report on one case in which independent evaluations were more favorable than those of developers. We then suggest possible indicators of analysts' biases against finding success that may characterize the work of developers who "bend over backwards" to find harm in their programs, and of independent evaluators who may seek to "get a scalp" of a developer or a program.
Description
Keywords
Keywords: Bias; Developers; Evaluators; Experiments
Citation
Collections
Source
Journal of Experimental Criminology
Type
Journal article
Book Title
Entity type
Access Statement
License Rights
Restricted until
2037-12-31
Downloads
File
Description