The Point as a Means: Expert Judgements in the Construction of Statistical Evidence for Public Reason

Abstract

This thesis investigates how professionals negotiate the construction of statistics as evidence in public reasoning. In the information-dense and rapid communications of contemporary public life, there have been rising concerns about the damaging effects of illegitimate statistics on the functions of public reasoning. The severity of these concerns has led to a number of policy responses drawn from frameworks that tend to present illegitimate statistics as the result of failure to follow -- either accidentally, or by malicious intent -- the set standards and procedures that would produce legitimate statistics. However, previous research within the sociology of quantification shows that the construction of statistics as evidence is a social process, and that the production of legitimate statistics includes human error, bias and judgement. Policy responses framing illegitimate statistics as the product of methodological failures reflect a defensive strategy with deep roots in the statistical profession. This strategy is exemplified by the 1834 founding motto of the London Statistical Society, Aliis exterendum, 'to be threshed out by others' -- separating statistics' mechanical facts from biased interpretation and securing its position amongst the sciences. Therefore, I explore the construction of legitimatised statistical evidence, investigating how professionals reflect on their own use of expert judgement against more mechanical decision-making. Drawing on the sociology of quantification, science and technology studies and sociocultural communication theory, I investigate individual communicative actions in professional practice as part of an ecosystem conceptualisation of public reasoning. For this I develop a tripartite conceptual framework, building on Bruno Latour's concept of evidence as immutable mobiles and Alain Desrosieres's explication of attitudes towards statistical realism, to propose that the success of statistics as legitimised evidence follows from communal investment in set conventions, which grants them mobility through different contexts as well as perceived certainty. I conducted expert interviews with 49 professionals working with statistics for public reason in Australia. Interviews were semi-structured to encourage reflection around the core negotiations regarding conventions, contexts and certainty, and transcripts analysed using thematic analysis. I found participants overwhelmingly remained committed to maintaining the Aliis exterendum view, speaking to shared professional values of reflecting reality and minimal intervention even as they advocated for conflicting courses of action. I argue, extending Desrosieres's theory, that this seeming contradiction can be understood as a consequence of varying conceptualisations and thresholds of truth, or reality, across professional communities. My results highlight the importance of individuals' expert judgements in navigating these conflicts between and beyond set conventions in everyday practice, even as participants themselves often minimised this contribution in upholding the Aliis exterendum line. Further, I argue that over time the privileging of statistical evidence has led to an imbalance in these professional cultures, where adherence to reality as the internal record of the state is prioritised over adherence to external, experiential reality -- in parallel, privileging statistical expertise over subject matter expertise. This imbalance presents implications for policy, as asymmetrical responses which dismiss the role of expert judgements in legitimised statistical evidence may worsen the concerns they seek to ameliorate. Show more