Measuring the Unmeasurable: Evaluations of Complex Programs in the Northern Territory

Abstract

This presentation will offer a critique of the kinds of indicators used to evaluate outcomes of different types of programs and asks the question: 'Are these indicators a true reflection of outcomes?' The critique is based on a number of evaluations carried out by members of the Charles Darwin University's Social Partnerships in Learning consortium over recent years. These evaluations have included a range of social programs funded by several government and non-government organisations. Many of the programs were designed to address health and well-being, community safety, family function, education and community capacity issues. What we sometimes found is that because of the complexity of the programs, the achievement of commonly reported aims (in terms of increased school attendance, reduced violent offences, fewer domestic violence reports and fewer child protection notifications) was nigh on impossible. When this became apparent, we looked for other indicators that more closely reflected the outcomes of programs. These included measures of changed community perceptions, increased family function and improved parent-child engagement. The refrain we frequently hear in relation to these indicators is 'yeah, but you can't measure that'. This presentation will respond to this criticism with some practical examples of tools that we have tried out to measure the unmeasurable. Show more