The Online Panels Benchmarking Study: a Total Survey Error comparison of fndings from probability-based surveys and nonprobability online panel surveys in Australia

Abstract

The pervasiveness of the internet has led online research, and particularly online research undertaken via nonprobability online panels, to become the dominant mode of sampling and data collection used by the Australian market and social research industry. There are broad-based concerns that the rapid increase in the use of nonprobability online panels in Australia has not been accompanied by an informed debate about the advantages and disadvantages of probability and nonprobability surveys. The 2015 Australian online Panels Benchmarking Study was undertaken to inform this debate, and report on the fndings from a single national questionnaire administered across three different probability samples and fve different nonprobability online panels. This study enables us to investigate whether Australian surveys using probability sampling methods produce results different from Australian online surveys relying on nonprobability sampling methods, where accuracy is measured relative to independent population benchmarks. In doing so, we build on similar international research in this area, and discuss our fndings as they relate to coverage error, nonresponse error, adjustment error and measurement error. Show more