Darren is an expert survey methodologist who has worked in social research and survey design since 1984, and founded the Social Research Centre in 2000. From 2010 to 2015, Darren played a leading role in the introduction of dual-frame telephone surveys to Australia. More recently he has been the driving force behind the establishment of Australia’s first probability-based online panel – the Life in Australia panel. In 2014 he was awarded the Research Industry Council of Australia’s Research Effectiveness Award for Innovation and Methodology. He is a Centre Visitor at the ANU Centre for Social Research and Methods and an Adjunct Professor with the University of Queensland’s Institute for Social Science Research (ISSR). He is also a Fellow of the Australian Market and Social Research Society and has QPMR (Qualified Practicing Market Researcher) accreditation.
Co-Presenters: Stephen Prendergast
The increase costs associated with mounting high quality CATI surveys and the rapid decline in response rates for such surveys has seen many in our industry turn to non-probability online panels. The advantages of these panels include reduced costs (relative to other modes of data collection), quick turnaround times, respondent convenience, reduced social desirability bias due to the self-completion mode of collection, the ability to target hard to reach populations, multimedia functionality and computerised questionnaires. However, what about the quality?
This paper presents the Australian and international research showing that surveys administered to members of non-probability panels are considerably more biased (generally more than double the bias) and more variable than surveys administered using probabilistic sampling methods. In fact, the estimates produced from some non-probability samples are sometimes dangerously inaccurate. However, this paper doesn’t just shine a light on a problem but presents the latest Australian and international research exploring ways to reduce this bias and improve the inferences which can be made from non-probability panels. In so doing we look at techniques such as model-based design weights, sample blending and calibration. These methods are equally applicable to both market and social researchers.