Does truth pay? : investigating the effectiveness of the Bayesian truth serum with an interim payment : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Psychology at Massey University, Albany, New Zealand

Abstract

Self-report data is vital in psychological research, but biases like careless and socially desirable responding (SDR) can compromise its validity. SDR is particularly problematic for sensitive questions, which are common in psychological studies and prone to distortion (Krumpal, 2013). While various methods have been developed to mitigate response biases, they each have limitations. Many proactive approaches rely on intrinsic motivation, which may be insufficient when questions are sensitive, cognitively demanding or socially undesirable (Crowne & Marlowe, 1964). The Bayesian Truth Serum (BTS; Prelec, 2004) offers an incentive-compatible alternative, aligning extrinsic motivation with truthfulness. It encourages considered and honest responses to subjective questions by scoring the truthfulness of responses and rewarding higher scores with a bonus payment. This approach leverages a Bayesian interpretation of population distributions and ties incentives to how ‘surprisingly common’ participants’ responses are. However, prior research has yielded mixed results, highlighting challenges such as participant scepticism and uncertainty about incentives. This study evaluated the effectiveness of the BTS in mitigating SDR in sensitive questions. It tested whether an interim payment could enhance the efficacy of the BTS by increasing trust. In a between-subject experimental survey, 877 participants were randomly assigned to one of three conditions: BTS, BTS with Interim Payment (BTS+IP) and Regular Incentive (RI). Contrary to the hypotheses, participants in the BTS conditions displayed lower agreement with socially undesirable statements compared to the RI condition. The interim payment did not significantly enhance the BTS’s effectiveness. Instead, response patterns diverged from the mechanism’s intended effects, raising concerns about its robustness. As the second registered report to challenge its efficacy, this study casts serious doubt on the BTS as a reliable tool for mitigating SDR and improving the validity of self-report data in psychological research. Beyond its empirical contributions, this thesis also reflects on the challenges and benefits of adopting the registered report format. Situated within meta-psychology, the reflection explores key moments of academic and personal growth. It may serve as a resource for students and supervisors considering the registered report approach in postgraduate research.