Criterion variance in signal detection theory : the interactive effect of knowledge of results and task difficulty on binary decision tasks : a thesis presented in partial fulfilment of the requirements for the degree of Master of Arts in Psychology at Massey University, Palmerston North, New Zealand

Abstract

Within traditional Signal Detection Theory (SDT) experiments decision noise is very rarely considered, with researchers clinging to the assumption that the decision criterion has no associated variability. This assumption is incorrect. Furthermore, two factors contribute to criterion fluctuation: task difficulty and the type of knowledge of results (KR) delivered to the observer. The accepted standard in SDT experiments is to provide veridical trial-by-trial feedback (TTKRe). This type of KR may adversely affect observer performance when the decision task is difficult, as the KR may appear highly inconsistent to the observer. The present study hypothesised that providing KR relative to the optimal criterion location (TTKRi) would minimise criterion fluctuation. The present Criterion Variance Model (CVM) assumes that the decision criterion in SDT is subject to fluctuation. Two hypotheses were derived to test the model: a) contrary to the assumption of SDT, the decision criterion in a signal detection task is a variable rather than a fixed value on the decision axis, and is present within binary discrimination tasks; and b) There will be an interaction effect between the type of TTKR provided and the difficulty level of the task. Specifically, TTKRi will enable more accurate decision making than TTKRe, but only for a difficult decision task. Forty-four observers took part in a simple binary decision task, discriminating whether a presented tone was high or low in frequency (Hz). All tones were easily discriminable from each other; thus, the experiment was free from sensory noise. Task difficulty was manipulated by varying the degree of overlap between the high and low distributions, from which the high and low tones were sampled. As predicted by the CVM, performance in a difficult decision task was affected by the type of KR provided. Observers who received TTKRe performed less well than observers who received TTKRi in the more difficult version of the task. Despite mean criterion location measures across groups approaching zero – the optimal location – criterion fluctuation was evident when observer error distributions were analysed. Furthermore, the degree of criterion fluctuation was large, and was associated with the level of task difficulty. A major caveat was the lack of a no KR condition. Consequently, the degree to which observers utilised the KR could not be fully assessed. Additionally, the number of tones may have been too small, possibly encouraging observers not to use the KR provided in a consistent manner. Further research should incorporate a no KR condition and increase the number of tonal stimuli while ensuring the tones are still separated by 3 or 4 JNDs. Despite these design issues, the results highlight the potential detrimental effects of veridical KR on performance, particularly under conditions of high uncertainty.