Hierarchical Bayesian modeling of criterion variance in probabilistic categorisation as an analogue to signal detection : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Psychology at Massey University, Manawatū, New Zealand

Abstract

Variance in the decision criterion across trials induces response inconsistencies which in turn result in suboptimal performance. Criterion variability is largely thought to be driven by internal mechanisms; however, factors external to the observer may also affect response consistency. Specifically, how trial-by-trial feedback is delivered can influence the stability of the criterion across trials. This thesis examined how two types of feedback (stochastic and deterministic) influenced performance in probabilistic categorization tasks, which served as analogues to the orthodox detection task. Critically, feedback that is related to the statistical properties of the stimulus distributions (i.e., feedback for which event had occurred) results in lowered performance when compared to feedback that is provided deterministically (i.e., relative to the optimal cut-off). This result held more consistently in conditions where there was greater (probabilistic) confusability among the stimuli. The effects upon the criterion were also examined by comparing dynamic signal detection models that allowed for trial-by-trial criterion shifts. Hierarchical Bayesian modeling was implemented to fit the dynamic criterion models, allowing for model comparisons to proceed using Bayes Factors. It was found that simple errorcorrecting models predicted the data less well than models that included shifts after correct decisions. However, criterion shifts after correct decisions can be better described by a weighted moving average criterion which shifts toward the current stimulus, rather than away. This finding arose through the explicit modeling of the stimulus magnitudes on each trial. Finally, a model was contrived that both allowed stimulus magnitudes to influence criterion shifts and make the effects of feedback more overt. The model suggests that the way feedback information is stored over trials drives shifts in the criterion, and that feedback will influence how storage is facilitated. However, the model could not completely describe the effects of feedback nor fit the empirical data as well as already established dynamic criterion models.