The practice of evaluative reasoning in the Aotearoa New Zealand public sector : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy at Massey University, Wellington, New Zealand

Abstract

This study argues that sound evaluative reasoning, defined as “the systematic means for arriving at evaluative conclusions . . . the principles that support inferences drawn by evaluators” (Fournier, 1995, p.1), is an essential element of evaluation quality. As such, evaluative reasoning is a lens through which to consider how to improve the quality of evaluations undertaken or commissioned by the Aotearoa New Zealand public sector. The argument is grounded in the theory of evaluation derived from western philosophy, specifically, informal logic. This theory underpins the conceptualisation and design of this study examining how evaluative reasoning is understood and practised by professionals who undertake public sector evaluation in Aotearoa New Zealand. A multiple method research design is used to generate diverse understandings of the topic and offer opportunities for abductive thinking. The methods used are Q methodology, metaevaluation, and key informant interviews with local and international evaluation experts. The findings from this study point to three ways in which evaluative reasoning has an impact on the quality of evaluation. It increases the robustness of the reasoning chain from value claim to evaluative conclusion/judgment; underpins the professional competencies required of evaluation practitioners; and reinforces the ethical dimensions of evaluation practice in a public sector context. Lastly, two abductively-derived conjectures point evaluators toward diverse ways of knowing in their reasoning from evaluative claim to evaluative conclusion/judgment. Amplifying the work of previous theorists, it is suggested that expert intuition and abductive inference provide further paths of evaluative knowing in addition to inductive logic and probative inference.