Why do evaluators intentionally seek process use? : exploring meaning and reason as explanation : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy at Massey University, Wellington, New Zealand - Blewden, Michael Brock (Massey University, 2014)
Process use describes the learning that occurs through stakeholder engagement in the evaluation process. It is more likely to occur when evaluators choose to pursue it through intentionally adopted practices. When it does occur, the value and utility of evaluation can be enhanced. This thesis explores reasons and seeks explanations for why evaluators are intentional in seeking process use and why they choose the practices they do to achieve it. The epistemological stance of constructivism and theoretical perspective of interpretivism are adopted. Epistemologically, process use is framed as a constructed phenomenon, interpretable only through individual experience and likely to have different meanings and manifestations in different contexts. The assumption is made that evaluators’ intent and practice regarding process use will be explained by understanding what the concept means to them and by understanding the constitutive influence of the contexts within which they practice. To address the research questions, 24 practicing evaluators in Aotearoa New Zealand were interviewed in-depth about their evaluation practice. Participants were intentionally selected by gender, ethnicity, and workplace context, and by criteria that enhanced the likelihood that they would be aware of process use. For this reason, they were more experienced evaluators. Their practice context was described through a literature review of developments in evaluation theory, through participants’ accounts of their understanding and approach to evaluation, and through participants’ descriptions of the settings they worked in. The values, beliefs, aspirations, and traditions that underpinned their practice were explored to reveal what was important to them as evaluators and what process use meant to them. How these factors explained types of process use, identified by participants as important and intentional within their recent practice, was explored. Participants’ intent and practice regarding process use was explained as an outcome of multiple converging factors. It was understandable given participants’ awareness of evaluation as a change process and their desire to address issues related to social justice, equality, and tikanga Maori. Process use was facilitated by practices that were utilization and learning focused, pragmatic and contextually responsive, and relational. These practices were explained by the social, cultural, organizational, political, and historical contexts within which the evaluators worked. Intent and practice regarding process use was also shown to simply reflect developments in contemporary evaluation practice and common practice traditions. It inevitably occurred when practice was participatory, relational, learning orientated, coconstructed, just, and fair. Participants’ intent and ability to conduct evaluation in these ways reflected their skills, credibility, and status as more experienced evaluators. Overall, the research findings show how evaluators’ intent and practice regarding process use can be traced to values, beliefs, aspirations, and traditions of importance to them. For many participants, process use was integral to their understanding of good evaluation. By identifying these explanatory relationships, this research shows that process use needs to be understood as more than just useful extra utility that is achievable through special effort or method. It inevitably occurs when the evaluator understands that they are essentially tasked with addressing relational, moral, socio-cultural, organizational, and historical concerns. Deeper examination of the role and responsibilities of the evaluator within this context of practice may be the most profitable way of further understanding the occurrence of process use.
Keywords: Evaluation research (Social action programs); Evaluation; Process use