Why do evaluators intentionally seek process use? : exploring meaning and reason as explanation : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy at Massey University, Wellington, New Zealand
Process use describes the learning that occurs through stakeholder engagement in the
evaluation process. It is more likely to occur when evaluators choose to pursue it through
intentionally adopted practices. When it does occur, the value and utility of evaluation can
be enhanced. This thesis explores reasons and seeks explanations for why evaluators are
intentional in seeking process use and why they choose the practices they do to achieve it.
The epistemological stance of constructivism and theoretical perspective of
interpretivism are adopted. Epistemologically, process use is framed as a constructed
phenomenon, interpretable only through individual experience and likely to have different
meanings and manifestations in different contexts. The assumption is made that evaluators’
intent and practice regarding process use will be explained by understanding what the
concept means to them and by understanding the constitutive influence of the contexts
within which they practice.
To address the research questions, 24 practicing evaluators in Aotearoa New Zealand
were interviewed in-depth about their evaluation practice. Participants were intentionally
selected by gender, ethnicity, and workplace context, and by criteria that enhanced the
likelihood that they would be aware of process use. For this reason, they were more
experienced evaluators. Their practice context was described through a literature review of
developments in evaluation theory, through participants’ accounts of their understanding
and approach to evaluation, and through participants’ descriptions of the settings they
worked in. The values, beliefs, aspirations, and traditions that underpinned their practice
were explored to reveal what was important to them as evaluators and what process use
meant to them. How these factors explained types of process use, identified by participants
as important and intentional within their recent practice, was explored.
Participants’ intent and practice regarding process use was explained as an outcome of
multiple converging factors. It was understandable given participants’ awareness of
evaluation as a change process and their desire to address issues related to social justice,
equality, and tikanga Maori. Process use was facilitated by practices that were utilization
and learning focused, pragmatic and contextually responsive, and relational. These practices
were explained by the social, cultural, organizational, political, and historical contexts
within which the evaluators worked.
Intent and practice regarding process use was also shown to simply reflect
developments in contemporary evaluation practice and common practice traditions. It
inevitably occurred when practice was participatory, relational, learning orientated, coconstructed,
just, and fair. Participants’ intent and ability to conduct evaluation in these
ways reflected their skills, credibility, and status as more experienced evaluators.
Overall, the research findings show how evaluators’ intent and practice regarding
process use can be traced to values, beliefs, aspirations, and traditions of importance to
them. For many participants, process use was integral to their understanding of good
evaluation. By identifying these explanatory relationships, this research shows that process
use needs to be understood as more than just useful extra utility that is achievable through
special effort or method. It inevitably occurs when the evaluator understands that they are
essentially tasked with addressing relational, moral, socio-cultural, organizational, and
historical concerns. Deeper examination of the role and responsibilities of the evaluator
within this context of practice may be the most profitable way of further understanding the
occurrence of process use.