"Kind of like outside the box but kinda in the box" : using cup of tea words to describe experiences of Māori evaluators with Māori communities in externally commissioned evaluations : a thesis presented in fulfilment of the requirements for the degree of Masters in Māori Arts at Massey University, Manawatu, New Zealand
He ringa miti tai heke - Hands that licks up the ebbing tide, Whanganui n.d. This Whanganui whakatauki describes how people who live on the river within the tidal reaches are accustomed to navigating and paddling the challenges the tides present therefore better able to manage their canoes than the people of the interior. For this study it describes the relevance of Kaupapa Māori when evaluating with Māori communities. The purpose of this research is to gather the experiences of other Māori evaluators working with Māori communities on externally commissioned evaluations to compare and gain an understanding of their realities. This research provides the groundwork for a Master of Arts (Māori) thesis. Research activities included a literature review, ethics application, key informant interviews, a thematic analysis and dissemination opportunities. This study examines the experiences of externally funded Māori evaluators working within Māori communities. Data was collected using Kaupapa Māori research methods informed by wider qualitative approaches from five wāhine Māori researchers and evaluators. These semi-structured interviews were undertaken kanohi ki te kanohi in single interview format. All interviews were transcribed and sent to study participants for review. A thematic analysis framework (Braun & Clarke, 2013) was used to analyse the findings and elicit themes and sub-themes from the data. Three key overarching themes emerged from the data: positioning of Kaupapa Māori in evaluation; managing multiple expectations of evaluators, communities and funders and the tensions of being both an insider and outsider as a Māori evaluator. For the discussion section an allegory of seat roles in a waka was used to describe and discuss this data. My findings show that Māori evaluators clearly understand what is required when evaluating with Māori communities and have high expectations of themselves to deliver quality evaluations that meets community needs. This is a complex space conflicted by the expectations of the communities they belong or are affiliated to, as well as obligations to evaluation commissioners. This research shows that Māori evaluators will go above what is regarded as conventional evaluation practice in order to address the expectations of themselves and other stakeholders. The study shows how Māori evaluators use approaches that are underpinned by Kaupapa Māori principles combined with western evaluation practices to provide evaluations that are relevant to the communities they work with. This includes using approaches that are collaborative, participatory and transformative to achieve programme and evaluation outcomes. Consequently, built into every evaluation design is the development of evaluation capacity through capability building and knowledge transfer. My findings also highlight the tensions and triumphs of being both an insider and outsider for Māori evaluators on such evaluations. Largely, these tensions have arisen as a result of managing the expectations of everyone involved. This includes the dual responsibility resulting from commonality of culture, shared whakapapa, or belonging to the community they are evaluating with. In addition, there is a responsibility to manage this within in the confines of contractual parameters such as evaluation outputs, timeframes and budgetary constraints. Māori evaluators have become adept at addressing these needs through evaluation design that works alongside those communities using culturally adapted methods that resonate with them. In addition, evaluators have provided extra resource to support this in the form of FTE capacity, upskilling and funding. The evaluators in my research described their individual journeys of becoming an evaluator and how that impacted on their current evaluation knowledge and practice. It points to their personal successes and those of the communities they supported as well as contributing to the wider evaluation space.