Evaluation in tertiary education : an investigation of the effectiveness of SGID (Small Group Instructional Diagnosis) as an evaluation technique to improve the quality of educational provision : being a thesis submitted in partial fulfilment of the requirements for the degree of Master of Educational Administration at Massey University

Abstract

Quality today plays an increasingly prominent role in higher education world-wide and educational institutions are keen to acquire knowledge and strategies to address quality issues. This major focus on quality raises questions about how to evaluate quality to discover what are the quality issues in higher education and how to improve them. This thesis is an evaluation of an evaluation method called, Small Group Instructional Diagnosis (SGID). The aim of this research is to investigate the effectiveness of SGID as an evaluation technique for improving the quality of educational provision. To achieve this aim SGID processes were carried out in an Institute of Technology involving eleven classes (204 students and eleven lecturers) across four faculties and covered degree, diploma and certificate courses. By evaluating the data obtained from these SGID processes this thesis attempts to evaluate the effectiveness of SGID to contribute to the improvement of educational provision by identifying the quality issues as perceived by students and by evaluating the improvement in the quality of educational provision, defined here as, an increased level of change in the student perspectives on those areas identified by students as needing improvement. Both qualitative and quantitative evaluation techniques were used in this study. Qualitative techniques were used to identify the quality issues as perceived by the students, and largely quantitative techniques were used to monitor whether change took place. Interviews were used to gather the perceptions of change by the lecturers to the issues identified by the students and to elicit comments as to how they perceived the SGID process. The results revealed that when given the opportunity, students identify quality issues pertaining not only to classroom practice but significantly to departmental and institutional practices as well. This process highlighted the wide ranging concerns of the students for their total learning milieu and revealed the restrictive nature of most traditional evaluation techniques Student feedback also indicated that students do not see themselves as principally responsible for their learning but rather see responsibility lying with lecturers and departments. Lecturer interview comments indicated that they found the SGID method to be a powerful method for identifying quality issues about the total student learning experience but caution that it not be over-used or used exclusively. It is concluded that the majority of course evaluation methods currently in use focus on instructional practice only and therefore fail to capture adequately student feedback on departmental and institutional performance. The need to develop autonomous and self-directed learners by helping students to take increased responsibility for their own learning is identified as a challenge for lecturers. The results on change indicated that greater change took place for those issues under the control of the lecturers and students, with a decreasing amount of change for department issues and little change with institutional issues. This may be because such data is not usually the subject of evaluations so no feedback communication channels have been established. There may also be an element of inadequate time encompassed by this study to allow for change to take place at departmental and institutional level. When open ended approaches like SGID are used the educational provider can understand and capture students' points of view without predetermining those views by prior selection of categories and/or questions. The results from this study are a reflection of reality as perceived by students and have the potential to assist both the spread of practices perceived as helpful to learning and improvement in those practices perceived as hindering learning.