An affect-sensitive intelligent tutoring system with an animated pedagogical agent that adapts to student emotion like a human tutor : a thesis presented in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science at Massey University, Albany, New Zealand - Alexander, Samuel Thomas Vaughan (Massey University, 2007)
One of the established strengths of human tutors is their ability to recognise and adapt to the emotions of students. This is a skill that has traditionally been lacking from Intelligent Tutoring Systems (ITSs); despite their ability to intelligently model and adapt to aspects of the student’s cognitive state, ITSs are generally completely unable to detect or adapt to aspects of the student’s affective state. In response to this shortcoming, this thesis explores the pioneering development of an emotion-sensitive ITS. With the empathy of effective human tutors as our blueprint, we investigate how an artificial tutor should adapt to the affective state of students, and develop an original affective tutoring strategies method. As a validation of the feasibility of an emotion-sensitive tutoring system, we implement and test our method in a functional Affective Tutoring System (ATS) for counting and addition, Easy with Eve, featuring an empathetic animated pedagogical agent, Eve. Eve is able to detect student affect using an in-house real time facial expression analysis system. To inform the system’s adaptation to student affect, the novel method for student modelling and emotion-sensitive tutoring strategies has been developed using a fuzzy, case-based reasoning approach. This approach is used to mine data about human tutor adaptations to student affect that was generated by an observational study of human tutors that was carried out in a local primary school. To test the impact of emotion detection and the presence of the animated agent, four different versions of the ATS were tested in local primary schools with a total of 59 participants. The findings from the study indicate that adding the detection of facial expressions to the student model did not improve student short-term performance, but there was mixed evidence that the presence of the animated agent Eve may cause students to perceive the system slightly more positively (a persona effect). This effect was marginally greater when the animated agent was enabled to detect and adapt to the affective state of students, which tentatively shows that emotion detection in an ATS may have a positive effect on student motivation.
Keywords: Computer-assisted instruction; Intelligent tutoring systems; Fields of Research::280000 Information, Computing and Communication Sciences::280200 Artificial Intelligence and Signal and Image Processing::280210 Simulation and modelling