A multimedia information system for the support of studies of behaviour : a dissertation presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Information Systems at Massey University

Abstract

The research presented in this thesis describes the conceptualisation of multimedia information systems to provide advanced support for the study of behaviour, the specification and implementation of such a system, called PAC, and the application of PAC in case studies in the fields of education and psychology. Researchers in disciplines such as education, psychology and sociology are concerned with the study of behaviour and record behaviour on video or audio to capture it for subsequent analysis. A variety of quantitative and qualitative analysis methods are employed across the disciplines. Despite different emphases in the study of behaviour across the disciplines, a common set of core analysis steps can be established and identified as description of behaviour, retrieval of descriptions according to common characteristics, and interpretation of these descriptions. With the advances in multimedia computing, data of multiple media formats have become very accessible even on end user computer systems. This suggests the investigation into new analysis tools for multimedia data to support the analysis of behaviour recordings. A conceptualisation for a new multimedia information system was developed which aimed at supporting the core analysis steps of description, retrieval and interpretation, providing means for the combined analysis of behaviour recordings of multiple media formats, and facilitating quantitative and qualitative interpretation techniques. This analysis support was complemented by the ideas of setting a single study into a domain context to form, over time, a knowledge basis for future studies and by the capture of study conclusions in multimedia format to allow rich, informative reporting of study results. The single most important conceptualisation for a new system was the design of the Flexible Structured Coding Language, FSCL. This coding language allows formulating rich and precise descriptions of behaviour in a very flexible way. Due to the structure of the coding language it is possible to correctly and completely retrieve the subject - verb - object relationships within the description sentences. Based on the conceptualisation, a specification for a concrete advanced multimedia information system to support the study of behaviour was developed and a system was implemented accordingly. This system, called PAC, was used in three case studies concerned with the study of behaviour in collaboration with domain experts from the disciplines of education and psychology. The case studies aimed at testing the concepts behind the implementation PAC. By conducting their analyses using PAC, the domain experts gained detailed familiarity with the conceptualisation developed in this research and confirmed the usefulness of this conceptualisation for the study of behaviour.