The development of a visual language for image processing applications : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science at Massey University, Palmerston North, New Zealand
The research described in this thesis is based on the hypothesis that computer support for the heuristic development of image processing algorithms can be improved by the provision of a human-computer interface that is suited to the task. Current interfaces are largely text based and are not specifically designed to provide this support. It is suggested that an interface incorporating aspects of menu-based, direct manipulation, and visual languages, can provide the necessary support. The research of this thesis begins with an analysis of the task of image processing algorithm development. It is found that in development, algorithms are more appropriately viewed as data-oriented networks of imaging operations than as process-oriented lists. The representation of algorithms in most current interfaces, particularly in text based systems, do not clearly convey the multi-threaded data paths in an algorithm. However, a data-oriented representation expresses such parallel paths clearly and naturally. The second finding of the analysis is that human designers employ a set of problem solving strategies or heuristics in the interactive development of algorithms. These strategies include the top-down decomposition of the imaging task, the identification and focus of critical sub-goals, the progressive refinement of an algorithm, and the modification of existing algorithms. These heuristics are used implicitly in the development of algorithms, but the ease with which they are used in text based interfaces is restricted by the lack of appropriate interactive facilities. An evaluation of interface techniques suggests that an interface that combines aspects of menu-based, direct manipulation, and visual languages, can support the required interaction for the heuristic development of algorithms. The required data-flow view can be provided by an iconic data flow language. Such a representation is highly visible, and can be interpreted by a user at a glance. Quick and convenient specification and editing of a data flow network can be performed via direct manipulation interaction facilities. The search for suitable operations can be facilitated by menu systems. On the basis of the arguments for the adoption of a data-flow representation of algorithms, a problem solving approach to algorithm development, and highly interactive facilities, a software package, called OpShop, has been implemented. Examples which compare OpShop to text based systems show that four major tasks involved in algorithm development are better supported with the new interface. These tasks are the visualisation of multi-threaded data paths, the interactive experimentation with algorithm parameters, the modification of algorithm topology, and the comparison of alternative algorithms. In these examples, the OpShop software represents the tangible outcome of the design for an interface that specifically supports the heuristic development of image processing algorithm.