Feasibility of dashboard GUI design and optimization from machine learning : a thesis presented in partial fulfilment of the requirements for the degree of Masters in Mechatronics Engineering, School of Food and Advanced Technologies, Massey University, Palmerston North, New Zealand

Abstract

Web dashboard Graphical User Interfaces (GUI’s) are becoming ever more important for today’s big data requirements. A number of industries from Internet of Things, to finance, to project management and everything in-between are producing enormous amounts of data and the need to display this data for users and internal employees in a useful way is incredibly necessary. This thesis attempts to answer the question; Is it feasible to design and optimize dashboard graphic user interfaces with machine learning? To explore this question, we devised two experiments. One using full dashboard images, and the other using dashboard cropped components. Two datasets were built using images pulled off the internet and processed using four software scripts built for the purpose of this project. These included an image downloader, image duplicate detector, and a cropping tool designed to allow the user to crop sub-images from a single image rapidly while assigning class labels to the main, and cropped images. This resulted in two data-sets of 1024 dashboard images of size 512x512 pixels, and 5133 dashboard component images of size 256x256 pixels. We labelled these data-set images as one of eight class labels of different dashboard types. These types were IoT, agriculture, finance, analytics, miscellaneous, fitness, project management, and social. Following this, a machine learning architecture was chosen based on an evaluation using a set of weighted criteria. The results of this evaluation settling on StyleGAN2-ADA developed by NVIDIA corporation. Chosen hardware was a purpose-built machine learning computer with two NVIDIA a-40 Graphics Processing Units (GPU’s) which was able to perform at speed with the high memory and processing requirements of this architecture. Initial training runs showed a mixing of colours and styles through all the generated images which was fixed by altering the architecture structure slightly. Subsequent parameter tuning experiments found that in built image augmentation methods of pixel blitting, geometry and colour transforms, along with a model parameter gamma value of 50 were useful in achieving better training results. The final two experiments using the two datasets that we created showed symptoms of model failure at a later stage in the training where the generated images were reduced in quality and almost identical. Image generations taken from a point in the training prior to this occurring showed patterns between the eight different dashboard class types. Although the image quality was not particularly good as training did not fully complete, we made a number of observations for full dashboard images. Bar and line graphs were extremely prevalent in most dashboards and found in similar positions (middle/top left), the algorithm favoured side menu dashboard format, and text was not readable in the generated images. Regarding dashboard components a number of classes suffered from low data numbers and generated purely text-based images, while others mirrored the components shown in the full dashboard image tests.