RGB-NIR side window demosaicing : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Mechatronics, Massey University, Manawatu, New Zealand. EMBARGOED until 29th March 2026.

Abstract

The review of literature and the current state-of-the-art reveals that the inclusion of near-infrared (NIR) data is extremely useful in agricultural robotics applications. The most suitable method for collecting this data uses a multispectral filter array (MSFA) data collection method. Specifically, the red, green, blue, and near-infrared (RGB-NIR) MSFA proposed by Monno et al. in [1] (see Figure 31). A clear gap in research has been identified in demosaicing methods for this MSFA. The only existing method is a proprietary adaptation of residual demosaicing also proposed by Monno et al. [1]. To further the commercial viability of RGB-NIR MSFAs in agricultural applications, a new demosaicing method is proposed. Called Side Window Demosaicing (SWD), it is an adaptation of the recent side window filtering (SWF) technique [2]. To test this method a demosaicing experiment was designed. The experiment followed the accepted method for assessing demosaicing algorithms; by taking a ground truth image, artificially mosaicing it, and then applying the demosaicing technique to estimate the original image. Three datasets of ground truth images were used: - The TokyoTech hyperspectral dataset was transformed into RGB-NIR multispectral images [1]. - The M-SIFT dataset [3]. - An RGB-NIR dataset collected using a prism camera. The algorithm’s efficacy was measured using MPSNR and SSIM then contrasted against existing literature. It was found to be worse than other state-of-the-art methods that demosaic similar density channels by approximately 20dB MPSNR and 0.01 SSIM. However, the comparison to existing literature is difficult, as almost all literature uses visible range MSFAs, and the inclusion of an NIR channel presents unique spectral correlation challenges. When used in the agricultural context of the problem, the performance of the algorithm improves by up to 11dB. This, coupled with the simplicity and flexibility of the algorithm relative to existing literature, makes SWD an attractively simple first principles approach for data collection in robotic applications.