Data-driven framework for high-Accuracy color restoration of RGBN multispectral filter array sensors under extremely low-light conditions

Document Type


Publication Date



RGBN multispectral filter array provides a cost-effective and one-shot acquisition solution to capture well-Aligned RGB and near-infrared (NIR) images which are useful for various optical applications. However, signal responses of the R, G, B channels are inevitably distorted by the undesirable spectral crosstalk of the NIR bands, thus the captured RGB images are adversely desaturated. In this paper, we present a data-driven framework for effective spectral crosstalk compensation of RGBN multispectral filter array sensors. We set up a multispectral image acquisition system to capture RGB and NIR image pairs under various illuminations which are subsequently utilized to train a multi-Task convolutional neural network (CNN) architecture to perform simultaneous noise reduction and color restoration. Moreover, we present a technique for generating high-quality reference images and a task-specific joint loss function to facilitate the training of the proposed CNN model. Experimental results demonstrate the effectiveness of the proposed method, outperforming the state-of-The-Art color restoration solutions and achieving more accurate color restoration results for desaturated and noisy RGB images captured under extremely low-light conditions.

Publication Source (Journal or Book title)

Optics Express

First Page


Last Page


This document is currently not available here.