Introduction

In this paper, we have designed a semantic image segmentation scheme based on a state-of-the-art segmentation framework (CRF) that employs color and Near-infrared (NIR) information captured with a standard camera and tested on a new manually segmented image database.

 

On this dataset, the best results with visible only information were obtained by using COLrgb + SIFTl as descriptors for the recognition part and a regularization using the visible image (visible baseline). The best results, when NIR images are also available, were obtained by COLrgbn + SIFTn used for the recognition part and the full 4-dimensional image (RGB+NIR) for regularization (best strategy). The results (in the paper) show 2% improvement of segmentation accuracy for our best strategy over the visible baseline.

 

In this supplementary material, we further provide qualitative results concerning these two methods (visible baseline and best strategy), along with the images and the ground truth segmentation.  As can be explained by NIR characteristic, we noticed that integrating NIR as additional information with conventional RGB images achieves higher detection rate, i.e. under-detection and miss-detection errors are lower (see visual examples here). Our best strategy also shows lower false-detection rate compared to the visible only baseline (click for visual examples here).

 

Finally, for some images, we observed that borders are more precisely detected when incorporating NIR information. This can be explained by the material dependency of NIR response that may reduce wrong edges due to clutter, or may result in more contrasted edges between classes. A few examples illustrating this can be found here.