Authors
Dominic Rüfenacht, Clément Fredembach, and Sabine Süsstrunk
Click here for direct access to the paper.
Abstract
We present a method to automatically detect shadows in a fast and accurate manner by employing the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. Dark objects, which confound many shadow detection algorithms, often have much higher reflectance in the NIR. We can thus build an accurate shadow candidate map based on image pixels that are dark both in the visible and NIR representations. We further refine the shadow map by incorporating ratios of the visible to the NIR image, based on the observation that commonly encountered light sources have very distinct spectra in the NIR band. The results are validated on a new database, which contains visible/NIR images for a large variety of real-world shadow creating illuminant pairs, as well as manually labelled shadow ground truth. Both quantitative and qualitative evaluations show that our method outperforms current state-of-the-art shadow detection algorithms in terms of accuracy and computational efficiency.
Qualitative and Quantitative Results
Since we are the first to use near-infrared (NIR) images for shadow detection, we had to create a new image set that contains both the visible and the NIR image, and create the ground truth. In total, we have a labelled set of 74 visible/NIR pairs, that represent a big variety of scenes, both in complexity and predominant illuminants. We created three subsets, namely outdoor, indoor flash, and indoor uncontrolled. The latter set consists of images that were taken indoors, with a mix of sunlight and fluorescent/incandescent light. In most cases, sunlight was the predominant illuminant for this image set.
We present the results in terms of overall accuracy (Acc) and Matthews Correlation Coefficient (MCC) on the different image sets, as well as on all images.
The MCC is computed as follows in terms of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN):
By clicking on one image set (leftmost column), a web site opens with the detailed results for all images of that image set. Note that by clicking on the thumbnail, the full resolution image opens.
|
Image set |
Statistic | Our method [1] |
Tian et al. [2] | Guo et al. [3] | |||
| Avg | Std | Avg | Std | Avg | Std | ||
 |
Acc | 89.3 | 6.7 | 81.7 | 13.0 | 57.1 | 20.8 |
| MCC | 0.79 | 0.13 | 0.79 | 0.16 | 0.15 | 0.25 | |
 |
Acc | 86.4 | 12.6 | 64.5 | 23.6 | 81.7 | 10.9 |
| MCC | 0.72 | 0.13 | 0.58 | 0.18 | 0.31 | 0.21 | |
 |
Acc | 93.2 | 6.7 | 94.2 | 5.1 | 48.8 | 22.8 |
| MCC | 0.91 | 0.08 | 0.91 | 0.09 | 0.20 | 0.24 | |
 |
Acc | 89.5 | 8.5 | 80.7 | 17.6 | 60.6 | 22.6 |
| MCC | 0.80 | 0.14 | 0.77 | 0.19 | 0.19 | 0.24 | |
Downloads
Here you can download the image dataset (linear images) as well as the ground truth masks used in this work. We also provide the code that was used to reproduce the results reported in the paper.
- RAW Image dataset (105 MB): Download
– Visible and NIR Image set. - Ground Truth (3 MB): Download
– Binary Ground Truth shadow masks for the Image set above.
- Matlab code (17 MB): Download
– Code used to generate all the results on this page.
– Use this code with the Image set above to reproduce the results of the Quantitative and Qualitative Results section.
– Contains our method, as well as the original implementations from Tian et al. [2] and Guo et al. [3]. It allows you to select different methods and generates an HTML file for comparison.
– See Readme.txt for further information on how to use the code.
– Code was tested on Windows 7 with Matlab 2011b.
Important Note: If you are having problems extracting the files above, please download and install 7Zip.
References
[1] D. Rüfenacht, C. Fredembach, and S. Süsstrunk, “Automatic and accurate shadow detection using near-infrared information.” To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
[2] J. Tian, L. Zhu, and Y. Tang, “Outdoor shadow detection by combining tricolor attenuation and intensity.” EURASIP J. Adv. Sig. Proc., 2012.
[3] R. Guo, Q. Dai, and D. Hoiem, “Single-Image shadow detection and removal using paired regions.” IEEE CVPR, pages 2033-2040, 2011.
Contact
Questions and remarks about the code and our method are welcome and can be sent to [email protected].