×

Multi-cue illumination estimation via a tree-structured group joint sparse representation. (English) Zbl 1398.68587

Summary: A multi-cue illumination estimation method based on tree-structured group joint sparse representation is proposed. Tests show that the proposed method works better than existing methods, most of which are based on using only a single cue type, for example, a binarized color histogram or simple image statistic such as the mean RGB. Most existing illumination estimation methods make their estimates using only one of three kinds of cues. They differ in which cue type they use, but the chosen cue is either based on (1) properties of the low-level RGB color distribution, (2) mid-level initial illuminant estimates provided by subordinate methods, or (3) high-level knowledge of scene content (e.g., indoor versus outdoor scene). The proposed multi-cue method combines the information provided by cues of all three of these types within the framework of a tree-structured group joint sparse representation (TGJSR). In TGJSR, the training data is grouped into a tree of subgroups. A test image under an unknown illuminant has its features reconstructed in terms of a joint sparse representation model derived from the grouped training data. The test image’s illumination is then estimated based on the weights involved in the joint sparse representation model. As a general framework, the proposed TGJSR framework can also easily be extended to incorporate any new features or cues that might be discovered in the future for illumination estimation.

MSC:

68T45 Machine vision and scene understanding
62H35 Image analysis in multivariate analysis
68U10 Computing methodologies for image processing
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Barnard, K; Cardei, V; Funt, B, A comparison of computational color constancy algorithms-part 1: methodology and experiments with synthesized data, IEEE Transactions on Image Processing, 11, 972-983, (2002) · doi:10.1109/TIP.2002.802531
[2] Barnard, K; Martin, L; Coath, A; Funt, B, Comparison of computational color constancy algorithms-part 2: experiments with image data, IEEE Transactions on Image Processing, 11, 985-996, (2002) · doi:10.1109/TIP.2002.802529
[3] Beigpour, S; Riess, C; Weijer, JV; Angelopoulou, E, Multi-illuminant estimation with conditional random fields, IEEE Transactions on Image Processing, 23, 83-96, (2014) · Zbl 1374.94041 · doi:10.1109/TIP.2013.2286327
[4] Bengio, S., Pereira, F., Singer, Y., Strelow, D. (2009). Group sparse coding. In: Proceedings of neural information processing systems conference (pp. 1-8).
[5] Bianco, S; Gasparini, F; Schettini, R, Consensus-based framework for illuminant chromaticity estimation, Journal of Electronic Imaging, 17, 023013, (2008) · doi:10.1117/1.2921013
[6] Bianco, S; Ciocca, G; Cusano, C; Schettini, R, Improving color constancy using indoor-outdoor image classification, IEEE Transactions on Image Processing, 17, 2381-2392, (2009) · Zbl 1373.62323 · doi:10.1109/TIP.2008.2006661
[7] Bianco, S; Ciocca, G; Cusano, C; Schettini, R, Automatic color constancy algorithm selection and combination, Pattern Recognition, 43, 695-705, (2009) · Zbl 1187.68431 · doi:10.1016/j.patcog.2009.08.007
[8] Brainard, DH; Freeman, WT, Bayesian color constancy, Journal of the Optical Society of America A, 14, 1393-1411, (1997) · doi:10.1364/JOSAA.14.001393
[9] Buchsbaum, G, A spatial processor model for object colour perception, Journal of the Franklin Institute, 310, 1-26, (1980) · doi:10.1016/0016-0032(80)90058-7
[10] Cardei, V., & Funt, B. (1999). Committee-based color constancy. In: Proceedings of IS&T/SID color imaging conference (CIC) (pp. 311-313).
[11] Cardei, V; Funt, B; Barnard, K, Estimating the scene illumination chromaticity using a neural network, Journal of the Optical Society of America A, 19, 2374-2386, (2002) · doi:10.1364/JOSAA.19.002374
[12] Chakrabarti, A; Hirakawa, K; Zickler, T, Color constancy with spatio-spectral statistics, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 1509-1519, (2012) · doi:10.1109/TPAMI.2011.252
[13] Chen, X., Pan, W., Kwok, J., & Garbonell, J. (2009). Accelerated gradient method for multi-task sparse learning problem. In: Proceedings of international conference of data mining (pp. 746-751).
[14] Ciurea, F., & Funt, B. (2003). A large image database for color constancy research. In: Proceedings of IS&T/SID color imaging conference (pp. 160-164).
[15] Cortes, C; Vapnik, V, Support-vector networks, Machine Learning, 20, 273-297, (1995) · Zbl 0831.68098
[16] Finlayson, G.G. (2013). Corrected-moment illuminant estimation. In: Proceedings of international conference on computer vision (ICCV) (pp. 1904-1911).
[17] Finlayson, G., & Trezzi, E. (2004). Shades of gray and color constancy. In Proceedings of IS&T/SID color imaging conference (pp. 37-41).
[18] Finlayson, G; Hordley, S; Hubel, P, Color by correlation: A simple, unifying framework for color constancy, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1209-1221, (2001) · doi:10.1109/34.969113
[19] Forsyth, DA, A novel algorithm for color constancy, International Journal of Computer Vision, 5, 5-36, (1990) · doi:10.1007/BF00056770
[20] Foster, DH, Colour constancy, Vision Research, 51, 674-700, (2010) · doi:10.1016/j.visres.2010.09.006
[21] Funt, B., & Mosny, M. (2012). Removing outliers in illumination estimation. In: Proceedings of IS&T color imaging conference (pp. 105-110).
[22] Funt, B; Shi, L, Maxrgb reconsidered, Journal of Imaging Science and Technology, 56, 020501-1-020501-10, (2012)
[23] Gao, S., Chia, L., & Tsang, I. (2011). Multi-layer group sparse coding for concurrent image classification and annotation. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2809-2816).
[24] Gao, S., Yang, K., Li, C., & Li, Y. (2013). A color constancy model with double-opponency mechanisms. In: Proceedings of international conference on computer vision (pp. 929-936).
[25] Gehler, P. V., Rother, C., Blake, A., & Minka, T. (2008). Bayesian color constancy revisited. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1-8).
[26] Geusebroek, JM; Smeulders, AWM, A six stimulus theory for stochastic texture, International Journal of Computer Vision, 62, 7-16, (2005) · doi:10.1007/s11263-005-4632-7
[27] Gijsenij, A; Gevers, T; Lucassen, M, A perceptual analysis of distance measures for color constancy algorithms, Journal of the Optical Society of America A, 26, 2243-2256, (2009) · doi:10.1364/JOSAA.26.002243
[28] Gijsenij, A; Gevers, T; Weijer, J, Generalized gamut mapping using image derivative structures for color constancy, International Journal of Computer Vision, 86, 127-139, (2010) · Zbl 1477.68506 · doi:10.1007/s11263-008-0171-3
[29] Gijsenij, A; Gevers, T; Weijer, J, Computational color constancy: survey and experiments, IEEE Transactions on Image Processing, 20, 2475-2489, (2011) · Zbl 1372.94092 · doi:10.1109/TIP.2011.2118224
[30] Gijsenij, A; Gevers, T; Weijer, J, Improving color constancy by photometric edge weighting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 918-929, (2011) · doi:10.1109/TPAMI.2011.197
[31] Gijsenij, A; Lu, R; Gevers, T, Color constancy for multiple light sources, IEEE Transactions on Image Processing, 21, 697-707, (2012) · Zbl 1372.94093 · doi:10.1109/TIP.2011.2165219
[32] Gijsenij, A; Gevers, T, Color constancy using natural image statistics and scene semantics, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 687-698, (2011) · doi:10.1109/TPAMI.2010.93
[33] Jenatton, R; Audibert, J; Bach, F, Structured variable selection with sparsity-inducing norms, Journal of Machine Learning Research, 12, 2777-2824, (2011) · Zbl 1280.68170
[34] Joze, HRV; Drew, MS, Exemplar-based colour constancy and multiple illumination, IEEE Transactions on Pattern Analysis and Machine Intelligence, 36, 860-873, (2014) · doi:10.1109/TPAMI.2013.169
[35] Land, EH, The retinex theory of color vision, Scientific America, 237, 108-128, (1977) · doi:10.1038/scientificamerican1277-108
[36] Li, B., Xiong, W., Hu, W., & Peng, H. (2013). Illumination estimation based on bilayer sparse coding. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1423-1429).
[37] Li, B; Xiong, W; Xu, D, A supervised combination strategy for illumination chromaticity estimation, ACM Transactions on Applied Perception, 8, 5, (2010) · doi:10.1145/1857893.1857898
[38] Li, B; Xiong, W; Hu, W; Funt, B, Evaluating combinational illumination estimation methods on real-world images, IEEE Transactions on Image Processing, 23, 1194-1209, (2014) · Zbl 1374.94199 · doi:10.1109/TIP.2013.2277943
[39] Liu, J., & Ye, J. (2010). Moreau-Yosida regularization for grouped tree structure learning. In: Proceedings of neural information processing systems conference (pp. 1-8).
[40] Logvinenko, AD; Funt, B; Godau, C, Metamer mismatching, IEEE Transactions on Image Processing, 23, 34-43, (2014) · Zbl 1374.94236 · doi:10.1109/TIP.2013.2283148
[41] Lowe, D. G. (2004). Distinctive image features from scale-invariant key-points. International Journal of Computer Vision, 60(2), 91-110.
[42] Lu, R., Gijsenij, A., Gevers, T., Nedovic, V., & Xu, D. (2009). Color Constancy using 3D Scene Geometry. In: Proceedings of international conference on computer vision (pp. 1749-1756).
[43] Nedovic, V; Smeulders, AWM; Redert, A; Geusebroek, JM, Stages as models of scene geometry, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1673-1687, (2010) · doi:10.1109/TPAMI.2009.174
[44] Nesterov, Y. (2007). Gradient methods for minimizing composite objective function. CORE, Catholic University Louvain, Louvain-la-Neuve, Technical Report.
[45] Parraga, CA; Vazquez-Corral, J; Vanrell, M, A new cone activation-based natural images dataset, Perception, 36, 180, (2009)
[46] Ruderman, DL; Cronin, TW; Chiao, CC, Statistics of cone responses to natural images: implications for visual coding, Journal of the Optical Society of America A, 15, 2036-2045, (1998) · doi:10.1364/JOSAA.15.002036
[47] Shi, L. (2012). Personal communication.
[48] Shi, L., & Funt, B. (2011). Re-processed version of the gehler color constancy dataset of 568 Images. Retrieved from http://www.cs.sfu.ca/ colour/data/
[49] Tan, R; Nishino, K; Ikeuchi, K, Color constancy through inverse-intensity chromaticity space, Journal of the Optical Society of America A, 21, 321-334, (2004) · doi:10.1364/JOSAA.21.000321
[50] Tseng, P, On accelerated proximal gradient methods for convex-concave optimization, SIAM Journal on Optimization, 6, 1-20, (2008)
[51] van de Weijer, J., Schmid, C., & Verbeek, J. (2007). Using High-level visual information for color constancy. In: Proceedings of international conference on computer vision (pp. 1-8).
[52] Weijer, J; Gevers, T; Gijsenij, A, Edge based color constancy, IEEE Transactions on Image Processing, 16, 2207-2214, (2007) · doi:10.1109/TIP.2007.901808
[53] Vazquez-Corral, J; Alejandro Párraga, C; Vanrell, M; Baldrich, R, Color constancy algorithms: psychophysical evaluation on a new dataset, Journal of Imaging Science and Technology, 53, 031105-031109, (2009) · doi:10.2352/J.ImagingSci.Technol.2009.53.3.031105
[54] Vazquez-Corral, J; Vanrell, M; Baldrich, R; Tous, F, Color constancy by category correlation, IEEE Transactions on Image Processing, 21, 1997-2007, (2012) · Zbl 1373.94417 · doi:10.1109/TIP.2011.2171353
[55] Wright, J., Ma, Y., Mairal, J., Sapiro, G. ( 2010). Sparse representation for computer vision and pattern recognition. In: Proceedings of the IEEE.
[56] Xiong, W; Funt, B, Estimating illumination chromaticity via support vector regression, Journal of Imaging Science and Technology, 50, 341-348, (2006) · doi:10.2352/J.ImagingSci.Technol.(2006)50:4(341)
[57] Yang, A; Zhou, Z; Balasubramanian, A; Sastry, S; Ma, Y, Fast L1-minimization algorithms for robust face recognition, IEEE Transactions on Image Processing, 22, 3234-3246, (2013) · doi:10.1109/TIP.2013.2262292
[58] Yosida, K. (1964). Functional analysis. Berlin: Springer. · Zbl 0152.32102
[59] Yuan, X., & Yan, S. (2010). Visual classification with multi-task joint sparse representation. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 3493-3500).
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.