×

zbMATH — the first resource for mathematics

Towards intelligent image retrieval. (English) Zbl 0988.68800
Summary: Research into techniques for the retrieval of images by semantic content is still in its infancy. This paper reviews recent trends in the field, distinguishing four separate lines of activity: automatic scene analysis, model-based and statistical approaches to object classification, and adaptive learning from user feedback. It compares the strengths and weaknesses of model-based and adaptive techniques, and argues that further advances in the field are likely to involve the increasing use of techniques from the field of artificial intelligence.

MSC:
68U99 Computing methodologies and applications
68P20 Information storage and retrieval of data
68U10 Computing methodologies for image processing
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] T. Kato, Database architecture for content-based image retrieval, in: A.A. Jambardino, W.R. Niblack (Eds.), Image Storage and Retrieval Systems, Proc SPIE 1662, 1992, 112-123.
[2] W.R. Niblack et al., The QBIC project: querying images by color, texture and shape, IBM Research Report RJ-9203, 1993.
[3] A. Pentland et al., Photobook—tools for content-based manipulation of image databases, Storage and Retrieval for Image and Video Databases II, Proc SPIE 2185, 1994, pp. 34-47.
[4] Idris, F.; Panchanathan, S., Review of image and video indexing techniques, J. visual commun. image representation, 8, 2, 146-166, (1997)
[5] Rui, Y., Image retrieval current techniques, promising directions, and open issues, J. visual commun. image representat., 10, 1, 39-62, (1999)
[6] de Marsicoi, M., Indexing pictorial documents by their content: a survey of current techniques, Image and vision comput., 15, 119-141, (1997)
[7] Swain, M.J.; Ballard, D.H., Color indexing, Int. J. comput. vision, 7, 1, 11-32, (1991)
[8] J.R. Smith, S.F. Chang, Querying by color regions using the VisualSEEk content-based visual query system, in: M.T. Maybury (Ed.), Intelligent Multimedia Information Retrieval, AAAI Press, Menlo Park, CA, 1997, pp. 23-41.
[9] Liu, F.; Picard, R.W., Periodicity directionality and randomnesswold features for image modelling and retrieval, IEEE trans. pattern anal. Mach. intell., 18, 7, 722-733, (1996)
[10] Manjunath, B.S.; Ma, W.Y., Texture features for browsing and retrieval of large image data, IEEE trans. pattern anal. Mach. intell., 18, 837-842, (1996)
[11] R. Mehrotra, J.E. Gary, Similar shape retrieval in shape data management, IEEE Comput. 28 (9) 57-62, Sep 1995.
[12] Jain, A.K.; Vailaya, A., Image retrieval using color and shape, Pattern recognition, 29, 8, 1233-1244, (1996)
[13] Gudivada, V.N.; Raghavan, V.V., Design and evaluation of algorithms for image retrieval by spatial similarity, ACM trans. inform. systems, 13, 2, 115-144, (1995)
[14] K.C. Liang, C.C.J. Kuo, Implementation and performance evaluation of a progressive image retrieval system, in: I.K. Sethi, R.C. Jain (Eds.), Storage and Retrieval for Image and Video Databases VI, Proc SPIE 3312, 1998, 37-48.
[15] S. Ravela, R. Manmatha, Retrieving images by appearance, Proceedings of IEEE International Conference on Computer Vision (ICCV98), Bombay, India, 1998, pp. 608-613.
[16] Gudivada, V.N.; Raghavan, V.V., Content-based image retrieval systems, IEEE comput., 28, 9, 18-22, (1995)
[17] J.P. Eakins, M.E Graham, Content-based image retrieval, JISC Technology Applications Programme Report 39, October 1999. Available online at http://www.unn.ac.uk/iidr/CBIR/report.html.
[18] Armitage, L.; Enser, P.G.B., Analysis of user need in image archives, J. inform. sci., 23, 4, 287-299, (1997)
[19] Eakins, J.P., Techniques for image retrieval, library and information briefings 85, (1998), British Library and South Bank University London
[20] Salton, G., The SMART retrieval system — experiments in automatic document processing, (1971), Prentice-Hall New Jersey
[21] Y. Rui et al., “Relevance feedback techniques in interactive content-based image retrieval” in: I.K. Sethi, R.C. Jain (Eds.), Storage and Retrieval for Image and Video Databases VI, Proc SPIE 3312, 1997, pp. 25-36.
[22] C. Meilhac et al., “Relevance feedback in Surfimage” Proceedings of Fourth IEEE Workshop on Applications of Computer Vision (WACV’98), 1998, pp. 266-267.
[23] Beckmann, N., The R*-Treean efficient and robust access method for points and rectangles, ACM SIGMOD record, 19, 2, 322-331, (1990)
[24] M. Wertheimer, “Untersuchungen zur Lehre von der Gestalt”, Psycholog. Forschung 4 (1923) 301-350. (Translated as laws of organization in perceptual forms, in: W.D. Ellis (Ed.), A Sourcebook of Gestalt Psychology, Humanities Press, New York, 1950).
[25] Luger, G.F.; Stubblefield, W.A., Artificial intelligence, (1997), Addison-Wesley Reading, MA
[26] Newell, A.; Simon, H.A., Computer science as empirical inquirysymbols and search, Commun. ACM, 19, 3, 113-126, (1976)
[27] R.A. Brooks et al., Alternative essences of intelligence, Proceedings of 15th National Conference on Artificial Intelligence (AAAI-98), 1998, pp. 961-968.
[28] Rosch, E., Basic objects in natural categories, Cognitive psychol., 8, 382-439, (1976)
[29] D.A. Forsyth et al., “Finding pictures of objects in large collections of images”, in: P.B. Heidorn, B. Sandore (Eds.), Digital Image Access and Retrieval: 1996 Clinic on Library Applications of Data Processing Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign, 1997, pp. 118-139.
[30] J.M. Buijs, M.S. Lew, Visual learning of simple semantics in ImageScape, VISUAL99: Third International Conference on Visual Information and Information Systems. Lecture Notes in Computer Science, Vol. 1614, Springer, Berlin, 1999, pp. 131-138.
[31] T. Hermes et al., Image retrieval for information systems, in: W.R. Niblack, R.C. Jain (Eds.), Storage and Retrieval for Image and Video Databases III, Proc SPIE 2420, 1995, pp. 394-405.
[32] A. Oliva et al., Global semantic classification of scenes using, power spectrum templates, CIR-99: The Challenge of Image Retrieval, Newcastle upon Tyne, UK, February 1999.
[33] M. Szummer, R. Picard, Indoor-outdoor image classification, IEEE International Workshop on Content-based Access of Image and Video Databases (CAIVD98), Bombay, India, 1998, pp. 42-51.
[34] P. Lipson et al., Configuration-based scene classification and image indexing, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR-97), Puerto Rico, 1997, pp. 1007-1013.
[35] Vailaya, A., On image classificationcity images vs landscapes, Pattern recognition, 31, 12, 1921-1936, (1998)
[36] S. Paek et al., Integration of visual and text-based approaches for the content labeling and classification of photographs, ACM SIGIR’99 Workshop on Multimedia Indexing and Retrieval, Berkeley, CA, August 1999.
[37] A. Vailaya, A.K. Jain, Incremental learning for Bayesian classification of images, IEEE International Conference on Image Processing(ICIP’99), Kobe, Japan, October 1999.
[38] A. Vailaya, A.K. Jain, Detecting sky and vegetation in outdoor images, in Storage and Retrieval for Media Databases 2000, Proc SPIE 3972, January 2000, pp. 411-420.
[39] Brooks, R.A., Model-based three-dimensional interpretations of two-dimensional images, IEEE trans. pattern anal. Mach. intell., 5, 2, 140-150, (1983)
[40] Matsuyama, T.; Hwang, V., Sigmaa knowledge-based aerial image understanding system, (1990), Plenum New York
[41] Draper, B.A., The SCHEMA system, Int. J. comput vision, 2, 209-250, (1989)
[42] Srihari, R.K., Automatic indexing content-based retrieval of captioned images, IEEE computer, 28, 9, 49-56, (1995)
[43] G. Durand et al., Extraction of composite visual objects from audiovisual materials, in: S. Panchanathan et al. (Eds.), Multimedia Storage and Archiving Systems IV, Proc SPIE 3846, 1999, pp. 194-203.
[44] Campbell, N.W., Interpreting image databases by region classification, Pattern recognition, 30, 4, 555-567, (1997)
[45] A. Martinez, J.R. Serra, Semantic Access to a Database of Images: an approach to object-related image retrieval, Proceedings of IEEE Multimedia Systems (ICMCS), Florence, Italy, 1999, pp. 624-629.
[46] S. Belongie et al., Color and texture-based image segmentation using EM and its application to content-based image retrieval, Proceedings of IEEE International Conference on Computer Vision (ICCV-98), Bombay, India, 1998, pp. 675-682.
[47] T. Leung, J. Malik, Recognizing surfaces using three-dimensional textons, Seventh IEEE International Conference on Computer Vision (ICCV-99), Vol. 2, Corfu, Greece, 1999, pp. 1010-1017.
[48] C. Bregler, J. Malik, Learning appearance based models: mixtures of second moment experts, in: M.C. Mozer et al. (Eds.), Advances in Neural Information Processing Systems 9, MIT Press, 1997, pp. 845-851.
[49] H. Schneiderman, T. Kanade, Probabilistic modeling of local appearance and spatial relationships for object recognition, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 98), Santa Barbara, CA, 1998, pp. 45-51.
[50] Rowley, H.A., Neural network-based face detection, IEEE trans. pattern anal. Mach. intell., 20, 1, 23-38, (1998)
[51] B. Schiele, J.L. Crowley, The concept of visual classes for object classification, Proceedings of SCIA’97, Tenth Scandinavian Conference on Image Analysis, Lappeenranta, Finland, 1997, pp. 43-50.
[52] B. Schiele, J.L. Crowley, Object recognition using multidimensional receptive field histograms, Proceedings of Fourth European Conference on Computer Vision, Cambridge, UK, 1996, pp. 610-619.
[53] Lewis, P.H., Towards multimedia thesaurus support for media-based navigation, (), 111-118
[54] T. Minka, An image database browser that learns from user interaction, MIT Media Laboratory Technical Report No. 365, 1996.
[55] Michalski, R.S., A theory and methodology of inductive learning, Artificial intelligence, 20, 2, 111-161, (1983)
[56] G. Frederix, E.J. Pauwels, Automatic interpretation based on robust segmentation and shape extraction, VISUAL99: Third International Conference on Visual Information and Information Systems, Lecture Notes in Computer Science, Vol. 1614, Springer, Berlin, 1999, pp. 769-776.
[57] S.F. Chang et al., Semantic visual templates: linking visual features to semantics, in IEEE International Conference on Image Processing (ICIP’98), Chicago, Illinois, 1998, 531-535.
[58] M.E.J. Wood et al., “Iterative refinement by relevance feedback in content-based digital image retrieval”, Proceedings of ACM Multimedia 98, Bristol, UK, 1998, 13-20.
[59] C.S. Lee et al., Information embedding based on users’ relevance feedback for image retrieval, in: S. Panchanathan et al., (Eds.), Multimedia Storage and Archiving Systems IV, Proc SPIE 3846, 1999, 294-304.
[60] A. Jaimes, S.F. Chang, Model-based classification of visual information for content-based retrieval, Storage and Retrieval for Image and Video Databases, Proc SPIE 3656, 1999, pp. 402-414.
[61] D. McG Squire, T. Pun, A comparison of human and machine assessments of image similarity for the organization of image databases, Proceedings of SCIA’97, Tenth Scandinavian Conference on Image Analysis, Lappeenranta, Finland, 1997, pp. 51-58.
[62] Valiant, L.G., A theory of the learnable, commun. ACM, 27, 1134-1142, (1984) · Zbl 0587.68077
[63] S. Santini, R.C. Jain, “Do images mean anything?” Proceedings of IEEE International Conference on Image Processing (ICIP-97), 1997, 564-567.
[64] J.L. Kolodner, Case-based Reasoning. Morgan Kaufmann, San Mateo, CA, 1993. · Zbl 0811.68055
[65] Mitchell, T.M., Explanation-based generalizationa unifying view, Mach. learning, 1, 1, 47-80, (1986)
[66] Hall, R.P., Computational approaches to analogical reasoninga comparative study, Artif. intell., 39, 1, 39-120, (1989) · Zbl 0668.68097
[67] Fisher, D.H., Knowledge acquisition via incremental conceptual clustering, Mach. learning, 2, 139-172, (1987)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.