Exploiting global and local features for image retrieval

Abstract

Two lines of image representation based on multiple features fusion demonstrate excellent performance in image retrieval. However, there are some problems in both of them: 1) the methods defining directly texture in color space put more emphasis on color than texture feature; 2) the methods extract several features respectively and combine them into a vector, in which bad features may lead to worse performance after combining directly good and bad features. To address the problems above, a novel hybrid framework for color image retrieval through combination of local and global features achieves higher retrieval precision. The bag-of-visual words (BoW) models and color intensity-based local difference patterns (CILDP) are exploited to capture local and global features of an image. The proposed fusion framework combines the ranking results of BoW and CILDP through graph-based density method. The performance of our proposed framework in terms of average precision on Corel-1K database is 86.26%, and it improves the average precision by approximately 6.68% and 12.53% over CILDP and BoW, respectively. Extensive experiments on different databases demonstrate the effectiveness of the proposed framework for image retrieval.

Keywords

local binary patterns / hue / saturation / value (HSV) color space / graph fusion / image retrieval

References

[1] LiuY, ZhangD-s, LuG-j, MaW-ying. A survey of content-based image retrieval with high-level semantics [J]. Pattern Recognition, 2007, 40(1): 262-282

[2] PenattiO A B, SilvaF B, ValleE, Gouet-BrunetV, TorresR D S. Visual word spatial arrangement for image retrieval and classification [J]. Pattern Recognition, 2014, 47(2): 705-720

[3] DattaR, JoshiD, LiJ, WangJ Z. Image retrieval: Ideas, influences, and trends of the new age [J]. ACM Computing Surveys (CSUR), 2008, 40(2): 5

[4] SwainM J, BallardD H. Color indexing [J]. International Journal of Computer Vision, 1991, 7111-32

[5] StrickerM A, OrengoM. Similarity of color images [C]. //Proceedings of IS&T/SPIE's Symposium on Electronic Imaging: Science & Technology, 1995, SPIE, Bellingham: 381392

[6] HuangJ, KumarS R, MitraM, ZhuW-j, ZabihR. Image Indexing Using Color Correlograms [C]. //Proceedings of 1997 IEEE Conference on Computer Vision and Pattern Recognition, 1997, IEEE Computer Society, Los Alamitos, CA: 762768

[7] PassG, ZabihR, MillerJ. Comparing images using color coherence vectors [C]. //Proceedings of the Fourth ACM International Conference on Multimedia, 1997, ACM, New York: 6573

[8] OjalaT, PietikainenM, MaenpaaT. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971-987

[9] HuangD, ShanC-f, ArdabilianM, WangY-h, ChenL-ming. Local binary patterns and its application to facial image analysis: A survey [J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2011, 41(6): 765-781

[10] HeikkiläM, PietikäinenM, SchmidC. Description of interest regions with local binary patterns [J]. Pattern Recognition, 2009, 42(3): 425-436

[11] GuoY-m, ZhaoG-y, PietikäinenM. Texture Classification using a Linear Configuration Model based Descriptor [C]. //Proceedings of the British Machine Vision Conference (BMVC), 2011, Citeseer, Dundee, United Kingdom: 110

[12] TanX-y, TriggsB. Enhanced local texture feature sets for face recognition under difficult lighting conditions [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1635-1650

[13] GuoZ-h, ZhangL, ZhangD. A completed modeling of local binary pattern operator for texture classification [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1657-1663

[14] MuralaS, WuQ M. Local mesh patterns versus local binary patterns: Biomedical image indexing and retrieval [J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(3): 929-938

[15] ZhangG, MaZ-m, DengL-g, XuC-ming. Novel histogram descriptor for global feature extraction and description [J]. Journal of Central South University of Technology, 2010, 17: 580-586

[16] LoweD G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110

[17] BayH, EssA, TuytelaarsT, VAN GoolL. Speeded-up robust features (SURF) [J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359

[18] RubleeE, RabaudV, KonoligeK, BradskiG. ORB: An efficient alternative to SIFT or SURF [C]. //Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV), 2011, IEEE, Piscataway, NJ: 25642571

[19] SivicJ, ZissermanA. Video Google: A text retrieval approach to object matching in videos [C]. //Proceedings of Ninth IEEE International Conference on Computer Vision, 2003, IEEE, Piscataway, NJ: 14701477

[20] ShenG-l, WuX-jun. Content based image retrieval by combining color, texture and Centrist [C]. //2013 Constantinides International Workshop on Signal Processing (CIWSP 2013), 2013, IET, Stevenage, GBR: 14

[21] ElalamiM E. A novel image retrieval model based on the most relevant features [J]. Knowledge-Based Systems, 2011, 24(1): 23-32

[22] SubrahmanyamM, WuQ M J, MaheshwariR P, BalasubramanianR. Modified color motif co-occurrence matrix for image indexing and retrieval [J]. Computers & Electrical Engineering, 2013, 39(3): 762-774

[23] LiuG-h, YangJ-yu. Image retrieval based on the texton co-occurrence matrix [J]. Pattern Recognition, 2008, 41(12): 3521-3527

[24] LiuG-h, ZhangL, HouY-k, LiZ-y, YangJ-yu. Image retrieval based on multi-texton histogram [J]. Pattern Recognition, 2010, 43(7): 2380-2389

[25] LiuG-h, YangJ-yu. Content-based image retrieval using color difference histogram [J]. Pattern Recognition, 2013, 46(1): 188-198

[26] ZhangS-t, YangM, CourT, YuK, MetaxasD N. Query specific rank fusion for image retrieval [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(4): 803-815

[27] SinghaM, HemachandranK. Performance analysis of color spaces in image retrieval [J]. Assam University Journal of Science and Technology, 2011, 7(2): 94-104

[28] VadivelA, SuralS, MajumdarA K. An integrated color and intensity co-occurrence matrix [J]. Pattern Recognition Letters, 2007, 28(8): 974-983

[29] LiuL, ZhaoL-j, LongY-l, KuangG-y, FieguthP. Extended local binary patterns for texture classification [J]. Image and Vision Computing, 2012, 30(2): 86-99

[30] CrossG R, JainA K. Markov random field texture models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1983, 1: 25-39

[31] GuoZ-h, ZhangL, ZhangD. Rotation invariant texture classification using LBP variance (LBPV) with global matching [J]. Pattern Recognition, 2010, 43(3): 706-719

[32] ZhangX-f, DouH, JuT, ZhangS-ting. Fusing heterogeneous features for the image-guided diagnosis of intraductal breast lesions [C]. //Proceedings of 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), 2015, IEEE, Piscataway, NJ: 12881291

[33] WangJ Z, LiJ, WiederholdG. SIMPLIcity: Semantics-sensitive integrated matching for picture libraries [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(9): 947-963

[34] LiJ, WangJ Z. Automatic linguistic indexing of pictures by a statistical modeling approach [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(9): 1075-1088

[35] AntaniS, KasturiR, JainR. A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video [J]. Pattern Recognition, 2002, 35(4): 945-965

[36] LiuG-h, LiZ-y, ZhangL, XuYong. Image retrieval based on micro-structure descriptor [J]. Pattern Recognition, 2011, 44(9): 2123-2133

[37] BesirisD, ZigourisE. Dictionary-based color image retrieval using multiset theory [J]. Journal of Visual Communication and Image Representation, 2013, 24(7): 1155-1167

[38] LanceG N, WilliamsW T. Mixed-data classificatory programs I-agglomerative systems [J]. Australian Computer Journal, 1967, 1(1): 15-20

[39] ElalamiM E. A new matching strategy for content based image retrieval system [J]. Applied Soft Computing, 2014, 14(1): 407-418

[40] JalabH. Image retrieval system based on color layout descriptor and Gabor filters [C]. //Proceedings of 2011 IEEE Conference on Open Systems (ICOS), 2011, IEEE, Piscataway, NJ: 3236

[41] HiremathP S, PujariJ. Content based image retrieval using color, texture and shape features [C]. //Proceedings of 2007 International Conference on Advanced Computing and Communications (ADCOM 2007), 2007, IEEE, Piscataway, NJ: 780784

[42] BanerjeeM, KunduM K, MajiP. Content-based image retrieval using visually significant point features [J]. Fuzzy Sets and Systems, 2009, 160(23): 3323-3341

[43] GuoJ-m, PrasetyoH, SuH-sheng. Image indexing using the color and bit pattern feature fusion [J]. Journal of Visual Communication and Image Representation, 2013, 24(8): 1360-1379

[44] IrtazaA, JaffarM A, AleisaE, ChoiT S. Embedding neural networks for semantic association in content based image retrieval [J]. Multimedia Tools and Applications, 2014, 72(2): 1911-1931

[45] ZengS, HuangR, WangH-b, KangZhen. Image retrieval using spatiograms of colors quantized by Gaussian Mixture Models [J]. Neurocomputing, 2016, 171(1): 673-684