Please wait a minute...

Frontiers of Computer Science

Front. Comput. Sci.    2018, Vol. 12 Issue (5) : 1000-1012     https://doi.org/10.1007/s11704-017-6595-6
RESEARCH ARTICLE |
Local features and manifold ranking coupled method for sketch-based 3D model retrieval
Xiaohui TAN1, Yachun FAN2(), Ruiliang GUO3
1. College of Information Engineering, Capital Normal University, Beijing 100048, China
2. College of Information Science and Technology, Beijing Normal University, Beijing 100875, China
3. School of Fashion, Beijing Institute of Fashion Technology, Beijing 100028, China
Download: PDF(539 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

3D model retrieval can benefit many downstream virtual reality applications. In this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors.Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians (DoG) method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

Keywords sketch-based retrieval      3D model      manifold ranking      line drawing      local features     
Corresponding Authors: Yachun FAN   
Just Accepted Date: 29 September 2017   Online First Date: 30 October 2017    Issue Date: 21 September 2018
 Cite this article:   
Xiaohui TAN,Yachun FAN,Ruiliang GUO. Local features and manifold ranking coupled method for sketch-based 3D model retrieval[J]. Front. Comput. Sci., 2018, 12(5): 1000-1012.
 URL:  
http://journal.hep.com.cn/fcs/EN/10.1007/s11704-017-6595-6
http://journal.hep.com.cn/fcs/EN/Y2018/V12/I5/1000
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Xiaohui TAN
Yachun FAN
Ruiliang GUO
1 Zeleznik R C, Herndon K P, Hughes J F. SKETCH: an interface for sketching 3D scenes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. 1996, 163–170
https://doi.org/10.1145/237170.237238
2 Gao Y, Wang M, Tao D C, Ji R R, Dai Q H. 3-D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290–4303
https://doi.org/10.1109/TIP.2012.2199502
3 Walther D B, Chai B, Caddigan E, Beck D M, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666
https://doi.org/10.1073/pnas.1015666108
4 Eitz M, Hays J, Alexa M. How do humans sketch objects? ACMTransactions on Graphics, 2012, 31(4): 44
https://doi.org/10.1145/2185520.2185540
5 Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D. A search engine for 3D models. ACM Transactions on Graphics, 2003, 22(1): 83–105
https://doi.org/10.1145/588272.588279
6 Li B, Lu Y J, Ghumman A, Strylowski B, Gutierrez M, Sadiq S, Forster S, Feola N, Bugerin T. 3D Sketch-Based 3D model retrieval. In: Proceedings of the 5th ACM International Conference on Multimedia Retrieval. 2015, 555–558
https://doi.org/10.1145/2671188.2749349
7 Liu X L, Huang L, Deng C, Tao D C. Query-adaptive hash code ranking for large-scale multi-view visual search. IEEE Transactions on Image Processing, 2016, 25(10): 4514–4524
https://doi.org/10.1109/TIP.2016.2593344
8 Liu X L, Du B W, Deng C, Liu M, Lang B. Structure sensitive hashing with adaptive product quantization. IEEE Transactions on Cybernetics, 2016, 46(10): 2252–2264
https://doi.org/10.1109/TCYB.2015.2474742
9 DeCarlo D, Finkelstein A. Rusinkiewicz S, Santella . ASuggestive contours for conveying shape. ACM Transactions on Graphics, 2003, 22(3): 848–855
https://doi.org/10.1145/882262.882354
10 Saavedra J M, Bustos B, Schreck T, Yoon S M, Scherer M. Sketchbased 3D model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 47–50
11 Yoon S, Scherer M, Schreck T, Kuijper A. A sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 193–200
https://doi.org/10.1145/1873951.1873961
12 Yoon S M, Kuijper A. Sketch-based 3D model retrieval using compressive sensing classification. Electronics Letters, 2011, 47(21): 1181–1183
https://doi.org/10.1049/el.2011.2158
13 Judd T, Durand F, Adelson E. Apparent ridges for line drawing. ACM Transactions on Graphics, 2007, 26(3): 19
https://doi.org/10.1145/1276377.1276401
14 Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 1–10
https://doi.org/10.1145/2185520.2185527
15 Li B, Schreck T, Godil A, Alexa M, Boubekeur T, Bustos B, Chen J, Eitz M, Furuya T, Hildebrand K, Huang S, Johan H, Kuijper A, Ohbuchi R, Richter R, Saavedra J M, Scherer M, Yanagimachi T, Yoon G J, Yoon S M. Track: sketch-based 3D shape retrieval. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 109–118
16 Shao T J, Xu W W, Yin K K, Wang J D, Zhou K, Guo B N. Discriminative sketch-based 3D model retrieval via robust shape matching. In: Proceedings of Comput Graph Forum. 2011, 2011–2020
https://doi.org/10.1111/j.1467-8659.2011.02050.x
17 Li B, Lu Y, Li C, Godil A, Schreck T, Aono M, Burtscher M, Fu H, Furuya T, Johan H, Liu J, Ohbuchi R, Tatsuma A, Zou C. SHREC’14 track: extended large scale sketch-based 3D shape retrieval. In: Proceedings of Eurographics Workshop on 3D Object Retrieval. 2014
18 Li B, Lu Y, Godil A, Schreck T, Aono M, Johan H, Saavedra J M, Tashiro S. SHREC’13 track: scale sketch-based 3D shape retrieval. In: Proceedings of the 6th Eurographics Workshop on 3D Object Retrival. 2013, 89–96
19 Yoon S M, Yoon G, Schreck T. User-drawn sketch-based 3D object retrieval using sparse coding. Multimedia Tools and Applications. 2015, 74(13): 4707–4722
https://doi.org/10.1007/s11042-013-1831-z
20 Hu R, Wang T H, Collomosse J. A bag-of-regions approach to sketchbased image retrieval. In: Proceedings of the 18th IEEE International Conference on Image Processing. 2011, 3661–3664
21 Li B, Johan H. Sketch-based 3D model retrieval by incorporating 2D- 3D alignment. Multimedia Tools and Applications, 2013, 65(3): 363–385
https://doi.org/10.1007/s11042-012-1009-0
22 Li B, Johan H. A 3D model feature for retrieval. In: Proceedings of MMM. 2010, 185–195
23 Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509–522
https://doi.org/10.1109/34.993558
24 Ohbuchi R, Furuya T. Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 63–70
https://doi.org/10.1109/ICCVW.2009.5457716
25 Lowe D G. Distinctive Image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110
https://doi.org/10.1023/B:VISI.0000029664.99615.94
26 Wang F, Lin L F, Tang M. A new sketch-based 3D model retrieval approach by using global and local features. Graphical Models, 2014, 76(3): 128–139
https://doi.org/10.1016/j.gmod.2013.11.002
27 Wang F, Le K, Li Y. Sketch-based 3D shape retrieval using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1875–1883
https://doi.org/10.1109/CVPR.2015.7298797
28 Zhu F, Xie J, Fang Y. Learning cross-domain neural networks for sketch-based 3D shape retrieval. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016.
29 He J R, Li M J, Zhang H J, Tong H H, Zhang C S. Manifold-ranking based image retrieval. In: Proceedings of the 12th Annual ACM International Conference on Multimedia. 2004, 9–16
https://doi.org/10.1145/1027527.1027531
30 Tong H H, He J R, Li M J, Ma W Y, Zhan g H J, Zhang C S. Manifoldranking- based keyword propagation for image retrieval. EURASIP Journal on Applied Signal Processing, 2006, 2006: 190
31 Guan Z Y, Bu J J, Mei Q Z, Chen C, Wang C. Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 540–547
https://doi.org/10.1145/1571941.1572034
32 Agarwal S. Ranking on on graph data. In: Proceedings of the 23rd Internatioal Conference on Machine Learning. 2006, 25–32
https://doi.org/10.1145/1143844.1143848
33 Bu J J, Tan S L, Chen C, Wang C, Wu H, Zhang L J, He X F. Music recommendation by unified hypergraph: combining social media information and music content. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 391–400
https://doi.org/10.1145/1873951.1874005
34 Furuya T, Ohbuchi R. Similarity metric learning for sketch-based 3D object retrieval. Multimedia Tools and Applications, 2015, 74(23): 10367–10392
https://doi.org/10.1007/s11042-014-2171-3
35 Liu X L, Lang B, Xu Y, Cheng B. Feature grouping and local soft match for mobile visual search. Pattern Recognition Letters, 2012, 33(3): 239–246
https://doi.org/10.1016/j.patrec.2011.10.002
36 Yang Y1, Nie F P, Xu D B, Luo J, Zhuang Y T, Pan Y H. Multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742
https://doi.org/10.1109/TPAMI.2011.170
37 Liu X L, Deng C, Lang B, Tao D C, Li X L. Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2015, 25(2): 907–919
https://doi.org/10.1109/TIP.2015.2505180
Related articles from Frontiers Journals
[1] Ying-Ying XU, Li-Xiu YAO, Hong-Bin SHEN. Bioimage-based protein subcellular location prediction: a comprehensive review[J]. Front. Comput. Sci., 2018, 12(1): 26-39.
[2] Yongjin LIU, Minjing YU, Qiufang FU, Wenfeng CHEN, Ye LIU, Lexing XIE. Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey[J]. Front. Comput. Sci., 2016, 10(2): 216-232.
[3] Vahid MEHRDAD,Hossein EBRAHIMNEZHAD. 3D object retrieval based on histogram of local orientation using one-shot score support vector machine[J]. Front. Comput. Sci., 2015, 9(6): 990-1005.
[4] Song WANG, Seiichi UCHIDA, Marcus LIWICKI, Yaokai FENG. Part-based methods for handwritten digit recognition[J]. Front Comput Sci, 2013, 7(4): 514-525.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed