A survey of sketch based modeling systems

Abstract

As 3D technology, including computer graphics, virtual reality and 3D printing, has been rapidly developed in the past years, 3D models are gaining an increasingly huge demand. Traditional 3D modeling platforms such as Maya and ZBrush, utilize “windows, icons, menus, pointers” (WIMP) interface paradigms for fine-grained control to construct detailed models. However, the modeling progress can be tedious and frustrating and thus too hard for a novice user or even a well trained artist. Therefore, a more intuitive interface is needed. Sketch, an intuitive communication and modeling tool for human beings, becomes the first choice of modeling community. So far, various sketch-based modeling systems have been created and studied. In this paper, we attempt to show how these systems work and give a comprehensive survey. We review and categorize the systems in four aspects: the input, the knowledge they use, the modeling approach and the output. We also discuss about inherent challenges and open problems for researchers in the future.

Keywords

sketch-based modeling / interactive design / sketch comprehension

References

[1] Botsch M, Pauly M, Kobbelt L, Alliez P, Lévy B, Bischoff S, Röossl C. Geometric modeling based on polygonal meshes. In: Proceedings of the ACM SIGGRAPH Course Notes. 2007

[2] Perry R N, Frisken S F. Kizamu: a system for sculpting digital characters. In: Proceedings of the 28th ACM Annual Conference on Computer Graphics and Interactive Techniques. 2001, 47–56

[3] Zoran A, Shilkrot R, Nanyakkara S, Paradiso J. The hybrid artisans: a case study in smart tools. ACM Transactions on Computer-Human Interaction, 2014, 21(3): 15

[4] Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R. Real-time human pose recognition in parts from single depth images. Communications of the ACM, 2013, 56(1): 116–124

[5] Henry P, Krainin M, Herbst E, Ren X F, Fox D. RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research, 2012, 31(5): 647–663

[6] Smisek J, Jancosek M, Pajdla T. 3D with kinect. In: Fossati A, Gall J, Helmut Grabner H, . <Eds/>. Consumer Depth Cameras for Computer Vision: Research Topics and Applications. London: Springer-Verlag, 2013, 3–25

[7] Quan L. Image-based modeling. Springer Science & Business Media, 2010

[8] Fan L B, Wang R M, Xu L L, Deng J S, Liu L G. Modeling by drawing with shadow guidance. Computer Graphics Forum, 2013, 23(7): 157–166

[9] Yuan X R, Xu H, Nguyen M, Shesh A, Chen B Q. Sketch-based segmentation of scanned outdoor environment models. In: Proceedings of EG Workshop on Sketch-Based Interfaces and Modeling. 2005, 19–26

[10] Wu H Y, Pan C H, Pan J, Yang Q, Ma S D. A sketch-based interactive framework for real-time mesh segmentation. In: Proceedings of Computer Graphics International. 2007

[11] Schmidt R, Wyvill B, Sousa M C, Jorge J A. Shapeshop: sketch-based solid modeling with blobtrees. In: Proceedings of ACM SIGGRAPH 2007 courses. 2007

[12] Turquin E, Wither J, Boissieux L, Cani M P, Hughes J F. A sketchbased interface for clothing virtual characters. IEEE Computer Graphics and Applications, 2007, 27(1): 72–81

[13] Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3D freeform design. In: Proceedings of the 26th ACM Annual Conference on Computer Graphics and Interactive Techniques. 1999, 409–416

[14] Olsen L, Samavati F F, Sousa M C, Jorge J A. Sketch-based modeling: a survey. Computers & Graphics, 2009, 33(1): 85–103

[15] Cook M T, Agah A. A survey of sketch-based 3-D modeling techniques. Interacting with Computers, 2009, 21(3): 201–211

[16] Kho Y, Garland M. Sketching mesh deformations. In: Proceedings of ACM SIGGRAPH 2007 Courses. 2007

[17] Karpenko O A, Hughes J F. Smoothsketch: 3D free-form shapes from complex sketches. ACM Transactions on Graphics, 2006, 25(3): 589–598

[18] Xu K, Chen K, Fu H B, Sun W K L, Hu S M. Sketch2Scene: sketchbased co-retrieval and co-placement of 3D models. ACM Transactions on Graphics, 2013, 32(4): 123

[19] Iarussi E, Bommes D, Bousseau A. Bendfields: regularized curvature fields from rough concept sketches. ACM Transactions on Graphics, 2015, 34(3): 24

[20] Bae S H, Balakrishnan R, Singh K. ILoveSketch: as-natural-aspossible sketching system for creating 3D curve models. In: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology. 2008, 151–160

[21] Lee S, Feng D, Grimm C, Gooch B. A sketch-based user interface for reconstructing architectural drawings. Computer Graphics Forum, 2008, 27(1): 81–90

[22] Bae S H, Balakrishnan R, Singh K. EverybodyLovesSketch: 3D sketching for a broader audience. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology. 2009, 59–68

[23] Zimmermann J, Nealen A, Alexa M. SilSketch: automated sketchbased editing of surface meshes. In: Proceedings of the 4th Eurographics Workshop on Sketch-based Interfaces and Modeling. 2007, 23–30

[24] Zimmermann J, Nealen A, Alexa M. Sketchbased interfaces: sketching contours. Computers Graphics, 2008, 32(5): 486–499

[25] Kraevoy V, Sheffer A, van de Panne M. Modeling from contour drawings. In: Proceedings of the 6th ACM Eurographics Symposium on Sketch-Based Interfaces and Modeling. 2009, 37–44

[26] Nealen A, Sorkine O, Alexa M, Cohen-Or D. A sketch-based interface for detail-preserving mesh editing. ACM Transactions on Graphics, 2005, 24(3): 1142–1147

[27] Nam S H, Chai Y H. SPACESKETCH: shape modeling with 3D meshes and control curves in stereoscopic environments. Computers Graphics, 2012, 36(5): 526–533

[28] Rivers A, Durand F, Igarashi T. 3D modeling with silhouettes. ACM Transactions on Graphics, 2010, 29(4): 109

[29] Levi Z, Gotsman C. ArtiSketch: a system for articulated sketch modeling. Computer Graphics Forum, 2013, 32: 235–244

[30] Lee Y J, Zitnick C L, Cohen M F. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics, 2011, 30(4): 27

[31] Andre A, Saito S. Single-view sketch based modeling. In: Proceedings of the 8th ACM Eurographics Symposium on Sketch-Based Interfaces and Modeling. 2011, 133–140

[32] Gingold Y, Igarashi T, Zorin D. Structured annotations for 2D-to-3D modeling. ACM Transactions on Graphics, 2009, 28(5): 148

[33] Hoffman D D. Visual intelligence: how we create what we see. New York: W.W. Norton & Company, 2000

[34] Xu B X, Chang W, Sheffer A, Bousseau A, McCrae J, Singh K. True2form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics, 2014, 33(4): 131

[35] Schmidt R, Khan A, Singh K, Kurtenbach G. Analytic drawing of 3D scaffolds. ACM Transactions on Graphics, 2009, 28(5): 149

[36] Zou M, Holloway M, Carr N, Ju T. Topology-constrained surface reconstruction from cross-sections. ACM Transactions on Graphics, 2015, 34(4): 128

[37] Shtof A, Agathos A, Gingold Y, Shamir A, Cohen-or D. Geosemantic snapping for sketch-based modeling. Computer Graphics Forum, 2013, 32: 245–253

[38] Chen T, Zhu Z, Shamir A, Hu S M, Cohen-Or D. 3-sweep: extracting editable objects from a single photo. ACM Transactions on Graphics, 2013, 32(6): 195

[39] Chen X J, Kang S B, Xu Y Q, Dorsey J, Shum H Y. Sketching reality: realistic interpretation of architectural designs. ACM Transactions on Graphics, 2008, 27(2): 11

[40] Jiang N J, Tan P, Cheong L F. Symmetric architecture modeling with a single image. ACM Transactions on Graphics, 2009, 28(5): 113

[41] Chen C J, Neubert B, Xu Y Q, Deussen O, Kang S B. Sketch-based tree modeling using Markov random field. ACM Transactions on Graphics, 2008, 27(5): 109

[42] Tan P, Fang T, Xiao J X, Zhao P, Quan L. Single image tree modeling. ACM Transactions on Graphics, 2008, 27(5): 108

[43] Davis J, Agrawala M, Chuang E, Popović Z, Salesin D. A sketching interface for articulated figure animation. In: Proceedings of the 2003 ACMSIGGRAPH/Eurographics Symposium on Computer Animation. 2003, 320–328

[44] Kazmi I K, You L H, Yang X S, Jin X G, Zhang J J. Efficient sketchbased creation of detailed character models through data-driven mesh deformations. Computer Animation and Virtual Worlds, 2015, 26(3– 4): 469–481

[45] Yan F L, Gong M L, Cohen-Or D, Deussen O, Chen N Q. Flower reconstruction from a single photo. Computer Graphics Forum. 2014, 33(2): 439–447

[46] Pan Z R, Huang J, Tong Y Y, Zheng C X, Bao H J. Interactive localized liquid motion editing. ACM Transactions on Graphics, 2013, 32(6): 184

[47] Xu K, Kim V G, Huang Q X, Kalogerakis E. Data-driven shape analysis and processing. 2015, arXiv preprint arXiv:1502.06686

[48] Shin H J, Igarashi T. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In: Proceedings of Graphics Interface 2007. 2007, 63–70

[49] Xu K, Zheng H L, Zhang H, Cohen-Or D, Liu L G, Xiong Y S. Photo-inspired model-driven 3D object modeling. ACM Transactions on Graphics, 2011, 30(4): 80

[50] Su H, Huang Q X, Mitra N J, Li Y Y, Guibas L. Estimating image depth using shape collections. ACM Transactions on Graphics, 2014, 33(4):37

[51] Xie X H, Xu K, Mitra N J, Cohen-Or D, Gong W Y, Su Q, Chen B Q. Sketch-to-design: context-based part assembly. Computer Graphics Forum, 2013, 32(8): 233–245

[52] Williams L R, Jacobs D W. Stochastic completion fields: a neural model of illusory contour shape and salience. Neural computation, 1997, 9(4): 837–858

[53] Nealen A, Igarashi T, Sorkine O, Alexa M. Fibermesh: designing freeform surfaces with 3D curves. ACM Transactions on Graphics, 2007, 26(3): 41

[54] Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Rössl C, Seidel H P. Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ ACM SIGGRAPH Symposium on Geometry Processing. 2004, 175–184

[55] Pan H, Liu Y, Sheffer A, Vining N, Li C J, Wang W P. Flow aligned surfacing of curve networks. ACM Transactions on Graphics, 2015, 34(4): 127

[56] Alexa M. Differential coordinates for local mesh morphing and deformation. The Visual Computer, 2003, 19(2): 105–114

[57] De Paoli C, Singh K. SecondSkin: Sketch-based construction of layered 3D models. ACM Transactions on Graphics, 2015, 34(4): 126

[58] Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 31

[59] Funkhouser T, Kazhdan M, Shilane P, Min P, Kiefer W, Tal A, Rusinkiewicz S, Dobkin D. Modeling by example. ACM Transactions on Graphics, 2004, 23(3): 652–663

[60] Eitz M, Hildebrand K, Boubekeur T, Alexa M. Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Transactions on Visualization and Computer Graphics, 2011, 17(11): 1624–1636

[61] Eitz M, Hays J, Alexa M. How do humans sketch objects? ACMTransactions on Graphics, 2012, 31(4): 44

[62] Schneider R G, Tuytelaars T. Sketch classification and classificationdriven analysis using fisher vectors. ACM Transactions on Graphics, 2014, 33(6): 174

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg