Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey

Abstract

Line drawings, as a concise form, can be recognized by infants and even chimpanzees. Recently, how the visual system processes line-drawings attracts more and more attention from psychology, cognitive science and computer science. The neuroscientific studies revealed that line drawings generate similar neural actions as color photographs, which give insights on how to efficiently process big media data. In this paper, we present a comprehensive survey on line drawing studies, including cognitive mechanism of visual perception, computational models in computer vision and intelligent process in diverse media applications. Major debates, challenges and solutions that have been addressed over the years are discussed. Finally some of the ensuing challenges in line drawing studies are outlined.

Keywords

line drawings / cognitive computation / visual media / intelligent process

References

[1] Fischetti M. Computers vs. brains. Scientific American, 2011, 305(5):104–104

[2] FuX L, Cai L H, Liu Y, Jia J, Chen WF, Zhang Y, Zhao GZ, Liu YJ, Wu C X. A computational cognition model of perception, memory andjudgment. Science China Information Sciences, 2014, 57(3): 1–15

[3] Ullman S. High-level vision: Object recognition and visual cognition. Cambridge: MIT press, 2000

[4] Liu Y J, Fu Q F, Liu Y, Fu X L. A distributed computational cognitivemodel for object recognition. Science China Information Sciences, 2013, 56(9): 1–13

[5] Riddoch M, Humphreys G. Object Recognition. In: Rapp B, ed. Handbookof Cognitive Neuropsychology.Hove: Psychology Press

[6] Canny J. A computational approach to edge detection. IEEE Transactionson Pattern Analysis and Machine Intelligence, 1986 (6): 679–698

[7] Cole F, Golovinskiy A, Limpaecher A, Barros H S, Finkelstein A, Funkhouser T, Rusinkiewicz S. Where do people draw lines? ACMTransactions on Graphics, 2008, 27(3): 88

[8] Sayim B, Cavanagh P. What Line Drawings Reveal About the VisualBrain. Frontiers in Human Neuroscience, 2011, 5

[9] Yonas A, Arterberry M E. Infants perceive spatial structure specifiedby line junctions. Perception. 1994, 23(12): 1427–1435

[10] Itakura S. Recognition of line-drawing representations by a chimpanzee(Pan troglodytes). The Journal of general psychology, 1994,121(3): 189–197

[11] Cole F, Sanik K, DeCarlo D, Finkelstein A, Funkhouser T, Rusinkiewicz S, Singh M. How well do line drawings depict shape? ACM Transactions on Graphics, 2009, 28(3): 28

[12] Koenderink J J, Van Doorn A J, Kappers A M L. Surface perception inpictures. Perception & Psychophysics, 1992, 52(5): 487–496

[13] Koenderink J J, van Doorn A J, Kappers A M L, Todd J T. Ambiguityand the “mental eye” in pictorial relief. Perception, 2001, 30(4):431–448

[14] Marr D, Vision A. A Computational Investigation into the Human Representationand Processing of Visual Information. Cambridge: MITPress, 2010

[15] Watt R J, Rogers B J. Human vision and cognitive science. Research Directions in Cognitive Science: A European Perspective, 1989, 1: 9–22

[16] Sonka M, Hlavac V, Boyle R. Image processing, analysis, and machinevision. Cengage Learning, 2014

[17] Biederman I. Recognition-by-components: a theory of human imageunderstanding. Psychological review, 1987, 94(2): 115

[18] Biederman I, Ju G. Surface versus edge-based determinants of visual recognition. Cognitive Psychology, 1988, 20(1): 38–64

[19] Fu Q, Liu Y J, Chen W, Fu X. The time course of natural scene categorization in human brain: simple line-drawings vs. color photographs. Journal of Vision, 2013, 13(9): 1060

[20] Del Viva M M, Punzi G, Benedetti D. Information and Perception of Meaningful Patterns. PLoS One, 2013, 8(7): e69154

[21] Morgan M J. Features and the ‘primal sketch’. Vision Research, 2011, 51(7): 738–753

[22] Delorme A, Richard G, Fabre-Thorpe M. Key visual features for rapidcategorization of animals in natural scenes. Frontiers in Psychology, 2010, 1: 21

[23] Naber M, Hilger M, Einhäuser W. Animal detection and identificationin natural scenes: image statistics and emotional valence. Journal of Vision, 2012, 12(1): 25

[24] Derrington A M, Krauskopf J, Lennie P. Chromatic mechanisms in lateralgeniculate nucleus of macaque. The Journal of Physiology, 1984, 357(1): 241–265

[25] Campbell F W, Robson J G. Application of Fourier analysis to the visibilityof gratings. The Journal of physiology, 1968, 197(3): 551

[26] Joubert O R, Rousselet G A, Fabre-Thorpe M, Fize D. Rapid visualcategorization of natural scene contexts with equalized amplitude spectrumand increasing phase noise. Journal of Vision, 2009, 9(1): 2

[27] Jolicoeur P. The time to name disoriented natural objects. Memory &Cognition, 1985, 13(4): 289–303

[28] Farah M J, Hammond K M. Mental rotation and orientation-invariantobject recognition: dissociable processes. Cognition, 1988, 29(1): 29–46

[29] Westheimer G. The Fourier theory of vision. Perception, 2001, 30(5):531–542

[30] Oppenheim A V, Lim J S. The importance of phase in signals. In: Proceedings of the IEEE. 1981, 69(5): 529–541

[31] Piotrowski L N, Campbell F W. A demonstration of the visual importanceand flexibility of spatial-frequency amplitude and phase. Perception,1982, 11(3): 337–346

[32] Guyader N, Chauvin A, Peyrin C, Hérault J, Marendaz C. Image phaseor amplitude? Rapid scene categorization is an amplitude-based process. Comptes Rendus Biologies, 2004, 327(4): 313–318

[33] Chen W F, Liang J, Liu Y J, Fu Q F,Fu X L. Rapid natural scene categorization of line drawings is less influenced by amplitude spectra:evidence from a subliminal perception study. ASSC 18: Poster Session

[34] Morrone M C, Burr D C. Feature detection in human vision: a phasedependenten ergy model. In: Proceedings of the Royal Society of London. 1988, 221–245

[35] Devlin H, Tracey I, Johansen-Berg H, Clare S. What is Functional Magnetic Resonance Imaging (fMRI)? FMRIB Centre, Department of Clinical Neurology, University of Oxford

[36] Walther D B, Caddigan E, L F F, Beck D M. Natural scene categories revealed in distributed patterns of activity in the human brain. The Journalof Neuroscience, 2009, 29(34): 10573–10581

[37] Walther D B, Chai B, Caddigan E, Beck D M, Li F F. Simple linedrawings suffice for functional MRI decoding of natural scene categories. In: Proceedings of the National Academy of Sciences. 2011, 9661–9666

[38] Kim S G, Richter W, Uˇgurbil K. Limitations of temporal resolution infunctional MRI. Magnetic Resonance in Medicine, 1997, 37(4): 631–636

[39] Vanrullen R, Thorpe S J. The time course of visual processing: fromearly perception to decision-making. Journal of Cognitive Neuroscience, 2001, 13(4): 454–461

[40] Johnson J S, Olshausen B A. Timecourse of neural signatures of objectrecognition. Journal of Vision, 2003, 3(7): 4

[41] Fu Q F, Liu Y J, Dienes Z, Wu J H, Chen W F, Fu X L. Differenttime courses of natural scene categorization of color photographs andline-drawings: evidence from event-related potentials. Submitted forpublication, 2014

[42] Liu Y J, Ma C X, Fu Q, Fu X L, Qin S F, Xie L X. A Sketch-Based Approachfor Interactive Organization of Video Clips. ACM Transactionson Multimedia Computing, Communications, and Applications, 2014 ,11(1)

[43] Snodgrass J G, Vanderwart M. A standardized set of 260 pictures:norms for name agreement, image agreement, familiarity, and visualcomplexity. Journal of Experimental Psychology: Human Learningand Memory, 1980, 6(2): 174

[44] Magnié M N, Besson M, Poncet M, Dolisi C. The Snodgrass and Vanderwartset revisited: norms for object manipulability and for pictorialambiguity of objects, chimeric objects, and nonobjects. Journal ofClinical and Experimental Neuropsychology, 2003, 25(4): 521–560

[45] Rossion B, Pourtois G. Revisiting Snodgrass and Vanderwart’s objectpictorial set: the role of surface detail in basic-level object recognition. Perception, 2004, 33(2): 217–236

[46] Viggiano M P, Vannucci M, Righi S. A new standardized set of ecologicalpictures for experimental and clinical research on visual objectprocessing. Cortex, 2004, 40(3): 491–509

[47] Abel S, Weiller C, Huber W, Willmes K. Neural underpinnings formodel-oriented therapy of aphasic word production. Neuropsychologia, 2014, 57: 154–165

[48] Janssen N, Carreiras M, Barber H A. Electrophysiological effects of semanticcontext in picture and word naming. Neuroimage, 2011, 57(3):1243–1250

[49] Schnur T T. The persistence of cumulative semantic interference duringnaming. Journal of Memory and Language, 2014, 75: 27–44

[50] Strijkers K, Holcomb P J, Costa A. Conscious intention to speak proactively facilitates lexical access during overt object naming. Journal of Memory and Language, 2011, 65(4): 345–362

[51] Kang H, Lee S, Chui C. Coherent line drawing. In: Proceedings of 5thInternational Symposium on Non-photorealistic Animation and Rendering. 2007, 43–50

[52] Arbel T, Ferrie F P. Viewpoint selection by navigation through entropymaps. In: Proceedings of the 7th IEEE International Conference on Computer Vision. 1999, 248–254

[53] Laporte C, Arbel T. Efficient discriminant viewpoint selection for activebayesian recognition. International Journal of Computer Vision, 2006, 68(3): 267–287

[54] Secord A, Lu J, Finkelstein A, Sing M, Nealen A. Perceptual modelsof viewpoint preference. ACM Transactions on Graphics, 2011, 30(5):109

[55] Chen D Y, Tian X P, Shen Y T, Ouhyoung M. On visual similaritybased 3D model retrieval. Computer graphics forum. Publishing, 2003, 22(3): 223–232

[56] Cyr C M, Kimia B B. 3D object recognition using shape similiaritybased aspect graph. In: Proceedings the 8th IEEE International Conferenceon Computer Vision. 2001. 254–261

[57] Liu Y J, Luo X, Joneja A, Ma C X, Fu X L, Song D W. User-adaptivesketch-based 3-D CAD model retrieval. IEEE Transactions on AutomationScience and Engineering, 2013, 10(3): 783–795

[58] Duda R O, Hart P E, Stork D G. Pattern Classification. John Wiley &Sons, 1999.

[59] Chalechale A, Naghdy G, Mertins A. Sketch-based image matchingusing angular partitioning. IEEE Transactions on Systems, Man andCybernetics, 2005, 35(1): 28–41

[60] Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-basedshape retrieval. ACM Transactions on Graphics, 2012, 31(4): 31

[61] Hung C C, Carlson E T, Connor C E. Medial axis shape coding inmacaque inferotemporal cortex. Neuron, 2012, 74(6): 1099–1113

[62] Lescroart M D, Biederman I. Cortical representation of medial axisstructure. Cerebral Cortex, 2013, 23(3): 629–637

[63] Li Z, Qin S, Jin X. Skeleton-enhanced line drawings for 3D models. Graphical Models, 2014, 76(6): 620–632

[64] Lake B M, Salakhutdinov R, Tenenbaum J. One-shot learning by invertinga compositional causal process. Advances in neural information processing systems. 2013: 2526–2534

[65] Ma C X, Liu Y J, Yang H Y, Teng D X, Wang H A, Dai G Z. KnitSketch:a sketch pad for conceptual design of 2D garment patterns. IEEE Transactions on Automation Science and Engineering, 2011, 8(2): 431–437

[66] Sivic J, Zisserman A. Video Google: A text retrieval approach to objectmatching in videos

[67] Baeza-Yates R, Ribeiro-Neto B. Modern Information Retrieval. NewYork: ACM Press, 1999

[68] Sugihara K. Machine Interpretation of Line Drawings. Cambridge:MIT Press, 1986

[69] Hoffman D D. Visual Intelligence: How We Create WhatWe See. NewYork: W.W. Norton & Company, 2000

[70] Chen T, Cheng M M, Tan P, Ariel S, Hu S M. Sketch2Photo: internetimage montage. ACM Transactions on Graphics, 2009, 28(5): 124

[71] Cao Y, Wang C H, Zhang L Q, Zhang L. Edgel index for large-scalesketch-based image search. IEEE Conference on Computer Vision and Pattern Recognition, 2011: 761–768

[72] Truong B T, Venkatesh S. Video abstraction: a systematic review andclassification. ACM Transactions on Multimedia Computing, Communications, and Applications, 2007, 3(1): 3

[73] Ma C X, Liu Y J, Wang H A, Teng D X, Dai G Z. Sketch-based annotationand visualization in video authoring. IEEE Transactions onMultimedia, 2012, 14(4): 1153–1165

[74] Collomosse J P, McNeill G, Qian Y. Storyboard sketches for contentbased video retrieval. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 245–252

[75] Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3Dfreeform design. In: Proceedings of ACM SIGGRAPH Courses, 2007:21

[76] Nealen A, Igarashi T, Sorkine O, Alexa M. FiberMesh: designingfreeform surfaces with 3D curves. ACM Transactions on Graphics, 2007, 26(3): 41

[77] Liu Y J, Ma C X, Zhang D L. EasyToy: plush toy design using editablesketching curves. IEEE Computer Graphics and Applications, 2011,31(2): 49–57

[78] Zhu X, Jin X, Liu S, Zhoo H. Analytical solutions for sketch-basedconvolution surface modeling on the GPU. The Visual Computer, 2012,28(11): 1115–1125

[79] Yu C C, Liu Y J, Wu T F, Li K Y, Fu X L. A global energy optimization framework for 2.1D sketch extraction from monocular images. Graphical Models, 2014, 76(5): 507–521

[80] Liu Y J, Zhang J B, Hou J C, Ren J C. Cylinder detection in large-scalepoint cloud of pipeline plant. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(10): 1700–1707

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg