Lessons from human vision for robotic design

Abstract

The visual guidance of goal-directed movements requires transformations of incoming visual information that are different from those required for visual perception. For us to grasp an object successfully, our brain must use just-in-time computations of the object’s real-world size and shape, and its orientation and disposition with respect to our hand. These requirements have led to the emergence of dedicated visuomotor modules in the posterior parietal cortex of the human brain (the dorsal visual stream) that are functionally distinct from networks in the occipito-temporal cortex (the ventral visual stream) that mediate our conscious perception of the world. Although the identification and selection of goal objects and an appropriate course of action depends on the perceptual machinery of the ventral stream and associated cognitive modules, the execution of the subsequent goal-directed action is mediated by dedicated online control systems in the dorsal stream and associated motor areas. The dorsal stream allows an observer to reach out and grasp objects with exquisite ease, but by itself, deals only with objects that are visible at the moment the action is being programmed. The ventral stream, however, allows an observer to escape the present and bring to bear information from the past – including information about the function of objects, their intrinsic properties, and their location with reference to other objects in the world. Ultimately then, both streams contribute to the production of goal-directed actions. The principles underlying this division of labour between the dorsal and ventral streams are relevant to the design and implementation of autonomous robotic systems.

Keywords

Perception vs. action / Dorsal visual stream / Ventral visual stream / Tele-assistance / Grasping

References

[1] UngerleiderLG, MishkinM. IngleDJ, GoodaleMA, RJW. Analysis of Visual Behavior. Mansfield, 1982 Cambridge MIT Press 549

[2] GoodaleMA, MilnerAD. Trends. Neurosci., 1992, 15: 20

[3] MilnerAD, GoodaleMA. The Visual Brain in Action, 2006 2 Oxford UK OUP 1-297

[4] GoodaleMA, MilnerAD. Sight Unseen: An Exploration of Conscious and Unconscious Vision, 2013 2 Oxford UK OUP 1-218

[5] DijkermanHC, De HaanEH. Behav. Brain. Sci., 2007, 30: 189

[6] RauscheckerJP. Eur. J. Neurosci., 2015, 41: 579

[7] BridgemanB, LewisS, HeitG, NagleM. J. Exp. Psychol. [Hum. Percept.], 1979, 5: 692

[8] GoodaleMA, PélissonD, PrablancC. Nature, 1986, 320: 748

[9] GoodaleMA, MilnerAD, JakobsonLS, CareyDP. Nature, 1991, 349: 154

[10] HumphreyGK, GoodaleMA, JakobsonLS, ServosP. Perception, 1994, 23: 1457

[11] ServosP, GoodaleMA, HumphreyGK. Neuropsychologia, 1993, 31: 251

[12] LissauerH. Arch. Psychiatr., 1890, 21: 222

[13] BensonDF, GreenbergJP. Arch. Neurol., 1969, 20: 82

[14] CareyDP, HarveyM, MilnerAD. Neuropsychologia, 1996, 34: 329

[15] RiceNJ, McIntoshRD, SchindlerI, Mon-WilliamsM, DemonetJ, MilnerAD. Intact automatic avoidance of obstacles in patients with visual form agnosia. Exp. Brain Res., 2006, 174: 176

[16] GoodaleMA, MeenanJP, BülthoffHH, NicolleDA, MurphyKJ, RacicotC. Curr. Biol., 1994, 4: 604

[17] GoodaleMA. Vis. Res., 2011, 51: 1567

[18] BálintR. Monatsschr. Psychiatr. Neurol., 1909, 25: 51

[19] PereninM-T, VighettoA. Brain, 1988, 111: 643

[20] JakobsonLS, ArchibaldYM, CareyDP, GoodaleMA. Neuropsychologia, 1991, 29: 803

[21] SchindlerI, RiceNJ, McIntoshRD, RossettiY, VighettoA, MilnerAD. Automatic avoidance of obstacles is a dorsal stream function: Evidence from optic ataxia. Nature Neurosci., 2004, 7: 779

[22] GallivanJP, GoodaleMA. VallarG, CoslettHP. Handbook of Clinical Neurology, Vol. 151: The Parietal Lobe, 2018 Amsterdam Elsevier 449

[23] A. Alipour, J. Beggs, J. Brown, T. James, bioRxiv 2020.09.30.321299. https://doi.org/10.1101/2020.09.30.321299

[24] GlicksteinM. Trends. Neurosci., 2000, 23: 613

[25] KouiderS, DehaeneS. Philos. Trans. R. Soc., 2007, 362: 857

[26] HaggardP. Nat. Rev. Neurosci., 2017, 18: 196

[27] MilnerAD. Proc. R. Soc. B, 2012, 279: 2289

[28] ClarkA. Philos. Rev., 2001, 110: 495

[29] PookPK, BallardDH. Rob. Auton. Syst., 1996, 18: 259

[30] GoodaleMA, HumphreyGK. Cognition, 1998, 67: 181

[31] DesimoneR, DuncanJ. Ann. Rev. Neurosci., 1995, 18: 193

[32] GoodaleMA, HaffendenAM. Adv. Neurol., 2003, 93: 249

[33] CorbettaM, KincadeJM, ShulmanGL. J. Cogn. Neurosci., 2002, 14: 508

[34] BisleyJW, GoldbergME. Adv. Neurol., 2003, 93: 141

[35] GallivanJP, CantJS, GoodaleMA, FlanaganJR. Curr. Biol., 2014, 24: 1866

[36] JacksonSR, ShawA. J. Exp. Psychol. Hum. Percept. Perform., 2000, 26: 418

[37] GoodaleMA, GanelT. WagemansBJ. Oxford Handbook of Perceptual Organization, 2015 Oxford OUP 672

[38] RosenbaumDA, MarchakF, BarnesHJ, VaughanJ, SlottaJD, JorgensenMJ, MJ. JeannerodM. Attention and Performance XIII: Motor Representation and Control, 1990 Hillsdale, NJ Erlbaum 321

[39] GallivanJP, McLeanDA, ValyearKF, CulhamJC. Elife, 2013, 2: e00425

[40] ChenJ, SnowJC, CulhamJC, GoodaleMA. What role does “elongation” play in “tool-specific” activation and connectivity in the dorsal and ventral visual streams?. Cereb. Cortex, 2018, 28: 1117

[41] ChinellatoE, Del PobilAP. International Work-Conference on the Interplay between Natural and Artificial Computation, 2005 Heidelberg Springer p. 366

[42] ChinellatoE, Del PobilAP. Visual Neuroscience of Robotic Grasping: Achieving Sensorimotor Skills through Dorsal-Ventral Stream Integration, 2016 Heidelberg Springer 1-165

[43] HelmsM, VattamSS, GoelAK. Design Stud., 2009, 30: 606

[44] ShuLH, UedaK, ChiuI, CheongH. CIRP Ann. Manuf. Technol., 2011, 60: 673

[45] SaxenaA, DriemeyerJ, NgAY. Int. J. Robot. Res., 2008, 27: 157

[46] L.Y. Ku, L.E. Learned-Miller, R. Grupen, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2017), p. 2434

[47] R. Detry, J. Papon, L. Matthies, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2017), p. 3266

[48] JangE, VijayanarasimhanS, PastorP, IbarzJ, LevineS. Proc. Mach. Learn. Res., 2017, 78: 119

[49] FerrettiG, ChinellatoE. VallverdúBJ, MüllerV. Blended Cognition, 2019 Cham Springer 99