Riemannian geometric approach to optimal binocular rotation, pyramid based interpolation and bio-mimetic pan-tilt movement

Abstract

Over the past several years, we have been studying the problem of optimally rotating a rigid sphere about its center, where the rotation is actuated by a triplet of external torques acting on the body. The control objective is to repeatedly direct a suitable radial vector, called the gaze vector, towards a stationary point target in IR³. The orientation of the sphere is constrained to lie in a suitable submanifold of S O ( 3 ). Historically, the constrained rotational movements were studied by physiologists in the nineteenth century, interested in eye and head movements. In this paper we revisit the gaze control problem, where two visual sensors, are tasked to simultaneously stare at a point target in the visual space. The target position changes discretely and the problem we consider is how to reorient the gaze directions of the sensors, along the optimal pathway of the human eyes, to the new location of the target. This is done by first solving an optimal control problem on the human binocular system. Next, we use these optimal control and show that a pan-tilt system can be controlled to follow the gaze trajectory of the human eye requiring a nonlinear static feedback of the pan and tilt angles and their derivatives. Our problem formulation uses a new Riemannian geometric description of the orientation space. The paper also introduces a new, pyramid based interpolation method, to implement the optimal controller.

Keywords

Binocular vision / Riemannian geometry / Optimal control / Pyramid based interpolation / Bio-Mimetic Pan-Tilt rotation

References

[1] M. M. Rajamuni, E. Aulisa, B. K. Ghosh, Optimal control problems in binocular vision. Proc. 19 ^{t h} World Congress IFAC, 5283–5289 (2014). https://doi.org/10.3182/20140824-6-za-1003.02644.

[2] I. B. Wijayasinghe, B. K. Ghosh, Binocular eye tracking control satisfying Hering’s law. Proc. 52 ^{n d} IEEE Conf. Dec. Control, 6475–6480 (2013).

[3] T. Oki, B. K. Ghosh, Stabilization and trajectory tracking of version and vergence eye movements in human binocular control. Proc. 2015 Eur. Control Conf. (ECC), 1573–1580 (2015).

[4] J. Ruths, S. Ghosh, B. K. Ghosh, Optimal tracking of version and vergence eye movements in human binocular control. Proc. 2016 Eur. Control Conf. (ECC), 2410–2415 (2016).

[5] GhoshB. K., AthukorallageB.. Minimum energy optimal external torque control of human binocular vision. Control Theory Technol., 2020, 18: 431-458 https://doi.org/10.1007/s11768-020-00015-x

[6] PolpitiyaA. D., DayawansaW. P., MartinC. F., GhoshB. K.. Geometry and control of human eye movements. IEEE Trans. Autom. Control, 2007, 52(2):170-180

[7] RaphanT.. Modeling control of eye orientation in three dimension I, Role of muscle pulleys in determining saccadic trajectory. J Neurophysiol, 1998, 79: 2653-2667

[8] SnellR. S., LempM. A.. Clinical Anatomy of the Eye, 2 ^{n d} Edition, 2013 Malden Wiley

[9] NielsenJ. B.. How we walk: Central control of muscle activity during human walking. Neuroscience, 2003, 9(3):195-204

[10] AngelakiD. E., HessB. J. M.. Control of eye orientation: where does the brains role end and the muscle’s begin. 2003 Fed. Eur. Neurosci. Soc. Eur. J. Neurosci., 2004, 19: 1-10

[11] RobinsonD. A.. The use of control systems analysis in the neurophysiology of eye movements. Annu. Rev. Neurosci., 1981, 4: 463-503

[12] Von HelmholtzH.. Handbuch der Physiologischen Optik, 1867 Voss Leipzig

[13] DondersF. C.. The 11 ^{t h} yearly report of the Netherlands hospital for necessitous eye patients (in dutch), 1870 Utrecht, The Netherlands Van de Weijer PW.

[14] RobinsonD. A.. The mechanics of human saccadic eye movement. J Physiol Wiley Online Libr., 1964, 174(2):245-264

[15] BrockettR. W.. Lie theory and control systems defined on spheres. Siam J Appl. Math., 1973, 25(2):213-225

[16] AbrahamR., MarsdenJ. E.. Foundations of Mechanics, 1987 Reading Addison-Wesley

[17] Arnol’dV. I.. Mathematical Methods of Classical Mechanics, 1989 New York Springer Verlag

[18] SmaleS.. Topology and mechanics. i. Invent. Math., 1970, 10(4):305-331

[19] BulloF., MurrayR. M., SastriS.. Control on the sphere and reduced attitude stabilization. IFAC Proc Vol, 1995, 28(14):495-501

[20] MartinC., SchovanecL.. Muscle mechanics and dynamics of ocular motion. J. Math. Syst. Estimation Control, 1998, 8: 1-15

[21] MillerJ., RobinsonD.. A model of the mechanics of binocular alignment. J. Math. Syst. Estimation Control, 1984, 17: 436-470

[22] QuaiaC., OpticanL.. Commutative saccadic generator is sufficient ot control a 3D ocular plant with pulleys. J Neurophysiol, 1998, 79: 3197-3215

[23] DouglasTweed, HaslwanterT., FetterM.. Optimizing gaze control in three dimensions. Science, 1998, 281: 1363-1366

[24] Bijoy GhoshK., Indika WijayasingheB.. Dynamics of human head and eye rotations under Donders’ constraint. IEEE Trans. Autom. Control, 2012, 57(10):2478-2489

[25] GhoshB. K., WijayasingheI. B., KahagalageS. D.. Geometric approach to head/eye control. IEEE Access, 2014, 2: 316-332

[26] WijayasingheI., RuthsJ., BüttnerU., GhoshB. K., GlasauerS., KremmydaO., LiJr-S. Potential and optimal control of human head movement using Tait–Bryan parametrization. Automatica, 2014, 50(2):519-529

[27] HeppK.. On Listing’s law. Commun Math Phys, 1990, 132: 285-292

[28] HaslwanterT.. Mathematics of three dimensional eye rotations. Vis Res, 1995, 35: 1727-1739

[29] Van OpstalA. J.. 200 years Franciscus Cornelis Donders. Strabismus, 2018, 26(4):159-162

[30] TweedD. B., VilisT.. Implications of rotational kinematics for the oculomotor system in three dimensions. J. Neurophysiol, 1987, 58: 832-849

[31] FetterM., HaslwanterT., MisslischH., TweedD. B.. Kinematic principles of three-dimensional gaze control, 1997 Amsterdam Harwood Academic Publ

[32] FetterM., HaslwanterT., MisslischH., TweedD. B.. Visual, motor, or visuomotor?, 1997 Amsterdam Harwood Academic Publ

[33] CrawfordJ. D., GuittonD.. Visual-motor transformations required for accurate and kinematically correct saccades. J. Neurophysiol., 1997, 78: 1447-1467

[34] KlierE. M., MengH., AngelakiD. E.. Three-dimensional kinematics at the level of the oculomotor plant. J. Neurosci., 2006, 26: 2732-2737

[35] SchnabolkC., RaphanT.. Modelling three-dimensional velocity-to-position transformation in oculomotor control. J. Neurophysiol., 1994, 71: 623-638

[36] DemerJ. L., MillerJ. M., PoukensV., VintersH. V., GlasgowB. J.. Evidence for fibromuscular pulleys of the recti extraocular muscles. Invest Ophthalmol Vis. Sci., 1995, 36: 1125-1136

[37] A. A. Handzel, T. Flash, D. S. Touretzky, M. C. Mozer, M. E. Hasselmo, in Advances in Neural Information Pocessing Systems, 8. The geometry of eye rotations and Listing’s law, (1966), pp. 117–123.

[38] OpstalJ. V.. Three dimensional kinematics underlying gaze control, 1988 New York Springer Verlag

[39] BoothbyW. M.. An introduction to differentiable manifolds and Riemannian geometry, 2003 Orlando Academic Press Inc.

[40] do CarmoM. P.. Riemannian Geometry, 1993 Boston Birkhäuser

[41] JayarathneN.. Geometric Mechanics and Optimal Control Problems in Multi-sensor Visual Tracking. PhD thesis, 2020 Lubbock Texas Tech University

[42] DuarteN. F., RakovicM., TasevskiJ., CocoM. I., BillardA., Santos-VictorJ.. Action anticipation: Reading the intentions of humans and robots. IEEE Robot. Autom. Lett., 2018, 3(4):4132-4139

[43] MohammadY., OkadaS., NishidaT.. Autonomous development of gaze control for natural human-robot interaction. Int. Conf. Intell. User Interfaces Proc. IUI, 2010, 2(7):63-70

[44] KandaT., IshiguroH., OnoT.. Development and evaluation of an interactive humanoid robot “robovie”. Proc. 2002 IEEE Int. Conf. Robot. Autom. (Cat. No. 02CH37292), 2002, 2: 1848-1855

[45] AdmoniH. B.. Scassellati. Social eye gaze in human-robot interaction: A review. J. Human-Robot Interact., 2017, 6(1):25-63

[46] ZimmermanW. B. J.. Multiphysics modeling with finite element methods, 2006 Hackensack World Scientific Publishing Company Incorporated

[47] KuipersJ.. Quaternions and rotation sequences, 1998 Princeton Princeton University Press

[48] OkiT., GhoshB. K.. A transverse local feedback linearization approach to human head movement control. Control Theory Technol, 2017, 15(4):245-257

[49] O’ReillyO. M.. The dual Euler basis: constraints, potentials, and Lagrange’s equations in rigid body dynamics. ASME J. Appl. Mech., 2007, 74(2):1-10

[50] DunnF., ParberryI.. 3D math primer for graphics and game develepment, 2011 Boca Raton CRC Press, Taylor and Francis Group (Hardcover)

[51] O’ReillyO. M.. Intermediate dynamics for engineers: an unified treatment of Newton - Euler and Lagrangian mechanics, 2008 Cambridge Cambridge University Press (Hardcover)

[52] MurrayR. M., LiZ., SastryS. S.. A Mathematical Introduction to Robotic Manipulation, 1994 Boca Raton CRC Press

[53] IndikaW., EugenioA., BüttnerU., GhoshB. K., GlasauerS., KremmydaO.. Potential and optimal target fixating control of the human head/eye complex. IEEE Trans. Control Syst. Technol., 2015, 23(2):796-804

[54] KellerH.. Numerical Methods for Two-Point Boundary Value Problems, 1968 New York Blaisdell

[55] AscherU., MattheijR. M. M., RussellR. D.. Numerical Solution of Boundary Value Problems for Ordinary Differential Equations, 1988 Englewood Cliffs Prentice Hall

[56] GreengardL.. Spectral integration and two point boundary value problems. J. SIAM Numer. Anal., 1991, 28: 1071-1080

[57] ElnagarG., KazemiM. A., RazzaghiM.. The pseudospectral Legendre method for discretizing optimal control problems. IEEE Trans. Autom. Control, 1995, 40(10):1793-1796

[58] CanutoC., HussainiM. Y., QuarteroniA., ZangT. A.. Spectral Methods, 2006 Berlin Springer

[59] J. Ruths, B. K. Ghosh, Helmholtzian strategy to stay in focus with binocular vision. Proc. 17 ^{t h} Eur. Control Conf., 2380–2386 (2018).

[60] Y. Tang, M. Chen, C. Wang, L. Luo, J. Li, G. Lian, X. Zou, Recognition and localization methods for vision-based fruit picking robots: A review. Front. Plant Sci.11(510) (2020). https://doi.org/10.3389/fpls.2020.00510.

[61] I. B. Wijayasinghe, H. L. Miller, S. K. Das, N. L. Bugnariu, D. O. Popa, in Sensors for Next-Generation Robotics III, volume 9859. Human-like object tracking and gaze estimation with pkd android (International Society for Optics and Photonics, 2016), p. 985906.