FPGA-Based Real-Time Implementation of Closed-Loop Brain-Controlled Eye Movement System

Guodao Zhang , Yupeng Li , Yanjie Lu , Genfu Yang , Xiaotian Pan , Jing Wang

CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) : 1626 -1643.

PDF (4420KB)
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) :1626 -1643. DOI: 10.1049/cit2.70172
ORIGINAL RESEARCH
research-article
FPGA-Based Real-Time Implementation of Closed-Loop Brain-Controlled Eye Movement System
Author information +
History +
PDF (4420KB)

Abstract

This paper presents a comprehensive closed-loop brain-eye movement framework, spanning parametric modelling, dynamic analysis and real-time FPGA implementation. A simplified first-order oculomotor model was derived from the classical Rob inson formulation to preserve the essential neural-mechanical feedback structure while enabling efficient hardware realization. Through systematic parametric variations of neural and mechanical properties and multiple classes of brain-generated inputs (step, ramp, sinusoidal and noisy), the system exhibited stable closed-loop behaviour with fast convergence and low steady-state errors. Time-domain and phase-plane analyses confirmed that the eye accurately tracked diverse target trajectories across wide parameter ranges, with all equilibrium points classified as asymptotically stable. The discrete-time model was implemented on a Xilinx Virtex-4 FPGA using only adders, subtractors and shifters, completely avoiding multipliers. The design achieved a maximum clock frequency of 118 MHz, operated at 100 MHz with a closed-loop latency of nine cycles (90 ns), and consumed approximately 280 mW total power while utilising less than 4% of the device resources. To validate the system in a practical setting, a real-time hardware platform with a 3 × 3 target grid (nine fixation points) was developed. Experimental results demonstrated that the closed-loop brain-eye architecture reliably steered the eye towards each target with minimal tracking error and smooth dynamic response. Overall, this work bridges physiological oculomotor models and neuromorphic hardware, offering a scalable resource-efficient platform for real-time brain-eye closed-loop emulation and control.

Keywords

brain–eye interaction / closed-loop control / FPGA implementation / neuromorphic hardware / oculomotor system / real-time system

Cite this article

Download citation ▾
Guodao Zhang, Yupeng Li, Yanjie Lu, Genfu Yang, Xiaotian Pan, Jing Wang. FPGA-Based Real-Time Implementation of Closed-Loop Brain-Controlled Eye Movement System. CAAI Transactions on Intelligence Technology, 2026, 11 (5) : 1626-1643 DOI:10.1049/cit2.70172

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

P. Bonazzi, S. Bian, G. Lippolis, Y. Li, S. Sheik, and M. Magno, “ Retina: Low-Power Eye Tracking With Event Camera and Spiking Hardware,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2024), 5684-5692.

[2]

A. Tanwear, X. Liang, Y. Liu, et al., “ Spintronic Sensors Based on Magnetic Tunnel Junctions for Wireless Eye Movement Gesture Control,” IEEE Transactions on Biomedical Circuits and Systems 14, no. 6 (2020): 1299-1310, https://doi.org/10.1109/tbcas.2020.3027242.

[3]

T. Saquib and D. Terzopoulos, “ Biomimetic Oculomotor Control With Spiking Neural Networks,” Machine Vision and Applications 35, no. 1 (2024): 14, https://doi.org/10.1007/s00138-023-01494-z.

[4]

R. J. Alitappeh, A. John, B. Dias, A. J. van Opstal, and A. Bernardino, “ Emergence of Human Oculomotor Behavior in a Cable-Driven Biomimetic Robotic Eye Using Optimal Control,” IEEE Transactions on Cognitive and Developmental Systems 16, no. 4 (2024): 1546-1560, https://doi.org/10.1109/tcds.2024.3376072.

[5]

C. Wu, N. Chen, T. Sun, P. Tan, P. Wang, and G. Li, “ Synergizing Deepseek’s Artificial Intelligence Innovations With Brain-Computer Interfaces,” Brain-X 3, no. 2 (2025): e70035, https://doi.org/10.1002/brx2.70035.

[6]

D. A. Robinson, “ The Mechanics of Human Saccadic Eye Movement,” Journal of Physiology 174, no. 2 (1964): 245-264, https://doi.org/10.1113/jphysiol.1964.sp007485.

[7]

D. A. Robinson, “ Models of the Saccadic Eye Movement Control System,” Kybernetik 14, no. 2 (1973): 71-83, https://doi.org/10.1007/bf00288906.

[8]

M. E. Broucke, “ Model of the Oculomotor System Based on Adaptive Internal Models,” IFAC-PapersOnLine 53, no. 2 (2020): 16430-16437, https://doi.org/10.1016/j.ifacol.2020.12.729.

[9]

M. T. Islam, F. Hazzazi, A. Hoque, S. Haghiri, M. A. Chaudhary, and M. Ghanbarpour, “ FPGA Implementation of Nerve Cell Using Izhikevich Neuronal Model as Spike Generator (SG),” IEEE Access 12 (2023): 2303-2312, https://doi.org/10.1109/ACCESS.2023.3343156.

[10]

M. A. Chaudhary, F. Hazzazi, and M. Ghanbarpour, “ Digital System Implementation and Large-Scale Approach in Neuronal Modeling Using Adex Biological Neuron,” IEEE Transactions on Circuits and Systems II: Express Briefs 71, no. 5 (2024): 2814-2818, https://doi.org/10.1109/TCSII.2024.3353333.

[11]

M. Ghanbarpour, A. Naderi, B. Ghanbari, S. Haghiri, and A. Ahmadi, “ Digital Hardware Implementation of Morris-Lecar, Izhikevich, and Hodgkin-Huxley Neuron Models With High Accuracy and Low Resources,” IEEE Transactions on Circuits and Systems I: Regular Papers 70, no. 11 (2023): 4447-4455, https://doi.org/10.1109/TCSI.2023.3303941.

[12]

G. Ghanbarpour and M. Ghanbarpour, “ Low-Error, High-Speed, and Large-Scale Hardware Implementation of Retinal Photoreceptor Cells: Cone and Rod Cells,” AEU-International Journal of Electronics and Communications 185 (2024): 155456, https://doi.org/10.1016/j.aeue.2024.155456.

[13]

M. Ghanbarpour, S. Haghiri, F. Hazzazi, M. Assaad, M. A. Chaudhary, and A. Ahmadi, “ Investigation on Vision System: Digital FPGA Implementation in Case of Retina Rod Cells,” IEEE Transactions on Biomedical Circuits and Systems 18, no. 2 (2023): 299-307, https://doi.org/10.1109/TBCAS.2023.3323324.

[14]

M. Ghanbarpour, A. Naderi, S. Haghiri, and A. Ahmadi, “ An Efficient Digital Realization of Retinal Light Adaptation in Cone Photoreceptors,” IEEE Transactions on Circuits and Systems I: Regular Papers 65, no. 12 (December 2021): 5072-5080, https://doi.org/10.1109/tcsi.2021.3111202.

[15]

A. J. Leigh, M. Heidarpur, and M. Mirhassani, “ A Resource-Efficient and High-Accuracy CORDIC-Based Digital Implementation of the Hodgkin-Huxley Neuron,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems 31, no. 9 (2023): 1377-1388, https://doi.org/10.1109/tvlsi.2023.3296057.

[16]

J. Lu, F. Min, L. Gan, and S. Yang, “ Dynamic Analysis of Coupled Hindmarsh-Rose Neurons With Enhanced FPGA Implementation,” Chaos, Solitons & Fractals 191 (2025): 115889, https://doi.org/10.1016/j.chaos.2024.115889.

[17]

S. Nazari and S. Jamshidi, “ Efficient Digital Design of the Nonlinear Behavior of Hindmarsh-Rose Neuron Model in Large-Scale Neural Population,” Scientific Reports 13, no. 1 (2024): 3833, https://doi.org/10.1038/s41598-024-54525-8.

[18]

Y. Chen, X. Wu, H. Ba, et al., “ Digital Approach in Case of FPGA Realization of Quartic Neuron Model (QNM) Using Cost-Effective Mathematical Modifications,” IEEE Transactions on Circuits and Systems I: Regular Papers 71, no. 12 (2024): 6327-6335, https://doi.org/10.1109/tcsi.2024.3400399.

[19]

J. Sun, H. Chen, X. Ji, G. Zhang, C. Wang, and X. Miao, “ An Investigation on the Three-Dimensional Memristive Morris-Lecar Model With Magnetic Induction Effects: Simulation of Biological Behaviors and Cost-Effective Digital Circuit Implementation,” IEEE Transactions on Circuits and Systems I: Regular Papers 72, no. 6 (2025): 2690-2700, https://doi.org/10.1109/TCSI.2025.3534329.

[20]

S. Liu, B. Ni, X. Wang, et al., “ Efficient Low-Cost Design for High-Frequency, Accurate Hardware Implementation of Cardiac Purkinje Fiber Cells for Biomedical Engineering Approaches,” Engineering Science and Technology, an International Journal 60 (2024): 101888, https://doi.org/10.1016/j.jestch.2024.101888.

[21]

R. Chen, S. Xiang, D. Yu, et al., “ An Investigation on Inner Ear Operation in Sense of Hearing: High-Speed FPGA-based Circuits for Digital Realization of 2D Cochlea Modeling,” Alexandria Engineering Journal 118 (2025): 371-383, https://doi.org/10.1016/j.aej.2025.01.054.

[22]

S. Lin, S. Xiang, R. Chen, L. Li, Y. Ge, and X. Gao, “ Digital Low-Cost FPGA Implementation of Two-Coupled and Grid-Based Network of 2D Artificial Cochlea Using the Hopf Resonator Approach,” IEEE Transactions on Circuits and Systems I: Regular Papers 72, no. 5 (2024): 2178-2188, https://doi.org/10.1109/TCSI.2024.3494268.

[23]

V. Lopez, C. Montalvo, A. Varela, O. Cham, and E. Galvan, “ A Review of Design Methodologies and Evaluation Techniques for FPGA-Based Visual Object Tracking Systems,” International Journal of Combinatorial Optimization Problems and Informatics 15, no. 5 (2024), https://doi.org/10.61467/2007.1558.2024.v15i5.571.

[24]

P. Zhang, K. Jia, Y. Liu, Y. He, L. Chen, and C. Liu, “ RTL Design and FPGA Implementation of a Real-Time Eye Tracking System Using a Novel Projection Algorithm,” in Fifth International Conference on Signal Image Processing and Communication (ICSIPC 2025) (2025), 366-373.

[25]

N. Subramani, A. Varshini, A. Anil, and C. Thanusha, “ A Real-Time Human Eye Pupil Tracking System Using FPGA,” AIP Conference Proceedings (2025), https://doi.org/10.1063/5.0247030.

[26]

G. K. Kumar and B. Narayanam, “ FPGA Implementation of Eye Movement Detection Algorithm,” Microprocessors and Microsystems (2021): 104364, https://doi.org/10.1016/j.micpro.2021.104364.

[27]

C. Domingo, R. Stoean, and G. Caparros, “ FPGA Design and Hardware Implementation of a Convolutional Neural Network for Classification of Saccadic Eye Movements,” in Annals of the University of Craiova-Mathematics and Computer Science Series (2018), 263-274.

[28]

L. Manfredi, E. S. Maini, P. Dario, et al., “ Implementation of a Neurophysiological Model of Saccadic Eye Movements on an Anthropomorphic Robotic Head,” in 2006 6th IEEE-RAS International Conference on Humanoid Robots (2006), 438-443.

[29]

A. Al-Rahayfeh and M. Faezipour, “ Eye Tracking and Head Movement Detection: A State-of-Art Survey,” IEEE Journal of Translational Engineering in Health and Medicine (2013), https://doi.org/10.1109/JTEHM.2013.2289879.

[30]

F. Wadehn, T. Weber, D. J. Mack, T. Heldt, and H. A. Loeliger, “ Model-Based Separation, Detection, and Classification of Eye Movements,” IEEE Transactions on Biomedical Engineering 67, no. 2 (2020): 588-600, https://doi.org/10.1109/tbme.2019.2918986.

[31]

D. A. Robinson, “ The Use of Control Systems Analysis in the Neurophysiology of Eye Movements,” Annual Review of Neuroscience 4, no. 1 (1981): 463-503, https://doi.org/10.1146/annurev.ne.04.030181.002335.

[32]

P. A. Sylvestre and K. E. Cullen, “ Quantitative Analysis of Abducens Neuron Discharge Dynamics During Saccadic and Slow Eye Movements,” Journal of Neurophysiology 82, no. 5 (1999): 2612-2632, https://doi.org/10.1152/jn.1999.82.5.2612.

[33]

V. Radisavljevic-Gajic, “ Dynamics of Eye Movements Under Time Varying Stimuli,” Journal of Eye Movement Research 11, no. 1 (2018): 6, https://doi.org/10.16910/jemr.11.1.6.

[34]

M. Heidarpur, A. Ahmadi, M. Ahmadi, and M. Rahimi Azghadi, “ CORDIC-SNN: On-Fpga STDP Learning With Izhikevich Neurons,” in IEEE Transactions on Circuits and Systems-I (2019), 2651-2661.

[35]

M. Heidarpur, P. Khosravifar, A. Ahmadi, and M. Ahmadi, “ CORDIC-Astrocyte: Tripartite Glutamate-IP3-Ca2+ Interaction Dynamics on FPGA,” IEEE Transactions on Biomedical Circuits and Systems 14, no. 1 (February 2020): 36-47, https://doi.org/10.1109/tbcas.2019.2953631.

[36]

H. Soleimani and E. M. Drakakis, “ An Efficient and Reconfigurable Synchronous Neuron Model,” IEEE Transactions on Circuits and Systems II: Express Briefs (2017), https://doi.org/10.1109/TCSII.2017.2697826.

[37]

Y. Shinji, H. Okuno, and Y. Hirata, “ Artificial Cerebellum on FPGA: Realistic Real-Time Cerebellar Spiking Neural Network Model Capable of Real-World Adaptive Motor Control,” Frontiers in Neuroscience (2024), https://doi.org/10.1109/ICASSP.2012.6288193.

[38]

T. Ando, V. G. Moshnyaga, and K. Hashimoto, “ A Low-Power FPGA Implementation of Eye Tracking,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2012), 1573-1576.

[39]

S. Yang, J. Wang, N. Zhang, B. Deng, Y. Pang, and M. R. Azghadi, “ CerebelluMorphic: Large-Scale Neuromorphic Model and Architecture for Supervised Motor Learning,” IEEE Transactions on Neural Networks and Learning Systems 33, no. 9 (2022): 4398-4412, https://doi.org/10.1109/TNNLS.2021.3057070.

[40]

B. Kowalski, X. Huang, S. Steven, and A. Dubra, “ Hybrid FPGA-CPU Pupil Tracker,” Biomedical Optics Express 12, no. 10 (2021): 6496-6513, https://doi.org/10.1364/BOE.433766.

[41]

D. Das, M. H. Chowdhury, A. Chowdhury, K. Hasan, Q. D. Hossain, and R. C. C. Cheung, “ Application Specific Reconfigurable Processor for Eyeblink Detection From Dual-Channel EOG Signal,” Journal of Low Power Electronics and Applications 13, no. 4 (2023): 61, https://doi.org/10.3390/jlpea13040061.

[42]

A. M. Z. Khaki, S. Yang, H. Kim, A. Singh, and B.-G. Lee, “ A Resource-Efficient Multi-Function Embedded Eye Tracker System Implemented on FPGA,” IEEE Access 11 (2023): 142931-142939, https://doi.org/10.1109/ACCESS.2023.3342839.

[43]

K. Dohi, Y. Hatanaka, K. Negi, Y. Shibata, and K. Oguri, “ Deep-Pipelined FPGA Implementation of Ellipse Estimation for Eye Tracking,” in 22nd International Conference on Field Programmable Logic and Applications (FPL) (2012), 458-463.

[44]

B. Zhang, Y. Gao, J. Li, and H. K.-H. So, “ Co-Designing a Sub-Millisecond Latency Event-Based Eye Tracking System With Submanifold Sparse CNN,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2024), 5771-5779.

[45]

T. Han, A. Li, Q. Chen, and C. Gao, “ JaneEye: A 12-nm 2K-FPS 18.9-µJ/Frame Event-Based Eye Tracking Accelerator,” in 2026 31st Asia and South Pacific Design Automation Conference (ASP-DAC) (IEEE, 2026), 170-176.

[46]

C. A. Ruiz-Beltrán, A. Romero-Garcés, M. González-García, R. Marfil, and A. Bandera, “ FPGA-Based CNN for Eye Detection in an Iris Recognition at a Distance System,” Electronics 12, no. 22 (2023): 4713, https://doi.org/10.3390/electronics12224713.

[47]

H. Wang, W. Liu, and Q. Sun, “ Process Only Where You Look: Hardware and Algorithm Co-Optimization for Efficient Gaze-Tracked Foveated Rendering in Virtual Reality,” preprint, arXiv (2024).

[48]

S. Yang, J. Wang, X. Hao, et al., “ BiCoSS: Toward Large-Scale Cognition Brain With Multigranular Neuromorphic Architecture,” IEEE Transactions on Neural Networks and Learning Systems 33, no. 7 (July 2022): 2801-2815, https://doi.org/10.1109/tnnls.2020.3045492.

[49]

S. Yang, B. Deng, J. Wang, et al., “ Scalable Digital Neuromorphic Architecture for Large-Scale Biophysically Meaningful Neural Network With Multi-Compartment Neurons,” IEEE Transactions on Neural Networks and Learning Systems 31, no. 1 (January 2020): 148-162, https://doi.org/10.1109/tnnls.2019.2899936.

PDF (4420KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉