Multimodal feature extraction and fusion for determining RGP lens specification base-curve through Pentacam images

Abstract

Aim: Patients diagnosed with irregular astigmatism often require specific methods of vision correction. Among these, the use of a rigid gas permeable (RGP) lens is considered one of the most effective treatment approaches. This study aims to propose a new automated method for accurate RGP lens base-curve detection.

Methods: A multi-modal feature fusion approach was developed based on Pentacam images, incorporating image processing and machine learning techniques. Four types of features were extracted from the images and integrated through a serial feature fusion mechanism. The fused features were then evaluated using a multi-layered perceptron (MLP) network. Specifically, the features included: (1) middle-layer outputs of a convolutional autoencoder (CAE) applied to RGB map combinations; (2) ratios of colored areas in the front cornea map; (3) a feature vector from cornea front parameters; and (4) the radius of the reference sphere/ellipse in the front elevation map.

Results: Evaluations were performed on a manually labeled dataset. The proposed method achieved a mean squared error (MSE) of 0.005 and a coefficient of determination of 0.79, demonstrating improved accuracy compared to existing techniques.

Conclusions: The proposed multi-modal feature fusion technique provides a reliable and accurate solution for RGP lens base-curve detection. This approach reduces manual intervention in lens fitting and represents a significant step toward automated base-curve determination.

Keywords

Pentacam four refractive maps / multi-modal feature fusion / convolutional autoencoder / rigid gas permeable (RGP) lens base-curve detection

References

[1] Rigid Gas Permeable Lenses [Internet]. Bausch & Lomb Incorporated; c2010 [cited 2025 Oct 20]. Available from:https://bauschgp.com/wp-content/uploads/2014/10/RGP-MATERIAL-GUIDE.pdf

[2] Ortiz-Toquero S, Rodriguez G, de Juan V, Martin R. Gas permeable contact lens fitting in keratoconus: Comparison of different guidelines to back optic zone radius calculations. Indian J Ophthalmol. 2019; 67:1410-6.

[3] Asiri N, Hussain M, Al Adel F, Alzaidi N. Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey. Artif Intell Med. 2019; 99:101701.

[4] Hashemi S, Veisi H, Jafarzadehpur E, Rahmani R, Heshmati Z. An image processing approach for rigid gas-permeable lens base-curve identification. SIViP. 2020; 14:971-9.

[5] Hashemi S, Veisi H, Jafarzadehpur E, Rahmani R, Heshmati Z. A CCA Approach for Multiview Analysis to Detect Rigid Gas Permeable Lens Base Curve. Proceedings of 2019 IEEE Western New York Image and Signal Processing Workshop (WNYISPW); 2019 Oct 4; Rochester, USA. IEEE; 2019. pp. 1-5.

[6] Hashemi S, Veisi H, Jafarzadehpur E, Rahmani R, Heshmati Z. Multi-view deep learning for rigid gas permeable lens base curve fitting based on Pentacam images. Med Biol Eng Comput. 2020; 58:1467-82.

[7] Ebrahimi L, Veisi H, Hashemi S, Jafarzadepour E. The fusion of multi-view features in Pentacam four refractive maps by using neural network to determine Rigid Gas Permeable (RGP) lens properties. Proceedings of 26th Computer Society of Iran Computer Conference (CSICC); 2021 Mar 3-4; Tehran, Iran.

[8] Zhang B, Zhou J. Multi-feature representation for burn depth classification via burn images. Artif Intell Med. 2021; 118:102128.

[9] Belin MW, Khachikian SS. Keratoconus/ectasia detection with the oculus pentacam: Belin/Ambrósio enhanced ectasia display. Highlights Ophthalmol. 2007; 35:5-12.

[10] Liu Y, Chen X, Wang Z, Wang ZJ, Ward RK, Wang X. Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf Fusion. 2018; 42:158-73.

[11] Ker J, Wang L, Rao J, Lim T. Deep learning applications in medical image analysis. IEEE Access. 2018; 6:9375-89.

[12] Abdelmotaal H, Mostafa MM, Mostafa ANR, Mohamed AA, Abdelazeem K. Classification of Color-Coded Scheimpflug Camera Corneal Tomography Images Using Deep Learning. Transl Vis Sci Technol. 2020; 9:30.

[13] Li X, Zhang G, Huang HH, Wang Z, Zheng W. Performance analysis of GPU-based convolutional neural networks. Proceedings of 2016 45th International Conference on Parallel Processing (ICPP); 2016 Aug 16-19; Philadelphia, USA. IEEE;2016. pp. 67-76.

[14] Al-Waisy AS, Qahwaji R, Ipson S, Al-Fahdawi S, Nagem TAM. A multi-biometric iris recognition system based on a deep learning approach. Pattern Anal Applic. 2018; 21:783-802.

[15] Rayhan F, Galata A, Cootes TF. ChoiceNet: CNN learning through choice of multiple feature map representations. Pattern Anal. Applic. 2021; 24:1757-67.

[16] Pulgar FJ, Charte F, Rivera AJ, del Jesus MJ. Choosing the proper autoencoder for feature fusion based on data complexity and classifiers: Analysis, tips and guidelines. Inf Fusion. 2020; 54:44-60.

[17] Huang H, Ma Z, Zhang G, Wu H. Dimensionality reduction based on multi-local linear regression and global subspace projection distance minimum. Pattern Anal Applic. 2021; 24:1713-30.

[18] Castanedo F. A review of data fusion techniques. ScientificWorldJournal. 2013; 2013:704504.

[19] Haghighat M, Abdel-Mottaleb M, Alhalabi W. Discriminant Correlation Analysis: Real-Time Feature Level Fusion for Multimodal Biometric Recognition. IEEE Trans Inf Forensics Secur. 2016; 11:1984-96.

[20] Sun QS, Zeng SG, Liu Y, Heng PA, Xia DS. A new method of feature fusion and its application in image recognition. Pattern Recognit. 2005; 38:2437-48.

[21] Ceccarelli F, Mahmoud M. Multimodal temporal machine learning for Bipolar Disorder and Depression Recognition. Pattern Anal. Applic. 2022; 25:493-504.

[22] Li Y, Yang M, Zhang Z. A Survey of Multi-View Representation Learning. IEEE Trans Knowl Data Eng. 2019; 31:1863-83.

[23] Ortiz-Toquero S, Rodriguez G, de Juan V, Martin R. Rigid Gas Permeable Contact Lens Fitting Using New Software in Keratoconic Eyes. Optom Vis Sci. 2016; 93:286-92.

[24] Lentes de contacto gas permeable [Internet].Córneas irregulars; [cited Year Month Day]. Available from:https://www.conoptica.es/images/documentos/productos/KAKC-N_F_I_PRO.pdf

[25] Sorbara L. Correction of Keratoconus with GP Contact Lenses. In: Fonn D, Woods C, Sivak A, editors. Waterloo: Centre for Contact Lens Research; 2010.

[26] Rajabi MT, Mohajernezhad-Fard Z, Naseri SK, Jafari F, Doostdar A, Zarrinbakhsh P, et al. Rigid contact lens fitting based on keratometry readings in keratoconus patients: predicting formula. Int J Ophthalmol. 2011; 4:525-8.

[27] Romero-Jiménez M, Santodomingo-Rubido J, González-Méijome JM. An assessment of the optimal lens fit rate in keratoconus subjects using three-point-touch and apical touch fitting approaches with the rose K2 lens. Eye Contact Lens. 2013; 39:269-72.

[28] GAS Permeable (GP) Lens Fitting Guide [Internet]. Bausch + Lomb; c2025 [cited2025 Oct 20]. Available from:http://bauschgp.com/wp-content/uploads/2014/10/GP-FITTING-GUIDE-2013.pdf

[29] Mannis MJ, Zadnik K, Coral-Ghanem C, Kara-José N. Contact Lenses in Ophthalmic Practice. New Yoerk: Springer; 2004.

[30] Your Trusted Source for Quality Custom Contact Lenses. [Internet].Valleycontax; [cited 2025 Oct 29]. Available from:https://www.valleycontax.com/

[31] Ortiz-Toquero S, Rodriguez G, de Juan V, Martin R. New web-based algorithm to improve rigid gas permeable contact lens fitting in keratoconus. Cont Lens Anterior Eye. 2017; 40:143-50.

[32] Ghaderi M, Sharifi A, Jafarzadeh E. Detection of irregular astigmatism through Pentacam images using mixture of multilayer perceptron experts. Proceedings of 1st International Conference on New Perspective in Electrical and Computer Engineering; 2016 Sep 9; Tehran, Iran. 2016. pp. 1-9.

[33] Risser G, Mechleb N, Muselier A, Gatinel D, Zéboulon P. Novel deep learning approach to estimate rigid gas permeable contact lens base curve for keratoconus fitting. Cont Lens Anterior Eye. 2023; 46:102063.

[34] Hashemi H, Mehravaran S. Day to Day Clinically Relevant Corneal Elevation, Thickness, and Curvature Parameters Using the Orbscan II Scanning Slit Topographer and the Pentacam Scheimpflug Imaging Device. Middle East Afr J Ophthalmol. 2010; 17:44-55.

[35] Belin MW, Khachikian SS. New Advances and Technology with Pentacam. In: Keratoconus/Ectasia Detection with the Oculus Pentacam: Belin/Ambrósio Enhanced Ectasia Display. Wetzlar: OCULUS Optikgeräte GmbH; 2008. pp. 3-7.

[36] Yang C, Guan N. Correlation maximization machine for multi-modalities multiclass classification. Pattern Ana. Applic. 2020; 23:349-58.