A tactile glove for object recognition based on palmar pressure and joint bending strain sensing

Abstract

With the rapid development of flexible electronics, the tactile systems for object recognition are becoming increasingly delicate. This paper presents the design of a tactile glove for object recognition, integrating 243 palm pressure units and 126 finger joint strain units that are implemented by piezoresistive Velostat film. The palm pressure and joint bending strain data from the glove were collected using a two-dimensional resistance array scanning circuit and further converted into tactile images with a resolution of 32×32. To verify the effect of tactile data types on recognition precision, three datasets of tactile images were respectively built by palm pressure data, joint bending strain data, and a tactile data combing of both palm pressure and joint bending strain. An improved residual convolutional neural network (CNN) model, SP-ResNet, was developed by light-weighting ResNet-18 to classify these tactile images. Experimental results show that the data collection method combining palm pressure and joint bending strain demonstrates a 4.33% improvement in recognition precision compared to the best results obtained by using only palm pressure or joint bending strain. The recognition precision of 95.50% for 16 objects can be achieved by the presented tactile glove with SP-ResNet of less computation cost. The presented tactile system can serve as a sensing platform for intelligent prosthetics and robot grippers.

Keywords

tactile glove / object recognition / Velostat / joint bending strain sensors / palmar pressure sensors / convolutional neural network

References

[1] ZHU M L, HE T, LEE C K. Technologies toward next generation human machine interfaces: from machine learning enhanced tactile sensing to neuromorphic sensory systems. Applied Physics Reviews, 2020, 7(3): 031305.

[2] KOSCH T, KAROLUS J, ZAGERMANN J, et al. A survey on measuring cognitive workload in human-computer interaction. ACM Computing Surveys, 2023, 55(13s): 1-39.

[3] XIONG Y X, SHEN Y K, TIAN L, et al. A flexible, ultra-highly sensitive and stable capacitive pressure sensor with convex microarrays for motion and health monitoring. Nano Energy, 2020, 70: 104436.

[4] CHEN J, LI L, ZHU Z, et al. Bioinspired design of highly sensitive flexible tactile sensors for wearable healthcare monitoring. Materials Today Chemistry, 2022, 23: 100718.

[5] SONG Z, FAN X Y, DONG J Y, et al. The third-person perspective full-body illusion induced by visual-tactile stimulation in virtual reality for stroke patients. Consciousness and Cognition, 2023, 115: 103578.

[6] LIAO X Q, SONG W T, ZHANG X Y, et al. Hetero-contact microstructure to program discerning tactile interactions for virtual reality. Nano Energy, 2019, 60: 127-136.

[7] LEE K T, CHEE P S, LIM E H, et al. Artificial intelligence (AI)-driven smart glove for object recognition application. Materials Today: Proceedings, 2022, 64: 1563-1568.

[8] ZHANG X X, LI S B, YANG J, et al. Tactile perception object recognition based on an improved support vector machine. Micromachines, 2022, 13(9): 1538.

[9] HE T X, YU S C, WANG Z Y, et al. From data quality to model quality: an exploratory study on deep learning//The 11th Asia-Pacific Symposium on Internetware, September 17-18, 2019, Fukuoka, Japan. New York: ACM, 2019: 1-6.

[10] SUNDARAM S, KELLNHOFER P, LI Y Z, et al. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 2019, 569(7758): 698-702.

[11] QIU Y, WANG Z Q, ZHU P C, et al. A multisensory-feedback tactile glove with dense coverage of sensing arrays for object recognition. Chemical Engineering Journal, 2023, 455: 140890.

[12] LU X, SUN D, YIN H B, et al. 3-D tactile-based object recognition for robot hands using force-sensitive and bend sensor arrays. IEEE Transactions on Cognitive and Developmental Systems, 2023, 15(4): 1645-1655.

[13] DZEDZICKIS A, SUTINYS E, BUCINSKAS V, et al. Polyethylene-carbon composite (velostat®) based tactile sensor. Polymers, 2020, 12(12): 2905.

[14] CHEN S J, LI M, HUANG Y K, et al. Matrix-addressed flexible capacitive pressure sensor with suppressed crosstalk for artificial electronic skin. IEEE Transactions on Electron Devices, 2020, 67(7): 2940-2944.

[15] WU J F. Scanning approaches of 2-D resistive sensor arrays: a review. IEEE Sensors Journal, 2017, 17(4): 914-925.

[16] PRUTCHI D, ARCAN M. Dynamic contact stress analysis using a compliant sensor array. Measurement, 1993, 11(3): 197-210.

[17] TAKEI K, TAKAHASHI T, HO J C, et al. Nanowire active-matrix circuitry for low-voltage macroscale artificial skin. Nature Materials, 2010, 9(10): 821-826.

[18] VIDAL-VERDÚ F, OBALLE-PEINADO Ó, SÁNCHEZ-DURÁN J A, et al. Three realizations and comparison of hardware for piezoresistive tactile sensors. Sensors, 2011, 11(3): 3249-3266.

[19] DEMKOV Y N, OSTROVSKII V N. Zero-range potentials and their applications in atomic physics. Berlin: Springer Science & Business Media, 2013.

[20] SAXENA R S, BHAN R K, SAINI N K, et al. Virtual ground technique for crosstalk suppression in networked resistive sensors. IEEE Sensors Journal, 2011, 11(2): 432-433.

[21] SUPRAPTO S S, SETIAWAN A W, ZAKARIA H, et al. Low-cost pressure sensor matrix using velostat//2017 5th International Conference on Instrumentation, Communications, Information Technology, and Biomedical Engineering, November 6-7, 2017, Bandung, Indonesia. New York: IEEE, 2017: 137-140.

[22] LIN W K, WANG B, PENG G X, et al. Skin-inspired piezoelectric tactile sensor array with crosstalk-free row+column electrodes for spatiotemporally distinguishing diverse stimuli. Advanced Science, 2021, 8(3): 2002817.

[23] LI Z W, LIU F, YANG W J, et al. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 6999-7019.

[24] SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1): 60.

[25] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition, June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 770-778.

[26] HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3////2019 IEEE/CVF International Conference on Computer Vision, October 27-November 2, 2019, Seoul, Korea. New York: IEEE, 2019: 1314-1324.

[27] XIE C, ZHU H Y, FEI Y Q. Deep coordinate attention network for single image super-resolution. IET Image Processing, 2022, 16(1): 273-284.

[28] HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 20-25, 2021, Nashville, TN, USA. New York: IEEE, 2021: 13708-13717.

[29] CHEN J R, KAO S H, HE H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 17-24, 2023, Vancouver, BC, Canada. New York: IEEE, 2023: 12021-12031.

[30] DENIL M, SHAKIBI B, DINH L, et al. Predicting parameters in deep learning//27th International Conference on Neural Information Processing Systems, December 5-8, 2013, Lake Tahoe, Nevada, USA, New York: Curran Associates Inc, 2013: 2148-2156.

[31] KINGMA D P, WELLING M. Auto-encoding variational bayes.2013.

[32] CHARBUTY B, ABDULAZEEZ A. Classification based on decision tree algorithm for machine learning. Journal of Applied Science and Technology Trends, 2021, 2(1): 20-28.

[33] HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. 2017.