Innovative solutions for convolutional neural network performance: A TRIZ-based reverse engineering approach

Abstract

Convolutional neural networks (CNNs) are widely used in computer vision for tasks like image classification and detection. These models work well when the number of image classes is small, but as the number of classes increases, accuracy tends to drop due to overfitting. There are several methods to address this issue, such as data augmentation, preprocessing, class weighting, transfer learning, and adjusting technical parameters. This study introduces a novel approach utilizing the theory of inventive problem-solving (TRIZ) methodology to systematically analyze and enhance these existing methods. Using reverse engineering, we deconstructed current solutions and aligned them with TRIZ principles to propose more innovative and effective approaches for improving CNN performance. The results show that TRIZ provides a structured and creative framework for solving accuracy decline issues in CNN models, offering the potential for broader applications in other machine learning architectures.

Keywords

Convolutional Neural Network / Image Classification / Reverse Engineering / Theory of Inventive Problem Solving

References

[1] Alomar, K., Aysel, H.I., & Cai, X. (2023). Data augmentation in classification and segmentation: A survey and new strategies. Journal Imaging, 9, 46.

[2] Atasever, S., Azginoğlu, N., Terzi, D.S., & Terzi. (2023). A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning. Clinical Imaging, 94, 18-41.

[3] Brad, S., & Brad, E. (2023). Using TRIZ to handle small datasets in artificial intelligence. In: Acta Technica Napocensis Series Applied Mathematics, Mechanics, and Engineering, Technical University of Cluj-Napoca, Romania, p66.

[4] Gadd, K. (2011). TRIZ for Engineers: Enabling Inventive Problem Solving. John Wiley and Sons, United States.

[5] Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2019). A survey on deep learning based face recognition. Computer Vision and Image Understanding, 189, 102805.

[6] Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., et al. (2017). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 36, 61-78.

[7] Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.

[8] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.

[9] Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., et al. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60-88.

[10] Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.

[11] Perez, L., & Wang, J. (2017). The Effectiveness of Data Augmentation in Image Classification Using Deep Learning; [arXiv preprint].

[12] Sheu, D.D., & Lee, H.K. (2011). A proposed process for systematic innovation. International Journal of Production Research, 49(3), 847-868.

[13] Shorten, C., & Khoshgoftaar, T.M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 60.

[14] Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. (2018). A survey on deep transfer learning. In: International Conference on Artificial Neural Networks. Springer, Cham, p270-279.

[15] Tribuana, D., & Arda, A.L. (2024). Image preprocessing approaches toward better learning performance with CNN. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 8 (1), 1-9.

[16] Yang, Z., Sinnott, R.O., Bailey, J., & Ke, Q. (2023). A survey of automated data augmentation algorithms for deep learning-based image classification tasks. Knowledge and Information Systems, 65, 2805-2861.