A Real-Time Detection Method for Fashion Necklines Based on Deep Learning

Abstract

Accurate detection of fashion design attributes is essential for trend analyses and recommendation systems. Among these attributes, the neckline style plays a key role in shaping garment aesthetics. However, the presence of complex backgrounds and varied body postures in real-world fashion images presents challenges for reliable neckline detection. To address this problem, this research builds a comprehensive fashion neckline database from online shop images and proposes an efficient fashion neckline detection model based on the YOLOv8 architecture(FN-YOLO). First, the proposed model incorporates a BiFormer attention mechanism into the backbone, enhancing its feature extraction capability. Second, a lightweight multi-level asymmetry detector head(LADH) is designed to replace the original head, effectively reducing the computational complexity and accelerating the detection speed. Last, the original loss function is replaced with Wise-IoU, which improves the localization accuracy of the detection box. The experimental results demonstrate that FN-YOLO achieves a mean average precision(mAP) of 81.7%, showing an absolute improvement of 3.9% over the original YOLOv8 model, and a detection speed of 215.6 frame/s, confirming its suitability for real-time applications in fashion neckline detection.

Keywords

fashion neckline detection / deep learning / detection and classification / real time / YOLOv8

References

[1] KUMAR A. From mass customization to mass personalization:a strategic transformation[J]. International Journal of Flexible Manufacturing Systems, 2007, 19(4):533-547.

[2] LEE S H N, CHOW P S. Investigating consumer attitudes and intentions toward online fashion renting retailing[J]. Journal of Retailing and Consumer Services, 2020,52:101892.

[3] NAM Y R, KIM D E. A study on the comparison of 3D virtual clothing and real clothing by neckline type[J]. Fashion & Textile Research Journal, 2021, 23(2):247-260.

[4] SHOUKAT S. Now and then:the neckline history of women[J]. American Scientific Research Journal for Engineering,Technology,and Sciences, 2016, 26(2):33-52.

[5] TERVEN J, CóRDOVA-ESPARZA D M, ROMERO-GONZáLEZ J A. A comprehensive review of YOLO architectures in computer vision:from YOLOv1 to YOLOv8 and YOLO-NAS[J]. Machine Learning and Knowledge Extraction, 2023, 5(4):1680-1716.

[6] DONATI L, IOTTI E, MORDONINI G, et al. Fashion product classification through deep learning and computer vision[J]. Applied Sciences, 2019, 9(7):1385.

[7] LAO B, JAGADEESH K.Convolutional neural networks for fashion classification and object detection[C]//The 2015 Chinese Conference on Computer Vision (CCCV). Berlin: Springer, 2015:120-129.

[8] WANG W G, XU Y L, SHEN J B, et al. Attentive fashion grammar network for fashion landmark detection and clothing category classification[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE,2018:4271-4280.

[9] LI D, WAN X F, WANG J, et al. Clothing style recognition approach based on the curvature feature points on the contour[J]. Journal of Donghua University (Natural Science), 2018, 44(1):87-92. (in Chinese)

[10] SEO Y, SHIN K S. Hierarchical convolutional neural networks for fashion image classification[J]. Expert Systems with Applications, 2019,116:328-339.

[11] SUN G L, WU X, CHEN H H, et al. Clothing style recognition using fashion attribute detection[C]//The 8th International Conference on Mobile Multimedia Communications. New York: ACM,2015:145-148.

[12] YUE X D, ZHANG C, FUJITA H, et al. Clothing fashion style recognition with design issue graph[J]. Applied Intelligence, 2021, 51(6):3548-3560.

[13] TANG Z, GE Y M. CNN model optimization and intelligent balance model for material demand forecast[J]. International Journal of System Assurance Engineering and Management, 2022, 13(3):978-986.

[14] GUO S, HUANG W L, ZHANG X, et al. The iMaterialist fashion attribute dataset[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). New York: IEEE, 2019:3113-3116.

[15] LIU Z W, LUO P, QIU S, et al. DeepFashion:powering robust clothes recognition and retrieval with rich annotations[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016:1096-1104.

[16] GE Y Y, ZHANG R M, WANG X G, et al. DeepFashion2:a versatile benchmark for detection,pose estimation,segmentation and re-identification of clothing images[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2019:5332-5340.

[17] CHEN H Z, GALLAGHER A, GIROD B. Describing clothing by semantic attributes[C]//Computer Vision-ECCV 2012. Berlin: Springer Berlin Heidelberg, 2012:609-623.

[18] LIU R, JOSEPH A A, XIN M M, et al. Personalized clothing prediction algorithm based on multi-modal feature fusion[J]. International Journal of Engineering and Technology Innovation, 2024, 14(2):216-230.

[19] ZHU R H, XIN B J, DENG N, et al. Review of fabric defect detection based on computer vision[J]. Journal of Donghua University (English Edition), 2023, 40(1):18-26.

[20] NANDYAL S, TENGLI N S.An efficient framework for classifying the clothing items based on fashion and fabric of the images[C]//2020 IEEE International Conference on Technology,Engineering,Management for Societal Impact Using Marketing,Entrepreneurship and Talent (TEMSMET). New York: IEEE, 2020:1-5.

[21] PENG T, ZHOU X Z, LIU J P, et al. A textile fabric classification framework through small motions in videos[J]. Multimedia Tools and Applications, 2021, 80(5):7567-7580.

[22] AMIN M S, WANG C B, JABEEN S. Fashion sub-categories and attributes prediction model using deep learning[J]. The Visual Computer, 2023, 39(9):3851-3864.

[23] ZOU X X, KONG X H, WONG W, et al.FashionAI:a hierarchical dataset for fashion understanding[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New York: IEEE, 2019:296-304.

[24] XU Z B, ZHANG L, ZHANG Y H, Research on clothing collar types based on complex network extraction and support vector machine classification[J]. Journal of Textile Research, 2021, 42(6):146-152. (in Chinese)

[25] LU D, WENG Q. A survey of image classification methods and techniques for improving classification performance[J]. International Journal of Remote Sensing, 2007, 28(5):823-870.

[26] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.

[27] RAWAT W, WANG Z H. Deep convolutional neural networks for image classification:a comprehensive review[J]. Neural Computation, 2017, 29(9):2352-2449.

[28] NOH S K. Recycled clothing classification system using intelligent IoT and deep learning with AlexNet[J]. Computational Intelligence and Neuroscience, 2021, 2021(1):5544784.

[29] SINGH M, DALMIA S, RANJAN R K, et al. Dress pattern classification using ResNet based convolutional neural networks[C]//Information Systems and Management Science. Cham: Springer International Publishing, 2023:91-103.

[30] JIANG P Y, ERGU D J, LIU F Y, et al. A review of YOLO algorithm developments[J]. Procedia Computer Science, 2022,199:1066-1073.

[31] LIU W, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]//Computer Vision-ECCV 2016. Cham: Springer International Publishing, 2016:21-37.

[32] CHEN C L, ZHENG Z Y, XU T Y, et al. YOLO-based UAV technology:a review of the research and its applications[J]. Drones, 2023, 7(3):190.

[33] DIWAN T, ANIRUDH G, TEMBHURNE J V. Object detection using YOLO:challenges,architectural successors,datasets and applications[J]. Multimedia Tools and Applications, 2023, 82(6):9243-9275.

[34] CHUNG M A, LIN Y J, LIN C W. YOLO-SLD:an attention mechanism-improved YOLO for license plate detection[J]. IEEE Access, 2024,12:89035-89045.

[35] THWE Y, JONGSAWAT N, TUNGKASTHAN A. A semi-supervised learning approach for automatic detection and fashion product category prediction with small training dataset using FC-YOLOv4[J]. Applied Sciences, 2022, 12(16):8068.

[36] LEE C H, LIN C W. A two-phase fashion apparel detection method based on YOLOv4[J]. Applied Sciences, 2021, 11(9):3782.

[37] WANG C Y, MARK LIAO H Y, WU Y H, et al.CSPNet:a new backbone that can enhance learning capability of CNN[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New York: IEEE, 2020:1571-1580.

[38] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.

[39] LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018:8759-8768.

[40] ZHENG Z H, WANG P, REN D W, et al. Enhancing geometric factors in model learning and inference for object detection and instance segmentation[J]. IEEE Transactions on Cybernetics, 2022, 52(8):8574-8586.

[41] LI X, WANG W, WU L, et al. Generalized focal loss:learning qualified and distributed bounding boxes for dense object detection[J]. Advances in Neural Information Processing Systems, 2020,33:21002-21012.

[42] REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized intersection over union:a metric and a loss for bounding box regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2019:658-666.

[43] WOO S, PARK J, LEE J Y, et al. CBAM:convolutional block attention module[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018:3-19.

[44] YANG L, ZHANG R Y, LI L, et al. SimAM:a simple,parameter-free attention module for convolutional neural networks[C]//The International Conference on Machine Learning. New York: PMLR, 2021:11863-11874.

[45] ZHANG Q L, YANG Y B. SA-net:shuffle attention for deep convolutional neural networks[C]//2021 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). New York: IEEE, 2021:2235-2239.

[46] ZHU L, WANG X J, KE Z H, et al. BiFormer:vision transformer with bi-level routing attention[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2023:10323-10333.

[47] LIU S T, HUANG D, WANG Y H. Learning spatial fusion for single-shot object detection[EB/OL].(2019-11-21)[2024-11-14]. https://arxiv.org/abs/1911.09516.

[48] ZHANG J R, CHEN Z H, YAN G X, et al. Faster and lightweight:an improved YOLOv5 object detector for remote sensing images[J]. Remote Sensing, 2023, 15(20):4974.

[49] GUO Y R, SHEN Q, ZHANG S Y, et al. An airborne target recognition model based on SPD,PConv and LADH detection heads[C]//Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023). Singapore: Springer Nature Singapore, 2024:325-337.

[50] HUA B S, TRAN M K, YEUNG S K.Pointwise convolutional neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018:984-993.

[51] DU S J, ZHANG B F, ZHANG P, et al.An improved bounding box regression loss function based on CIOU loss for multi-scale object detection[C]//2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML). New York: IEEE, 2021:92-98.

[52] TONG Z J, CHEN Y H, XU Z W, et al. Wise-IoU:bounding box regression loss with dynamic focusing mechanism[EB/OL].(2023-01-24)[2024-11-14]. https://arxiv.org/abs/2301.10051.

[53] WANG C Y, YEH I H, LIAO H Y. YOLOv9:learning what you want to learn using programmable gradient information[EB/OL].(2024-02-29)[2024-11-14]. https://arxiv.org/abs/2402.13616.

[54] WANG A, CHEN H, LIU L H, et al. YOLOv10:real-time end-to-end object detection[EB/OL].(2024-05-23)[2024-11-14]. https://arxiv.org/abs/2405.14458v2.

[55] GEVORGYAN Z. SIoU loss:more powerful learning for bounding box regression[EB/OL].(2022-05-25)[2024-11-14]. https://arxiv.org/abs/2205.12740.

[56] WANG J, XU C, YANG W, YU L. A normalized Gaussian Wasserstein distance for tiny object detection[EB/OL].(2022-06-14)[2024-11-14]. https://arxiv.org/abs/2110.13389.

[57] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2):336-359.

Funding

Fundamental Research Funds for the Central Universities, China(2232020G-08)

Fundamental Research Funds for the Central Universities, China(2232020E-03)

Shanghai University Knowledge Service Platform, China(13S107024)