Recognition of the behaviors of dairy cows by an improved YOLO

Abstract

The physiological well-being of dairy cows is intimately tied to their behavior. Detecting aberrant dairy cows early and reducing financial losses on farms are both possible with real-time and reliable monitoring of their behavior. The behavior data of dairy cows in real environments have dense occlusion and multi-scale issues, which affect the detection results of the model. Therefore, we focus on both data processing and model construction to improve the results of dairy cow behavior detection. We use a mixed data augmentation method to provide the model with rich cow behavior features. Simultaneously refining the model to optimize the detection outcomes of dairy cow behavior amidst challenging conditions, such as dense occlusion and varying scales. First, a Res2 backbone was constructed to incorporate multi-scale receptive fields and improve the YOLOv3’s backbone for the multi-scale feature of dairy cow behaviors. In addition, YOLOv3 detectors were optimized to accurately locate individual dairy cows in different dense environments by combining the global location information of images, and the Global Context Predict Head was designed to enhance the performance of recognizing dairy cow behaviors in crowded surroundings. The dairy cow behavior detection model we built has an accuracy of 90.6%, 91.7%, 80.7%, and 98.5% for the four behaviors of dairy cows standing, lying, walking, and mounting, respectively. The average accuracy of dairy cow detection is 90.4%, which is 1.2% and 12.9% higher than the detection results of YOLOV3, YOLO-tiny and other models respectively. In comparison to YOLOv3, the Average Precision evaluation of the model improves by 2.6% and 1.4% for two similar features of walking and standing behaviors, respectively. The recognition results prove that the model generalizes better for recognizing dairy cow behaviors using behavior videos in various scenes with multi-scale and dense environment features.

Keywords

Dairy cow behaviors / dense environment / YOLOv3 / multi-scale / attention module

References

[1] Hu H,Shen W.Cow identification based on fusion of deep parts features.Biosystems Engineering2020;192:245-56

[2] Kumar S,Sai Ram Satwik K.Deep learning framework for recognition of cattle using muzzle point image pattern.Measurement2018;116:1-17

[3] Uenishi S,Kojima T.A novel accelerometry approach combining information on classified behaviors and quantified physical activity for assessing health status of cattle: a preliminary study.Applied Animal Behaviour Science2021;235:105220

[4] Niloofar P,Lazarova-molnar S.Data-driven decision support in livestock farming for improved animal health, welfare and greenhouse gas emissions: overview and challenges.Computers and Electronics in Agriculture2021;190:106406

[5] Shen W,Zhang Y,Fu Q.Automatic recognition of ingestive-related behaviors of dairy cows based on triaxial acceleration.Information Processing in Agriculture2020;7:427-43

[6] Benaissa S,Plets D.Classification of ingestive-related cow behaviors using RumiWatch halter and neck-mounted accelerometers.Applied Animal Behaviour Science2019;211:9-16

[7] Barker ZE,Codling EA.Use of novel sensors combining local positioning and acceleration to measure feeding behavior differences associated with lameness in dairy cattle.J Dairy Sci2018;101:6310-21

[8] Arablouei R,Kusy B,Greenwood PL.In-situ classification of cattle behavior using accelerometry data.Computers and Electronics in Agriculture2021;183:106045

[9] Benaissa S,Plets D.Calving and estrus detection in dairy cattle using a combination of indoor localization and accelerometer sensors.Computers and Electronics in Agriculture2020;168:105153

[10] Ren XH,Zhang M,Zhang XY.Dairy cattle's behaviour recognition method based on support vector machine classification model.Transactions of the Chinese Society for Agricultural2019;50:290-6.

[11] Wang K,Duan QL.Identification of sow oestrus behaviour based on MFO-LSTM.Transactions of the Chinese Society of Agricultural Engineering2020;36:211-9.

[12] Khin MP,Mar CC,Horii Y. Cattle pose classification system using deeplabcut and svm model. 2022 IEEE 11th Global Conference on Consumer Electronics (GCCE); 2022 Oct 494-5; Osaka, Japan.

[13] Minar MR. Recent advances in deep learning: an overview. Available from: https://arxiv.org/abs/1807.08169 [Last accessed on 25 Jan 2024]

[14] Jiang B,Wang B.MGLNN: semi-supervised learning via multiple graph cooperative learning neural networks.Neural Netw2022;153:204-14

[15] Zhang HM,Han WT,Niu DD.Detection method of maize seedlings number based on improved YOLO.Transactions of the Chinese Society for Agricultural2021;52:221-9.

[16] Roy AM.DenseSPH-YOLOv5: An automated damage detection model based on densenet and swin-transformer prediction head-enabled YOLOv5 with attention mechanism.Advanced Engineering Informatics2023;56:102007

[17] Hu Z,Lou T.Dual attention-guided feature pyramid network for instance segmentation of group pigs.Computers and Electronics in Agriculture2021;186:106140

[18] Xiao J,Wang K.Cow identification in free-stall barns based on an improved mask R-CNN and an SVM.Computers and Electronics in Agriculture2022;194:106738

[19] Bonneau M,Troupe W.Outdoor animal tracking combining neural network and time-lapse cameras.Computers and Electronics in Agriculture2020;168:105150

[20] Su Q,Zhai M.An intelligent method for dairy goat tracking based on Siamese network.Computers and Electronics in Agriculture2022;193:106636

[21] Chen C,Steibel J.Recognition of aggressive episodes of pigs based on convolutional neural network and long short-term memory.Computers and Electronics in Agriculture2020;169:105166

[22] Chen C,Steibel J,Han J.Classification of drinking and drinker-playing in pigs by a video-based deep learning method.Biosystems Engineering2020;196:1-14

[23] Chen C,Steibel J,Han J.Recognition of feeding behaviour of pigs and determination of feeding time of each pig by a video-based deep learning method.Computers and Electronics in Agriculture2020;176:105642

[24] Yan HW,Fui QL.Multi-target detection based on feature pyramid attention and deep convolution network for pigs.Transactions of the Chinese Society of Agricultural Engineering2020;36:193-202

[25] Li X,Zhang R,He J.Deep cascaded convolutional models for cattle pose estimation.Computers and Electronics in Agriculture2019;164:104885

[26] Wei S,Kanade T. Convolutional pose machines. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016 4724-32.

[27] Newell A,Deng J. Stacked hourglass networks for human pose estimation. In: Leibe B, Matas J, Sebe N, Welling M, editors. Computer Vision – ECCV 2016. Cham: Springer International Publishing; 2016. pp. 483-99.

[28] Bulat A. Human pose estimation via convolutional part heatmap regression. In: Leibe B, Matas J, Sebe N, Welling M, editors. Computer Vision – ECCV 2016. Cham: Springer International Publishing; 2016. pp. 717-32.

[29] Wang SH,Liu D.Automatic recognition method of dairy cow estrus behaviour based on machine visionAutomatic recognition method of dairy cow estrus behaviour based on machine vision.Transactions of the Chinese Society for Agricultural2020;51:241-9.

[30] Krizhevsky A,Hinton GE.ImageNet classification with deep convolutional neural networks.Commun ACM2017;60:84-90

[31] Yin X,Shang Y,Song H.Using an efficientnet-LSTM for the recognition of single cow’s motion behaviours in a complicated environment.Computers and Electronics in Agriculture2020;177:105707

[32] Benaissa S,Plets D.On the use of on-cow accelerometers for the classification of behaviours in dairy barns.Res Vet Sci2019;125:425-33

[33] Wang SH.Estrus behavior recognition of dairy cows based on improved YOLO v3 model.Transactions of the Chinese Society for Agricultural Machinery2021;52:141-50.

[34] Zhang HM,Zhou LX,Li SQ.Multi-target skeleton extraction method of beef cattle based on improved YOLO v3.Transactions of the Chinese Society for Agricultural Machinery2022;53:285-93.

[35] Ma S,Li T.Basic motion behavior recognition of single dairy cow based on improved Rexnet 3D network.Computers and Electronics in Agriculture2022;194:106772

[36] Ji JT,Gao RH,Zhao KX.Ruminant behavior analysis method of dairy cows with improved flownet 2.0 optical flow algorithm.Transactions of the Chinese Society for Agricultural Machinery2023;54:235-42.

[37] Guo JJ,Xu LQ,Feng DC.Pigeon behavior detection model based on improved YOLO v4.Transactions of the Chinese Society for Agricultural Machinery2023;54:347-55.

[38] Hu T,Jiang C.Grazing sheep behaviour recognition based on improved YOLOV5.Sensors2023;23:4752 PMCID:PMC10223245

[39] Bochkovskiy A,Liao HYM.YOLOv4: optimal speed and accuracy of object detection.ArXiv2020; preprint arXiv:10934.

[40] Redmon J,Girshick R. You only look once: unified, real-time object detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016.p.779-788.

[41] Gao SH,Zhao K. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence; 2021.p.652-662.

[42] Cao Y,Lin S. Gcnet: Non-local networks meet squeeze-excitation networks and beyond. Proceedings of the IEEE/CVF international conference on computer vision workshops; 2019.