Tracking guided actions recognition for cows

Abstract

Background: Cows actions are important factors of cows health and their well-being. By monitoring the individual cows actions, we prevent cows diseases and realize modern precision cows rearing. However, traditional cows actions monitoring is usually conducted through video recording or direct visual observation, which is time-consuming and laborious, and often lead to misjudgement due to the subjective consciousness or negligence.

Methods: This paper proposes a method of cows actions recognition based on tracked trajectories to automatically recognize and evaluate the actions of cows. First, we construct a dataset including 60 videos to describe the popular actions existing in the daily life of cows, providing the basic data for designing our actions recognition method. Second, eight famous trackers are used to track and obtain temporal and spatial information of targets. Third, after studying and analysing the tracked trajectories of different actions about cows, a rigorous and effective constraint method is designed to realize actions recognition by us.

Results: Many experiments demonstrate that our method of actions recognition performs favourably in detecting the actions of cows, and the proposed dataset basically satisfies the actions evaluation for farmers.

Conclusion: The proposed tracking guided actions recognition provides a feasible way to maintain and promote cows health and welfare.

Key words： public framework; actions recognition; visual tracking; precision agriculture; cows health

ACKNOWLEDGMENTS

This work was supported by the National Natural Science Foundation of China (No. 61772209), Science and Technology Planning Project of Guangdong Province (Nos. 2019A050510034 and 2019B020219001), the Production Project of Ministry Education China (No. 201901240030), the College Students Innovations Special Project of China (No. 202010564026), and Guangzhou Key Laboratory of Intelligent Agriculture (No. 201902010081).

COMPLIANCE WITH ETHICS GUIDELINES

The authors Yun Liang and Xiaoming Chen declare that they have no conflict of interest or financial conflicts to disclose. All procedures performed in studies involving animals were in accordance with the ethical standards of the institution or practice at which the studies were conducted, and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

OPEN ACCESS

This article is licensed by the CC By under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creativecommons.org/licenses/by/4.0/.

References

1 Qiu, T., Chen, N., Li, K., Atiquzzaman, M. (2018). How can heterogeneous internet of things build our future: a survey. IEEE Comm. Surv. and Tutor., 20: 2011–2027

2 Fregonesi, J. (2001). Behaviour, performance and health indicators of welfare for dairy cows housed in strawyard or cubicle systems. Livest. Prod. Sci., 68: 205–216

3 Haley, D. B., (2001). Assessing cow comfort: effects of two floor types and two tie stall designs on the behaviour of lactating dairy cows. Appl. Anim. Behav. Sci., 71: 105–117

4 KrohnC.. (1994) Behaviour of dairy cows kept in extensive (loose housing/pasture) or intensive (tie stall) environments. iii. grooming, exploration and abnormal behaviour. Appl. Anim. Behav. Sci., 42, 73–86

5 Green, L. E., Hedges, V. J., Schukken, Y. H., Blowey, R. W. Packington, A. (2002). The impact of clinical lameness on the milk yield of dairy cows. J. Dairy Sci., 85: 2250–2256

6 Mattachini, G., Riva, E., Bisaglia, C., Pompe, J. C. (2013). Methodology for quantifying the behavioral activity of dairy cows in freestall barns. J. Anim. Sci., 91: 4899–4907

7 Thorup, V. M., Munksgaard, L., Robert, P. E., Erhard, H. W., Thomsen, P. T. Friggens, N. (2015). Lameness detection via leg-mounted accelerometers on dairy cows on four commercial farms. Animal, 9: 1704–1712

8 pez-Gatius, F., Santolaria, P., Mundet, I. niz, J. (2005). Walking activity at estrus and subsequent fertility in dairy cows. Theriogenology, 63: 1419–1429

9 Kiddy, C. (1977). Variation in physical activity as an indication of estrus in dairy cows. J. Dairy Sci., 60: 235–243

10 HouR.,ChenC.. (2017) Tube convolutional neural network (T-CNN) for action detection in videos. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5823−5832

11 KuehneH.,JhuangH.,GarroteE.,PoggioT.. (2011) Hmdb: a large video database for human motion recognition. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp. 25562563

12 SoomroK.,ZamirA. R.. (2012) Ucf101: a dataset of 101 human actions classes from videos in the wild. arXiv, 1212.0402

13 Liu, M., Meng, F., Chen, C. (2019). Joint dynamic pose image and space time reversal for human action recognition from videos. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 8762–8769

14 Kristan, M., Pflugfelder, R. (2015). The visual object tracking VOT2015 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 564–586

15 Saif, S., Tehseen, S. (2018). A survey of the techniques for the identification and classification of human actions from visual data. Sensors (Basel), 18: 3979

16 Wu, Y., Lim, J. Yang, M. (2015). Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell., 37: 1834–1848

17 Henriques, J. F., Caseiro, R., Martins, P. (2012). Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the 12th European conference on Computer Vision (ECCV), pp. 702–715

18 Hart, B. (1988). Biological basis of the behavior of sick animals. Neurosci. Biobehav. Rev., 12: 123–137

19 Norouzzadeh, M. S., Nguyen, A., Kosmala, M., Swanson, A., Palmer, M. S., Packer, C. (2018). Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. In: Proceedings of the National Academy of Science of the United States of America, pp. E5716–E5725

20 Sharma, S. U. Shah, D. (2016). A practical animal detection and collision avoidance system using computer vision technique. IEEE Access, 5: 347–358

21 Andriluka, M. Iqbal, U. Pishchulin, L., Milan, A., Gall, J., (2018). PoseTrack: A Benchmark for Human Pose Estimation and Tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5167–5176

22 Andriluka, M., Pishchulin, L., Gehler, P. (2014). 2D human pose estimation: New benchmark and state of the art analysis. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3686–3693

23 Kalogeiton, V., Weinzaepfel, P., Ferrari, V. (2017). Joint learning of object and action detectors. In: The IEEE International Conference on Computer Vision (ICCV), pp. 2001–2010

24 Gilani, S. O., Subramanian, R., Yan, Y., Melcher, D., Sebe, N. (2015). PET: An eye-tracking dataset for animal-centric Pascal object classes. In: Proceeding of the IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6

25 Jeong, S., Kim, G. (2017). Effective visual tracking using multi-block and scale space based on kernelized correlation filters. Sensors (Basel), 17: 433

26 Henriques, J. F., Caseiro, R., Martins, P. (2015). High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell., 37: 583–596

27 Wang, H., Liu, P., Du, Y., (2019). Online convolution network tracking via spatio-temporal context. Mulitmed. Tools Appl., 78: 257–270

28 Liu, N., Hong, H., (2019). Robust object tracking via constrained online dictionary learning. Mulitmed. Tools Appl., 78: 3689–3703

29 Gao, J., Wang, Q., Xing, J., Ling, H., Hu, W., Maybank, S. (2018). Tracking-by-fusion via Gaussian process regression extended to transfer learning. IEEE Trans. Pattern Anal. Mach. Intell., 42: 939–955

30 Li, F., Tian, C., Zuo, W., Zhang, L. Yang, M. (2018). Learning spatial-temporal regularized correlation filters for visual tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4904–4913

31 Zhang, K., Zhang, L. Yang, M. (2012). Real-time compressive tracking. In: Proceedings of the 12th European conference on Computer Vision (ECCV), pp. 864–877

32 Sevilla-Lara, L. (2012). Distribution fields for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1910–1917

33 Henriques, J. Caseiro, R., Martins, P., (2015). High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell., 37: 583–596

34 Li, Y., Zhu, J. Hoi, S. C. (2015). Reliable Patch Trackers: Robust visual tracking by exploiting reliable patches. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 353–361

35 Galoogahi, H. K., Fagg, A. (2017). Learning backgroundaware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1144–1152

36 Bertinetto, L., Valmadre, J., Henriques, J., Vedaldi, A. (2016). Fully-convolutional siamese networks for object tracking. In: European Conference on Computer Vision, pp. 850–865

37 Song, Y., Chao, M., Gong, L., Zhang, J. (2017). Crest: convolutional residual learning for visual tracking. In: IEEE International Conference on Computer Vision (ICCV), pp. 2574–2583

38 Zhang, Y., Cheng, L., Wu, J., Cai, J., Do, M. N. (2016). Action recognition in still images with minimum annotation efforts. IEEE Trans. Image Process., 25: 5479–5490

39 Poppe, R. (2010). A survey on vision-based human action recognition. Image Vis. Comput., 28: 976–990

40 Dollar, P., Rabaud, V. (2005). Behavior recognition via sparse spatio-temporal features. In: Proceedings of the IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72

41 Yin, L., Wei, X. (2006). A 3D facial expression database for facial behavior research. In: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 211–216

42 Tanfous, A. B., Drira, H. Amor, B. (2018). Coding kendall’s shape trajectories for 3D action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2840–2849

43 Tang, Y., Tian, Y., Lu, J., Li, P. (2018). Deep progressive reinforcement learning for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5323–5332

44 Gallego, G., Rebecq, H. (2018). A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3867–3876

45 Guo, G. (2014). A survey on still image based human action recognition. Pattern Recognit., 47: 3343–3361

46 ZhangT.,ZhangY.,CaiJ.KotA.. (2016) Efficient object feature selection for action recognition. In: Proceeding of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2707–2711

47 Feichtenhofer, C., Pinz, A. Wildes, R. (2017). Spatiotemporal multiplier networks for video action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7445–7454

48 Girdhar, R., Gkioxari, G., Torresani, L., Paluri, M. (2018). Detect-and-track: Efficient pose estimation in videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 350–359

49 Lu, X., Yao, H., Zhao, S., Sun, X., (2019). Action recognition with multi-scale trajectory-pooled 3D convolutional descriptors. Mulitmed. Tools Appl., 78: 507–523