An improved deep multiscale crowd counting network with perspective awareness

Abstract

Crowd counting is a challenging task, which is partly due to the multiscale variation and perspective distortion of crowd images. To solve these problems, an improved deep multiscale crowd counting network with perspective awareness was proposed. This network contains two branches. One branch uses the improved ResNet50 network to extract multiscale features, and the other extracts perspective information using a perspective-aware network formed by fully convolutional networks. The proposed network structure improves the counting accuracy when the crowd scale changes, and reduce the influence of perspective distortion. To accommodate various crowd scenarios, data-driven approaches are used to fine-tune the trained convolutional neural networks (CNN) model of the target scenes. The extensive experiments on three public datasets demonstrate the validity and reliability of the proposed method.

References

[1] Dalai N. and Triggs B., Histograms of Oriented Gradients for Human Detection, IEEE Computer Vision and Pattern Recognitio, 886 (2005).

[2] Rabaud V. and Belongie S., Counting Crowded Moving Objects, IEEE Computer Vision and Pattern Recognition, 705 (2006).

[3] Zhang Y., Zhou D., Chen S., Gao S. and Ma Y., Single-Image Crowd Counting via Multi-Column Convolutional Neural Network, IEEE Computer Vision and Pattern Recognition, 589 (2016).

[4] Sam D. B., Surya S. and Babu R.V., Switching Convolutional Neural Network for Crowd Counting, IEEE Computer Vision and Pattern Recognition, 4031 (2017).

[5] Cao X., Wang Z., Zhao Y. and Su F., Scale Aggregation Network for Accurate and Efficient Crowd Counting, European Conference on Computer Vision, 757 (2018).

[6] Li Y., Zhang X. and Chen D., CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1091 (2018).

[7] Idrees H., Soomro K. and Shah M., Detecting Humans in Dense Crowds Using Locally-Consistent Scale Prior and Global Occlusion Reasoning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986 (2015).

[8] Arteta C., Lempitsky V. and Zisserman A., Counting in the Wild, European Conference on Computer Vision, 483 (2016).

[9] ChenJ, SuW, WangZ. Neurocomputing, 2020, 382: 210

[10] Wang Q., Gao J., Lin W. and Li X., NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization, arXiv:2001.03360, 2020.

[11] He K., Zhang X. and Ren S., Deep Residual Learning for Image Recognition, IEEE Conference on Computer Vision and Pattern Recognition, 770 (2016).

[12] Zeng L., Xu X., Cai B., Qiu S. and Zhang T., Multi-Scale Convolutional Neural Networks for Crowd Counting, IEEE International Conference on Image Processing, 465 (2017).

[13] Guo D., Li K., Zha Z. and Wang M., DadNet: Dilated-attention-Deformable Convnet for Crowd Counting, 27th ACM International Conference on Multimedia, 1823 (2019).

[14] Jiang X., Xiao Z., Zhang B., Zhen X., Cao X., David D. and Shao L., Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks, IEEE Computer Vision and Pattern Recognition, 6126 (2019).

[15] Gao J., Wang Q. and Li X., PCC Net: Perspective Crowd Counting via Spatial Convolutional Network, arXiv:1905.10085, 2019.

[16] Sam D. B., Sajjan N. N. and Babu R. V., Divide and Grow: Capturing Huge Diversity in Crowd Images with Incrementally Growing CNN, IEEE Computer Vision and Pattern Recognition, 3618 (2018).

[17] LiuC, DuanY, DuJ, XuT. IEEE Access, 2020, 8: 48352