Vision-based survey method for extraordinary loads on buildings

Abstract

The statistical modeling of extraordinary loads on buildings has been stagnant for decades due to the laborious and error-prone nature of existing survey methods, such as questionnaires and verbal inquiries. This study proposes a new vision-based survey method for collecting extraordinary load data by automatically analyzing surveillance videos. For this purpose, a crowd head tracking framework is developed that integrates crowd head detection and reidentification models based on convolutional neural networks to obtain head trajectories of the crowd in the survey area. The crowd head trajectories are then analyzed to extract crowd quantity and velocities, which are the essential factors for extraordinary loads. For survey areas with frequent crowd movements during temporary events, the equivalent dynamic load factor can be further estimated using crowd velocity to consider dynamic effects. A crowd quantity investigation experiment and a crowd walking experiment are conducted to validate the proposed survey method. The experimental results prove that the proposed survey method is effective and accurate in collecting load data and reasonable in considering dynamic effects during extraordinary events. The proposed survey method is easy to deploy and has the potential to collect substantial and reliable extraordinary load data for determining design load on buildings.

Graphical abstract

Keywords

load survey / extraordinary load / live load / object detection / multiple-object tracking

References

[1] PeirJ. A stochastic live load model for buildings. Dissertation for the Doctoral Degree. Cambridge: Massachusetts Institute of Technology, 1972

[2] Peir J C, Cornell C A. Spatial and temporal variability of live loads. Journal of the Structural Division, 1973, 99(5): 903–922

[3] McGuire R K, Cornell C A. Live load effects in office buildings. Journal of the Structural Division, 1974, 100(7): 1351–1366

[4] MitchellG RWoodgate R W. Floor Loadings in Office Buildings: The Results of A Survey. Watford: Building Research Station, 1971

[5] Culver C G. Live-load survey results for office buildings. Journal of the Structural Division, 1976, 102(12): 2269–2284

[6] Jin X Y, Zhao J D. Development of the design code for building structures in China. Journal of the International Association for Bridge and Structural Engineering, 2012, 22(2): 195–201

[7] Andam K A. Floor live loads for office buildings. Building and Environment, 1986, 21(3–4): 211–219

[8] Choi E C C. Live load in office buildings point-in-time load intensity of rooms. Proceedings of the Institution of Civil Engineers. Structures and Buildings, 1992, 94(3): 299–306

[9] Kumar S. Live loads in office buildings: Point-in-time load intensity. Building and Environment, 2002, 37(1): 79–89

[10] Ellingwood B, Culver C. Analysis of live loads in office buildings. Journal of the Structural Division, 1977, 103(8): 1551–1560

[11] Chalk P L, Corotis R B. Probability model for design live loads. Journal of the Structural Division, 1980, 106(10): 2017–2033

[12] Harris M E, Bova C J, Corotis R B. Area-dependent processes for structural live loads. Journal of the Structural Division, 1981, 107(5): 857–872

[13] PaloheimoEOllila M. Research in the Live Loads in Persons. Finland: Ministry of Domestic Affairs, 1973

[14] Choi E C C. Live load for office buildings: Effect of occupancy and code comparison. Journal of Structural Engineering, 1990, 116(11): 3162–3174

[15] Choi E C C. Extraordinary live load in office buildings. Journal of Structural Engineering, 1991, 117(11): 3216–3227

[16] Andam K A. Variability in stochastic sustained and extraordinary load processes. Building and Environment, 1990, 25(4): 389–396

[17] Kumar S. Live loads in office buildings: Lifetime maximum load. Building and Environment, 2002, 37(1): 91–99

[18] Corotis R B, Jaria V A. Stochastic nature of building live loads. Journal of the Structural Division, 1979, 105(3): 493–510

[19] SentlerL. A Live Load Survey in Domestic Houses. Lund: Division of Building Technology, Lund Institute of Technology, 1974

[20] IssenL A. A literature review of fire and live load surveys in residences. Final Report National Bureau of Standards, 1978

[21] Ho L V, Trinh T T, de Roeck G, Bui-Tien T, Nguyen-Ngoc L, Abdel Wahab M. An efficient stochastic-based coupled model for damage identification in plate structures. Engineering Failure Analysis, 2022, 131: 105866

[22] Tran V T, Nguyen T K, Nguyen-Xuan H, Abdel Wahab M. Vibration and buckling optimization of functionally graded porous microplates using BCMO-ANN algorithm. Thin-walled Structures, 2023, 182: 110267

[23] Dang B L, Nguyen-Xuan H, Wahab M A. An effective approach for VARANS-VOF modelling interactions of wave and perforated breakwater using gradient boosting decision tree algorithm. Ocean Engineering, 2023, 268: 113398

[24] Cha Y J, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361–378

[25] WojkeNBewley APaulusD. Simple online and realtime tracking with a deep association metric. In: Proceedings of 2017 IEEE International Conference on Image Processing. Beijing: IEEE; 2017, 3645–3649

[26] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149

[27] HeK MZhang X YRenS QSunJ. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016, 770–778

[28] Xu Y, Bao Y Q, Chen J H, Zuo W M, Li H. Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images. Structural Health Monitoring, 2019, 18(3): 653–674

[29] Ghosh Mondal T, Jahanshahi M R, Wu R T, Wu Z Y. Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance. Structural Control and Health Monitoring, 2020, 27(4): e2507

[30] Nguyen D H, Wahab M A. Damage detection in slab structures based on two-dimensional curvature mode shape method and Faster R-CNN. Advances in Engineering Software, 2023, 176: 103371

[31] Nath N D, Behzadan A H, Paal S G. Deep learning for site safety: Real-time detection of personal protective equipment. Automation in Construction, 2020, 112: 103085

[32] Li Y, Chen J. Computer vision-based counting model for dense steel pipe on construction sites. Journal of Construction Engineering and Management, 2022, 148(1): 4021178

[33] Luo X C, Li H, Wang H, Wu Z Z, Dai F, Cao D P. Vision-based detection and visualization of dynamic workspaces. Automation in Construction, 2019, 104: 1–13

[34] Xiao B, Xiao H R, Wang J W, Chen Y. Vision-based method for tracking workers by integrating deep learning instance segmentation in off-site construction. Automation in Construction, 2022, 136: 104148

[35] Xiao B, Kang S C. Vision-based method integrating deep learning detection for tracking multiple construction machines. Journal of Computing in Civil Engineering, 2021, 35(2): 4020071

[36] HeK MGkioxari GDollárPGirshickR. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017, 2961–2969

[37] Wang Z C, Zhang Q L, Yang B, Wu T K, Lei K, Zhang B H, Fang T W. Vision-based framework for automatic progress monitoring of precast walls by using surveillance videos during the construction phase. Journal of Computing in Civil Engineering, 2021, 35(1): 4020056

[38] JocherGStoken AChaurasiaABorovecJ. Yolov5: v6.0. 2021. Available at the website of GitHub

[39] LiuSQiL QinH FShi JJiaJ Y. Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 8759–8768

[40] IoffeSSzegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. Lille: JMLR, 2015, 448–456

[41] Elfwing S, Uchibe E, Doya K. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 2018, 107: 3–11

[42] WangC YLiao H MWuY HChenP YHsiehJ W YehI H. CSPNet: A new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020, 390–391

[43] He K M, Zhang X Y, Ren S Q, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904–1916

[44] LinT YDollár PGirshickRHeK MHariharan BBelongieS. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017, 2117–2125

[45] Kalman R E. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 1960, 82(1): 35–45

[46] Kuhn H W. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 1955, 2(1–2): 83–97

[47] DingX HZhang X YMaN NHanJ GDingG G SunJ. Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021, 13733–13742

[48] Wang J P, Chen J. A comparative study on different walking load models. Structural Engineering and Mechanics, 2017, 63(6): 847–856

[49] Tubino F, Piccardo G. Serviceability assessment of footbridges in unrestricted pedestrian traffic conditions. Structure and Infrastructure Engineering, 2016, 12(12): 1650–1660

[50] ChenJWang H QPengY X. Experimental investigation on Fourier-series model of walking load and its coefficients. Journal of Vibration and Shock, 2014, 33(8): 11–15 (in Chinese)

[51] ZhaoD SChen J. Tests for correlation and modeling of individual 3-D continuous walking load. Journal of Vibration and Shock, 2019, 38(11): 166–172 (in Chinese)

[52] Wang J P, Chen J, Yokoyama Y, Xiong J C. Spectral model for crowd walking load. Journal of Structural Engineering, 2020, 146(3): 04019220

[53] PengD ZSun Z KChenZ RCaiZ RXieL L JinL W. Detecting heads using feature refine net and cascaded multi-scale architecture. In: Proceedings of 2018 24th International Conference on Pattern Recognition. Beijing: IEEE, 2018, 2528–2533

[54] LinT YMaire MBelongieSHaysJPeronaP RamananDDollár PZitnickC L. Microsoft COCO: Common objects in context. In: Proceedings of Computer Vision—ECCV 2014: 13th European Conference. Zurich: Springer, 2014, 740–755

[55] KingmaD PBa J. Adam: A method for stochastic optimization. 2014, arXiv: 14126980

[56] GoyalPDollár PGirshickRNoordhuisPWesolowski LKyrolaATullochAJiaY Q HeK M. Accurate, large minibatch sgd: Training imagenet in 1 hour. 2017, arXiv: 170602677

[57] LoshchilovIHutter F. Sgdr: Stochastic gradient descent with warm restarts. 2016, arXiv: 160803983

[58] Everingham M, Van Gool L, Williams C K I, Winn J, Zisserman A. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303–338

[59] ZhouK YYang Y XCavallaroAXiangT. Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019, 3702–3712

[60] MaN NZhang X YZhengH TSunJ. Shufflenet v2: Practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018, 116–131

[61] IandolaF NHan SMoskewiczM WAshrafKDallyW J KeutzerK. SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and < 0.5 MB model size. 2016, arXiv: 160207360

[62] HowardASandler MChuGChenL CChenB TanM XWang W JZhuY KPangR MVasudevan VLeQ VAdamH. Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019, 1314–1324

[63] DengJDong WSocherRLiL JLiK LiF F. Imagenet: A large-scale hierarchical image database. In: Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009, 248–255

[64] WangX GDoretto GSebastianTRittscherJTuP. Shape and appearance context modeling. In: Proceedings of 2007 IEEE 11th International Conference on Computer Vision. Rio de Janeiro: IEEE, 2007, 1–8

[65] ZhengLShen L YTianLWangS JWangJ D TianQ. Scalable person re-identification: A benchmark. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015, 1116–1124

Acknowledgements

The authors acknowledge the financial support provided by the National Natural Science Foundation of China (Grant No. 52178151). The helpful suggestions provided by Dr. Jinping Wang are greatly appreciated.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Competing interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2024 The Author(s). This article is published with open access at link.springer.com and journal.hep.com.cn