Deep reinforcement learning-based critical element identification and demolition planning of frame structures

Abstract

This paper proposes a framework for critical element identification and demolition planning of frame structures. Innovative quantitative indices considering the severity of the ultimate collapse scenario are proposed using reinforcement learning and graph embedding. The action is defined as removing an element, and the state is described by integrating the joint and element features into a comprehensive feature vector for each element. By establishing the policy network, the agent outputs the Q value for each action after observing the state. Through numerical examples, it is confirmed that the trained agent can provide an accurate estimation of the Q values, and handle problems with different action spaces owing to utilization of graph embedding. Besides, different behaviors can be learned by varying hyperparameters in the reward function. By comparing the proposed method and the conventional sensitivity index-based methods, it is demonstrated that the computational cost is considerably reduced because the reinforcement learning model is trained offline. Besides, it is proved that the Q values produced by the reinforcement learning agent can make up for the deficiencies of existing indices, and can be directly used as the quantitative index for the decision-making for determining the most expected collapse scenario, i.e., the sequence of element removals.

Graphical abstract

Keywords

progressive collapse / alternate load path / demolition planning / reinforcement learning / graph embedding

References

[1] StarossekU. Progressive Collapse of Structures. London: Thomas Telford, 2009

[2] Shi Y, Li Z, Hao H. A new method for progressive collapse analysis of RC frames under blast loading. Engineering Structures, 2010, 32(6): 1691–1703

[3] Zhao X, Yan S, Chen Y, Xu Z, Lu Y. Experimental study on progressive collapse-resistant behavior of planar trusses. Engineering Structures, 2017, 135: 104–116

[4] Izzuddin B A, Vlassis A G, Elghazouli A Y, Nethercot D A. Progressive collapse of multi-storey buildings due to sudden column loss—Part I: Simplified assessment framework. Engineering Structures, 2008, 30(5): 1308–1318

[5] Vlassis A G, Izzuddin B A, Elghazouli A Y, Nethercot D A. Progressive collapse of multi-storey buildings due to sudden column loss—Part II: Application. Engineering Structures, 2008, 30(5): 1424–1438

[6] Kiakojouri F, De Biagi V, Chiaia B, Sheidaii M R. Progressive collapse of framed building structures: Current knowledge and future prospects. Engineering Structures, 2020, 206: 110061

[7] Frangopol D M, Curley J P. Effects of damage and redundancy on structural reliability. Journal of Structural Engineering, 1987, 113(7): 1533–1549

[8] Jiang X, Chen Y. Progressive collapse analysis and safety assessment method for steel truss roof. Journal of Performance of Constructed Facilities, 2012, 26(3): 230–240

[9] Frangopol D M, Curley J P. Effects of damage and redundancy on structural reliability. Journal of Structural Engineering, 1987, 113(7): 1533–1549

[10] Fallon C T, Quiel S E, Naito C J. Uniform pushdown approach for quantifying building-frame robustness and the consequence of disproportionate collapse. Journal of Performance of Constructed Facilities, 2016, 30(6): 04016060

[11] BiondiniFFrangopol D MRestelliS. On structural robustness, redundancy and static indeterminacy. In: Proceedings of the Structures Congress 2008. Vancouver: American Society of Civil Engineers, 2008

[12] StarossekUHaberland M. Approaches to measures of structural robustness. Structure and Infrastructure Engineering, 2011, 7(7−8): 625−631

[13] Bao Y, Main J A, Noh S Y. Evaluation of structural robustness against column loss: Methodology and application to RC frame buildings. Journal of Structural Engineering, 2017, 143(8): 04017066

[14] Chen C, Zhu Y, Yao Y, Huang Y. Progressive collapse analysis of steel frame structure based on the energy principle. Steel and Composite Structures, 2016, 21(3): 553–571

[15] Bažant Z P, Verdure M. Mechanics of progressive collapse: Learning from World Trade Center and building demolitions. Journal of Engineering Mechanics, 2007, 133(3): 308–319

[16] Luccioni B M, Ambrosini R D, Danesi R F. Analysis of building collapse under blast loads. Engineering Structures, 2004, 26(1): 63–71

[17] Sun J, Jia Y, Yao Y, Xie X. Experimental investigation of stress transients of blasted RC columns in the blasting demolition of buildings. Engineering Structures, 2020, 210: 110417

[18] Zhu J, Zheng W, Sneed L H, Xu C, Sun Y. Green demolition of reinforced concrete structures: Review of research findings. Global Journal of Research in Engineering, 2019, 19(4): 1–18

[19] Isobe D. An analysis code and a planning tool based on a key element index for controlled explosive demolition. International Journal of High-Rise Buildings, 2014, 3(4): 243–254

[20] JiangJLu DLuXLiGYeJ. Research progress and development trends on progressive collapse resistance of building structures. Journal of Building Structures, 2022, 43(1): 1−28 (in Chinese)

[21] Ohsaki M. Random search method based on exact reanalysis for topology optimization of trusses with discrete cross-sectional areas. Computers & Structures, 2001, 79(6): 673–679

[22] Makode P V, Ramirez M R, Corotis R B. Reanalysis of rigid frame structures by the virtual distortion method. Structural Optimization, 1996, 11(2): 71–79

[23] Ellingwood B R, Dusenberry D O. Building design for abnormal loads and progressive collapse. Computer-Aided Civil and Infrastructure Engineering, 2005, 20(3): 194–205

[24] Chen C, Zhu Y, Yao Y, Huang Y, Long X. An evaluation method to predict progressive collapse resistance of steel frame structures. Journal of Constructional Steel Research, 2016, 122: 238–250

[25] Jiang L, Ye J. Risk-based robustness assessment of steel frame structures to unforeseen events. Civil Engineering and Environmental Systems, 2018, 35(1−4): 117–138

[26] Baker J W, Schubert M, Faber M H. On the assessment of robustness. Structural Safety, 2008, 30(3): 253–267

[27] Liao K W, Wen Y K, Foutch D A. Evaluation of 3D steel moment frames under earthquake excitations. II: Reliability and redundancy. Journal of Structural Engineering, 2007, 133(3): 471–480

[28] Feng D, Xie S, Xu J, Qian K. Robustness quantification of reinforced concrete structures subjected to progressive collapse via the probability density evolution method. Engineering Structures, 2020, 202: 109877

[29] SuttonR SBarto A G. Reinforcement Learning: An Introduction. Cambridge: MIT press, 2018

[30] Jordan M I, Mitchell T M. Machine learning: Trends, perspectives, and prospects. Science, 2015, 349(6245): 255–260

[31] Sun H, Burton H V, Huang H. Machine learning applications for building structural design and performance assessment: State-of-the-art review. Journal of Building Engineering, 2021, 33: 101816

[32] Salehi H, Burgueño R. Emerging artificial intelligence methods in structural engineering. Engineering Structures, 2018, 171: 170–189

[33] Zhu S, Ohsaki M, Guo X. Prediction of non-linear buckling load of imperfect reticulated shell using modified consistent imperfection and machine learning. Engineering Structures, 2021, 226: 111374

[34] Xue J, Xiang Z, Ou G. Predicting single freestanding transmission tower time history response during complex wind input through a convolutional neural network based surrogate model. Engineering Structures, 2021, 233: 111859

[35] Guo H, Zhuang X, Rabczuk T. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59(2): 433–456

[36] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345–359

[37] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[38] Zhuang X, Guo H, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87: 104225

[39] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 1–26

[40] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Analysis of three-dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis. Engineering with Computers, 2022, 1–22

[41] Li S, Snaiki R, Wu T. A knowledge-enhanced deep reinforcement learning-based shape optimizer for aerodynamic mitigation of wind-sensitive structures. Computer-Aided Civil and Infrastructure Engineering, 2021, 36(6): 733–746

[42] Hayashi K, Ohsaki M. Reinforcement learning and graph embedding for binary truss topology optimization under stress and displacement constraints. Frontiers in Built Environment, 2020, 6: 59

[43] Zhu S, Ohsaki M, Hayashi K, Guo X. Machine-specified ground structures for topology optimization of binary trusses using graph embedding policy network. Advances in Engineering Software, 2021, 159: 103032

[44] Makarov I, Kiselev D, Nikitinsky N, Subelj L. Survey on graph embeddings and their applications to machine learning problems on graphs. PeerJ. Computer Science, 2021, 7: e357

[45] DaiHKhalilE B ZhangYDilkina BSongL. Learning combinatorial optimization algorithms over graphs. 2017, arXiv: 1704.01665

[46] NguyenT TReddi V J. Deep reinforcement learning for cyber security. 2019, arXiv: 1906.05799

[47] Fan C, Zeng L, Sun Y, Liu Y. Finding key players in complex networks through deep reinforcement learning. Nature Machine Intelligence, 2020, 2(6): 317–324

[48] Nafday A M. System safety performance metrics for skeletal structures. Journal of Structural Engineering, 2008, 134(3): 499–504

[49] Kiakojouri F, Sheidaii M R, De Biagi V, Chiaia B. Progressive collapse of structures: A discussion on annotated nomenclature. Structures, 2021, 29: 1417–1423

[50] WatkinsC J C HDayanP. Q-learning. Machine Learning, 1992, 8(3−4): 279−292

[51] WilliamsR J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 1992, 8(3−4): 229−256

[52] SchulmanJWolski FDhariwalPRadfordAKlimovO. Proximal policy optimization algorithms. 2017, arXiv: 1707.06347

[53] UFC4-023-03 (Change 3). Design of Buildings to Resist Progressive Collapse. Washington, D.C.: United States Department of Defense, 2016

[54] CECS392:2014. Code for Anti-collapse Design of Building Structures. Beijing: China Planning Press, 2014 (in Chinese)

[55] Mnih V, Kavukcuoglu K, Silver D, Rusu A A, Veness J, Bellemare M G, Graves A, Riedmiller M, Fidjeland A K, Ostrovski G, Peterson S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529–533

[56] Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors. Nature, 1986, 323(6088): 533–536

[57] Tieleman T, Hinton G. Lecture 6. 5-RMSProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012, 4: 26–31

[58] SchaulTQuan JAntonoglouISilverD. Prioritized experience replay. 2015, arXiv: 1511.05952

[59] GSA. Progressive Collapse Analysis and Design Guidelines for New Federal Office Buildings and Major Modernization Projects. USA: US General Services Administration (GSA), 2005

[60] ANSYS®Multiphysics 19.0. Canonsburg, PA: Ansys Inc., 2018

[61] Riks E. An incremental approach to the solution of snapping and buckling problems. International Journal of Solids and Structures, 1979, 15(7): 529–551

[62] Crisfield M A. An arc-length method including line searches and accelerations. International Journal for Numerical Methods in Engineering, 1983, 19(9): 1269–1289

[63] KaszynskiA. PyMAPDL (version 0.59.4). Zenodo, 2021

Acknowledgments

The authors gratefully acknowledge the financial support provided by the China Scholarship Council (CSC) during a visit of Shaojun Zhu to Kyoto University (No. 201906260152). The second author acknowledges the support of JSPS KAKENHI (Grant No. JP20H04467). The third author acknowledges the support of Grant-in-Aid for Young Scientists (Start-up) (Grant No. JP21K20461).

RIGHTS & PERMISSIONS

2022 Higher Education Press