Bridging the gap: Neuro-Symbolic Computing for advanced AI applications in construction

Xiaochun LUO, Heng LI, SangHyun LEE

PDF(301 KB)
PDF(301 KB)
Front. Eng ›› 2023, Vol. 10 ›› Issue (4) : 727-735. DOI: 10.1007/s42524-023-0266-0
COMMENTS

Bridging the gap: Neuro-Symbolic Computing for advanced AI applications in construction

Author information +
History +

Abstract

Deep Learning (DL) has revolutionized the field of Artificial Intelligence (AI) in various domains such as computer vision (CV) and natural language processing. However, DL models have limitations including the need for large labeled datasets, lack of interpretability and explainability, potential bias and fairness issues, and limitations in common sense reasoning and contextual understanding. On the other side, DL has shown significant potential in construction for safety and quality inspection tasks using CV models. However, current CV approaches may lack spatial context and measurement capabilities, and struggle with complex safety and quality requirements. The integration of Neuro-Symbolic Computing (NSC), an emerging field that combines DL and symbolic reasoning, has been proposed as a potential solution to address these limitations. NSC has the potential to enable more robust, interpretable, and accurate AI systems in construction by harnessing the strengths of DL and symbolic reasoning. The combination of symbolism and connectionism in NSC can lead to more efficient data usage, improved generalization ability, and enhanced interpretability. Further research and experimentation are needed to effectively integrate NSC with large models and advance CV technologies for precise reporting of safety and quality inspection results in construction.

Keywords

advanced AI in construction / safety and quality inspection / Neuro-Symbolic Computing / Deep Learning / computer vision

Cite this article

Download citation ▾
Xiaochun LUO, Heng LI, SangHyun LEE. Bridging the gap: Neuro-Symbolic Computing for advanced AI applications in construction. Front. Eng, 2023, 10(4): 727‒735 https://doi.org/10.1007/s42524-023-0266-0

References

[1]
Badreddine, S d’Avila, Garcez A Serafini, L Spranger, M (2022). Logic tensor networks. Artificial Intelligence, 303: 103649
CrossRef Google scholar
[2]
Braun, A Tuttas, S Borrmann, A Stilla, U (2020). Improving progress monitoring by fusing point clouds, semantic data and computer vision. Automation in Construction, 116: 103210
CrossRef Google scholar
[3]
Brilakis, I Park, M W Jog, G (2011). Automated vision tracking of project related entities. Advanced Engineering Informatics, 25( 4): 713–724
CrossRef Google scholar
[4]
BrownT BMannBRyderNSubbiahMKaplanJ DDhariwalPNeelakantanAShyamPSastryGAskellA, . (2020). Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, BC: Curran Associates Inc., 1877–1901
[5]
CarionNMassaFSynnaeveGUsunierNKirillovAZagoruykoS (2020). End-to-end object detection with transformers. In: Proceedings of the 16th European Conference. Glasgow: Springer-Verlag, 213–229
[6]
Chi, S Caldas, C H (2011). Automated object identification using optical video cameras on construction sites. Computer-Aided Civil and Infrastructure Engineering, 26( 5): 368–380
CrossRef Google scholar
[7]
Chi, S Caldas, C (2012). Image-based safety assessment: Automated spatial safety risk identification of earthmoving and surface mining activities. Journal of Construction Engineering and Management, 138( 3): 341–351
CrossRef Google scholar
[8]
Chi, S Caldas, C H Kim, D Y (2009). A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques. Computer-Aided Civil and Infrastructure Engineering, 24( 3): 199–211
CrossRef Google scholar
[9]
CohenW WYangFMazaitisK R (2017). TensorLog: Deep learning meets probabilistic DBs. arXiv preprint, arXiv:1707.05390
[10]
Cussens, J (2001). Parameter estimation in stochastic logic programs. Machine Learning, 44( 3): 245–271
CrossRef Google scholar
[11]
d’Avila, Garcez A Lamb, L C (2023). Neurosymbolic AI: The 3rd wave. Artificial Intelligence Review, 56( 11): 12387–12406
CrossRef Google scholar
[12]
DanieleASerafiniL (2019). Knowledge enhanced neural networks. In: Proceedings of the 16th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence. Cuvu, Yanuca Island: Springer Cham, 542–554
[13]
Ding, L Fang, W Luo, H Love, P E D Zhong, B Ouyang, X (2018). A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory. Automation in Construction, 86: 118–124
CrossRef Google scholar
[14]
DuYFuZLiuQWangY (2022). Visual grounding with transformers. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). Taipei: IEEE, 1–6
[15]
Fang, Q Li, H Luo, X Ding, L Luo, H Rose, T M An, W (2018a). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in Construction, 85: 1–9
CrossRef Google scholar
[16]
Fang, Q Li, H Luo, X Ding, L Rose, T M An, W Yu, Y (2018b). A deep learning-based method for detecting non-certified work on construction sites. Advanced Engineering Informatics, 35: 56–68
CrossRef Google scholar
[17]
Fang, W Ding, L Luo, H Love, P E D (2018c). Falls from heights: A computer vision-based approach for safety harness detection. Automation in Construction, 91: 53–61
CrossRef Google scholar
[18]
Fang, W Love, P E D Ding, L Xu, S Kong, T Li, H (2021). Computer vision and deep learning to manage safety in construction: Matching images of unsafe behavior and semantic rules. IEEE Transactions on Engineering Management, 70( 12): 4120–4132
CrossRef Google scholar
[19]
Garnelo, M Shanahan, M (2019). Reconciling deep learning with symbolic artificial intelligence: Representing objects and relations. Current Opinion in Behavioral Sciences, 29: 17–23
CrossRef Google scholar
[20]
GirdharRGkioxariGTorresaniLPaluriMTranD (2018). Detect-and-track: Efficient pose estimation in videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE, 350–359
[21]
GirshickR (2015). Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 1440–1448
[22]
Gong, J Caldas, C H (2010). Computer vision-based video interpretation model for automated productivity analysis of construction operations. Journal of Computing in Civil Engineering, 24( 3): 252–263
CrossRef Google scholar
[23]
HeKGkioxariGDollárPGirshickR (2017). Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2980–2988
[24]
Hochreiter, S Schmidhuber, J (1997). Long short-term memory. Neural Computation, 9( 8): 1735–1780
CrossRef Google scholar
[25]
HuQYangBXieLRosaSGuoYWangZTrigoniNMarkhamA (2020). Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA: IEEE, 11105–11114
[26]
Huang, Y Wu, Q Wang, W Wang, L (2020). Image and sentence matching via semantic concepts and order learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42( 3): 636–650
CrossRef Google scholar
[27]
JainAGkanatsiosNMedirattaIFragkiadakiK (2022). Bottom up top down detection transformers for language grounding in images and point clouds. In: Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer-Verlag, 417–433
[28]
KahnemanDHintonGBengioYLeCunYRossiF (2020). AAAI-20 fireside chat with Daniel Kahneman
[29]
Kim, J Chung, D Kim, Y Kim, H (2022). Deep learning-based 3D reconstruction of scaffolds using a robot dog. Automation in Construction, 134: 104092
CrossRef Google scholar
[30]
Kim, P S Lee, D G Lee, S W (2018). Discriminative context learning with gated recurrent unit for group activity recognition. Pattern Recognition, 76: 149–161
CrossRef Google scholar
[31]
Kimmig, A Mihalkova, L Getoor, L (2015). Lifted graphical models: A survey. Machine Learning, 99( 1): 1–45
CrossRef Google scholar
[32]
KimmigAvan den BroeckGde RaedtL (2011). An algebraic prolog for reasoning about possible worlds. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. San Francisco, CA: AAAI Press, 209–214
[33]
KrizhevskyASutskeverIHintonG E (2012). ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, NV: Curran Associates Inc., 1097–1105
[34]
LeCun, Y Bengio, Y Hinton, G (2015). Deep learning. Nature, 521( 7553): 436–444
CrossRef Google scholar
[35]
Lei, L Zhou, Y Luo, H Love, P E D (2019). A CNN-based 3D patch registration approach for integrating sequential models in support of progress monitoring. Advanced Engineering Informatics, 41: 100923
CrossRef Google scholar
[36]
Li, Y Wei, H Han, Z Huang, J Wang, W (2020). Deep learning-based safety helmet detection in engineering management based on convolutional neural networks. Advances in Civil Engineering, 9703560
CrossRef Google scholar
[37]
Luo, X Li, H Cao, D Dai, F Seo, J Lee, S (2018). Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks. Journal of Computing in Civil Engineering, 32( 3): 04018012
CrossRef Google scholar
[38]
Luo, X Li, H Yang, X Yu, Y Cao, D (2019). Capturing and understanding workers’ activities in far-field surveillance videos with deep action recognition and Bayesian nonparametric learning. Computer-Aided Civil and Infrastructure Engineering, 34( 4): 333–351
CrossRef Google scholar
[39]
Luo, X Li, H Yu, Y Zhou, C Cao, D (2020). Combining deep features and activity context to improve recognition of activities of workers in groups. Computer-Aided Civil and Infrastructure Engineering, 35( 9): 965–978
CrossRef Google scholar
[40]
Manhaeve, R Dumančić, S Kimmig, A Demeester, T de, Raedt L (2021). Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence, 298: 103504
CrossRef Google scholar
[41]
MaoJGanCKohliPTenenbaumJ BWuJ (2019). The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In: Proceedings of the 7th International Conference on Learning Representations. New Orleans, LA: OpenReview.net
[42]
QuMTangJ (2019). Probabilistic logic neural networks for reasoning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, BC: Curran Associates Inc., 7712–7722
[43]
RadfordANarasimhanKSalimansTSutskeverI (2018). Improving language understanding by generative pre-training. OpenAI
[44]
Ren, S He, K Girshick, R Sun, J (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39( 6): 1137–1149
CrossRef Google scholar
[45]
Richardson, M Domingos, P (2006). Markov logic networks. Machine Learning, 62( 1–2): 107–136
CrossRef Google scholar
[46]
RiegelRGrayALuusFKhanNMakondoNYunus AkhalwayaIQianHFaginRBarahonaFSharmaUIkbalSKaranamHNeelamSLikhyaniASrivastavaS (2020). Logical neural networks. arXiv preprint, arXiv:2006.13155
[47]
Rodríguez-Gonzálvez, P Rodríguez-Martín, M Ramos, L F González-Aguilera, D (2017). 3D reconstruction methods and quality assessment for visual inspection of welds. Automation in Construction, 79: 49–58
CrossRef Google scholar
[48]
Seo, J Han, S Lee, S Kim, H (2015). Computer vision techniques for construction safety and health monitoring. Advanced Engineering Informatics, 29( 2): 239–251
CrossRef Google scholar
[49]
Sourek, G Aschenbrenner, V Zelezny, F Schockaert, S Kuzelka, O (2018). Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62: 69–100
CrossRef Google scholar
[50]
Teizer, J (2015). Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Advanced Engineering Informatics, 29( 2): 225–238
CrossRef Google scholar
[51]
Teizer, J Allread, B S Fullerton, C E Hinze, J (2010). Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system. Automation in Construction, 19( 5): 630–640
CrossRef Google scholar
[52]
Tran, S N d’Avila, Garcez A S (2018). Deep logic networks: Inserting and extracting knowledge from deep belief networks. IEEE Transactions on Neural Networks and Learning Systems, 29( 2): 246–258
CrossRef Google scholar
[53]
Vahdatikhaki, F Hammad, A (2015). Risk-based look-ahead workspace generation for earthwork equipment using near real-time simulation. Automation in Construction, 58: 207–220
CrossRef Google scholar
[54]
Wang, Q (2019). Automatic checks from 3D point cloud data for safety regulation compliance for scaffold work platforms. Automation in Construction, 104: 38–51
CrossRef Google scholar
[55]
Wang, Q Mao, Z Wang, B Guo, L (2017). Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29( 12): 2724–2743
CrossRef Google scholar
[56]
WuFJinGGaoMHeZYangY (2019). Helmet detection based on improved YOLO v3 deep model. In: Proceedings of the 16th International Conference on Networking, Sensing and Control (ICNSC). Banff, AB: IEEE, 363–368
[57]
Wu, H Zhong, B Li, H Love, P E D Pan, X Zhao, N (2021). Combining computer vision with semantic reasoning for on-site safety management in construction. Journal of Building Engineering, 42: 103036
CrossRef Google scholar
[58]
Yang, J Park, M W Vela, P A Golparvar-Fard, M (2015). Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future. Advanced Engineering Informatics, 29( 2): 211–224
CrossRef Google scholar
[59]
Yang, X Li, H Yu, Y Luo, X Huang, T Yang, X (2018). Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 33( 12): 1090–1109
CrossRef Google scholar
[60]
Zhang, K Zhang, Z Li, Z Qiao, Y (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23( 10): 1499–1503
CrossRef Google scholar
[61]
Zhong, B Li, H Luo, H Zhou, J Fang, W Xing, X (2020). Ontology-based semantic modeling of knowledge in construction: Classification and identification of hazards implied in images. Journal of Construction Engineering and Management, 146( 4): 04020013
CrossRef Google scholar
[62]
ZhuXSuWLuLLiBWangXDaiJ (2020). Deformable DETR: Deformable transformers for end-to-end object detection. arXiv preprint, arXiv:2010.04159

Competing Interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2023 Higher Education Press
AI Summary AI Mindmap
PDF(301 KB)

Accesses

Citations

Detail

Sections
Recommended

/