Deep learning based on connected vehicles for icing pavement detection

Abstract

Slippery road conditions, such as snowy, icy or slushy pavements, are one of the major threats to road safety in winter. The U.S. Department of Transportation (USDOT) spends over 20% of its maintenance budget on pavement maintenance in winter. However, despite extensive research, it remains a challenging task to monitor pavement conditions and detect slippery roadways in real time. Most existing studies have mainly explored indirect estimates based on pavement images and weather forecasts. The emerging connected vehicle (CV) technology offers the opportunity to map slippery road conditions in real time. This study proposes a CV-based slippery detection system that uses vehicles to acquire data and implements deep learning algorithms to predict pavements' slippery conditions. The system classifies pavement conditions into three major categories: dry, snowy and icy. Different pavement conditions reflect different levels of slipperiness: dry surface corresponds to the least slippery condition, and icy surface to the most slippery condition. In practice, more attention should be paid to the detected icy and snowy pavements when driving or implementing pavement maintenance and road operation in winter. The classification algorithm adopted in this study is Long Short-Term Memory (LSTM), which is an artificial Recurrent Neural Network (RNN). The LSTM model is trained with simulated CV data in VISSIM and optimized with a Bayesian algorithm. The system can achieve 100%, 99.06% and 98.02% prediction accuracy for dry pavement, snowy pavement and icy pavement, respectively. In addition, it is observed that potential accidents can be reduced by more than 90% if CVs can adjust their driving speed and maintain a greater distance from the leading vehicle after receiving a warning signal. Simulation results indicate that the proposed slippery detection system and the information sharing function based on the CV technology and deep learning algorithm (i.e., the LSTM network implemented in this study) are expected to deliver real-time detection of slippery pavement conditions, thus significantly eliminating the potential risk of accidents.

Keywords

Connected vehicles / Deep learning / Long short-term memory network / Pavement conditions / Slippery detection / VISSIM

References

[1] AhnC, PengH, TsengHE. Robust estimation of road frictional coefficient. IEEE Transactions on Control Systems Technology, 2011, 21(1):1-13

[2] Andrey, J. C., Mills, B. E., & Vandermolen, J. (2001). Weather information and road safety.

[3] Archer, J. (2005). Indicators for traffic safety assessment and prediction and their application in micro-simulation modelling: A study of urban and suburban intersections, KTH.

[4] AsamerJ, van ZuylenHJ, HeilmannB. Calibrating car-following parameters for snowy road conditions in the microscopic traffic simulator VISSIM. IET Intelligent Transport Systems, 2013, 7(1):114-121

[5] ASTM. (2011). Standard test method for skid resistance of paved surfaces using a full-scale tire.

[6] Brochu, E., Cora, V. M. & De Freitas, N. (2010). A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599.

[7] ChenC, ZhaoX, LiuH, RenG, ZhangY, LiuX. Assessing the influence of adverse weather on traffic flow characteristics using a driving simulator and VISSIM. Sustainability, 2019, 11(3):830

[8] Chen, J., Chen, S., Wang, Q., Cao, B., Feng, G., & Hu J. (2019a). iRAF: A deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet of Things Journal.

[9] ChenT, ZhangX, JiangH, AsaeikheybariG, GoelN, HooperMW, HuangM-C. Are you smoking? Automatic alert system helping people keep away from cigarettes. Smart Health, 2018, 9: 158-169

[10] Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2018a). Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal.

[11] Cooper, D. F., & Ferguson, N. (1976). Traffic studies at T-Junctions. 2. A conflict simulation Record. Traffic Engineering & Control 17(Analytic).

[12] Enriquez, D., Bautista, A., Field, P., Kim, S.-i., Jensen, S.-i., Ali, M., & Miller, J. (2012). CANOPNR: CAN-OBD programmable-expandable network-enabled reader for real-time tracking of slippery road conditions using vehicular parameters. Intelligent Transportation Systems (ITSC), 2012 15th International IEEE Conference on, IEEE.

[13] GoodfellowI, BengioY, CourvilleA, BengioY. Deep learning, 2016 MIT Press

[14] Graves, A., Mohamed, A.-r., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. Acoustics, speech and signal processing (ICASSP), 2013 IEEE International Conference on, IEEE.

[15] Heimann, B., Bouzid, N., Trabelsi, A. (2006). Road-wheel interaction in vehicles-a mechatronic view of friction. Mechatronics, 2006 IEEE International Conference on, IEEE.

[16] HintonG, DengL, YuD, DahlGE, MohamedA-R, JaitlyN, SeniorA, VanhouckeV, NguyenP, SainathTN. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 2012, 29(6):82-97

[17] HintonGE, SalakhutdinovRR. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786):504-507

[18] HochreiterS, SchmidhuberJ. Long short-term memory. Neural Computation, 1997, 9(8):1735-1780

[19] Hogema, J., & Janssen, W. (1996). Effects of intelligent cruise control on driving behaviour: a simulator study.

[20] Hou, Y., Gupta, A., Guan, T., Hu, S., Su, L., & Qiao, C. (2017). VehSense: Slippery road detection using smartphones. arXiv preprint arXiv:1705.03955.

[21] Irschik, D., & Stork, W. (2014). Road surface classification for extended floating car data. Vehicular Electronics and Safety (ICVES), 2014 IEEE International Conference on, IEEE.

[22] LeCunY, BengioY, HintonG. Deep learning. Nature, 2015, 521(7553):436

[23] Lerner, N., Singer, J., Huey, R., Brown, T., Marshall, D., Chrysler, S., Schmitt, R., Baldwin, C. L., Eisert, J. L., & Lewis, B. (2015). Driver-vehicle interfaces for advanced crash warning systems: Research on evaluation methods and warning signals.

[24] LiangL, YeH, LiGY. Toward intelligent vehicular networks: A machine learning framework. IEEE Internet of Things Journal, 2018, 6(1):124-135

[25] LintonMA, FuL. Connected vehicle solution for winter road surface condition monitoring. Transportation Research Record: Journal of the Transportation Research Board, 2016, 2551: 62-72

[26] LuN, ChengN, ZhangN, ShenX, MarkJW. Connected vehicles: Solutions and challenges. IEEE Internet of Things Journal, 2014, 1(4):289-299

[27] Olney, R. D., & Manorotkul, S. (2007). Driver adaptive collision warning system, Google Patents.

[28] Ozbay, K., Yang, H., Bartin B., & Mudigonda, S. (2008). Derivation and validation of new simulation-based surrogate safety measure. Transportation Research Record: Journal of the Transportation Research Board (2083): 105–113.

[29] PadarthyM, HeynsE. Identification and classification of slippery winter road conditions using commonly available vehicle variables. Transportation Research Record, 2019, 2673(2):60-70

[30] Panahandeh, G., Ek, E., & Mohammadiha, N. (2017). Road friction estimation for connected vehicles using supervised machine learning. In IEEE intelligent vehicles symposium, IV, 1262–1267, IEEE.

[31] Pisano, P. A., Goodwin, L. C. & Rossetti, M. A. (2008). US highway crashes in adverse road weather conditions. 24th Conference on International Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, New Orleans, LA.

[32] Prokeš, J. (2015). Realtime estimation of tyre-road friction for vehicle state estimator, Master’s thesis.

[33] Sak, H., Senior, A., & Beaufays, F. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. Fifteenth annual conference of the international speech communication association.

[34] Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25.

[35] Standard, S. S. V. (2009). Dedicated Short Range Communications (DSRC) Message Set Dictionary. SAE International, November.

[36] Van Der Horst, R., & Hogema, J. (1993). Time-to-collision and collision avoidance systems.

[37] WangJ, AlexanderL, RajamaniR. Friction estimation on highway vehicles using longitudinal measurements. Journal of Dynamic Systems, Measurement, and Control, 2004, 126(2):265-275

[38] Wright, J., Garrett, J. K., Hill, C. J., Krueger, G. D., Evans, J. H., Andrews, S., Wilson, C. K., Rajbhandari, R., & Burkhard, B. (2014). National connected vehicle field infrastructure footprint analysis, United States. Department of Transportation. Intelligent Transportation

[39] Yang, H. (2012). Simulation-based evaluation of traffic safety performance using surrogate safety measures, Rutgers University-Graduate School-New Brunswick.

[40] ZhangX, YangZ, ChenT, ChenD, HuangMC. Cooperative sensing and wearable computing for sequential hand gesture recognition. IEEE Sensors Journal, 2004, 19(14):5775-5783

[41] AhnC, PengH, TsengHE. Robust estimation of road frictional coefficient. IEEE Transactions on Control Systems Technology, 2011, 21(1):1-13