Assessing quality of crash modification factors estimated by empirical Bayes before-after methods

Abstract

Before-after study with the empirical Bayes (EB) method is the state-of-the-art approach for estimating crash modification factors (CMFs). The EB method not only addresses the regression-to-the-mean bias, but also improves accuracy. However, the performance of the CMFs derived from the EB method has never been fully investigated. This study aims to examine the accuracy of CMFs estimated with the EB method. Artificial realistic data (ARD) and real crash data are used to evaluate the CMFs. The results indicate that: 1) The CMFs derived from the EB before-after method are nearly the same as the true values. 2) The estimated CMF standard errors do not reflect the true values. The estimation remains at the same level regardless of the pre-assumed CMF standard error. The EB before-after study is not sensitive to the variation of CMF among sites. 3) The analyses with real-world traffic and crash data with a dummy treatment indicate that the EB method tends to underestimate the standard error of the CMF. Safety researchers should recognize that the CMF variance may be biased when evaluating safety effectiveness by the EB method. It is necessary to revisit the algorithm for estimating CMF variance with the EB method.

Keywords

traffic safety / empirical Bayes / crash modification factor / safety effectiveness evaluation

References

[1] American Association of State Highway and Transportation Officials (AASHTO). Highway safety manual [M]. Washington DC, 2010.

[2] PIARC. Roadway safety manual [M]. World Road Association, 2003.

[3] GrossF, PersaudB N, LyonC AGuide to developing quality crash modification factors [R], 2010, Washington DC, Federal Highway Administration

[4] WU L T. Examining the use of regression models for developing crash modification factors [D]. Texas A&M University, 2016.

[5] GoochJ P, GayahV V, DonnelE T. Quantifying the safety effects of horizontal curves on two-way, two-lane rural roads [J]. Accident Analysis & Prevention, 2016, 92: 71-81

[6] WoodJ S, GoochJ P, DonnellE T. Estimating the safety effects of lane widths on urban streets in Nebraska using the propensity scores-potential outcomes framework [J]. Accident Analysis & Prevention, 2015, 82: 180-191

[7] SasidharanL, DonnellE T. Propensity scores-potential outcomes framework to incorporate severity probabilities in the highway safety manual crash prediction algorithm [J]. Accident Analysis and Prevention, 2014, 71: 183-193

[8] HauerEObservational before-after studies in road safety: Estimating the effect of highway and traffic engineering measures on road safety [M], 1997, Tarrytown, N.Y., USA, Pergamon

[9] ShenJ, GanA. Development of crash reduction factors: Methods, problems, and research needs [J]. Transportation Research Record, 2003, 1840(1): 50-56

[10] NolandR B, AdedijiY. Are estimates of crash modification factors mis-specified? [J]. Accident Analysis & Prevention, 2018, 118: 29-37

[11] WuL T, LordD, ZouY J. Validation of crash modification factors derived from cross-sectional studies with regression models [J]. Transportation Research Record, 2015, 2514(1): 88-96

[12] WuL T, LordD. Examining the influence of link function misspecification in conventional regression models for developing crash modification factors [J]. Accident Analysis & Prevention, 2017, 102: 123-135

[13] WuL T, LordD, GeedipallyS R. Developing crash modification factors for horizontal curves on rural two-lane undivided highways using a cross-sectional study [J]. Transportation Research Record, 2017, 2636(1): 53-61

[14] HauerEThe art of regression modeling in road safety [M], 2015, New York, Springer

[15] HAUER E. Even perfect regressions may not tell the effect of interventions [C]// The 92nd Annual Meeting of Transportation Research Board. Washington DC, 2013.

[16] HauerE. Cause, effect and regression in road safety: A case study [J]. Accident Analysis & Prevention, 2010, 42(4): 1128-1135

[17] FHWA. CMF clearinghouse brochure [EB/OL]. [2019-01-20]. http://www.cmfclearing-house.org/collateral/CMF_brochure.pdf.

[18] MontellaA. Safety evaluation of curve delineation improvements: Empirical Bayes observational before-and-after study [J]. Transportation Research Record, 2009, 2103(1): 69-79

[19] WU L T, MENG Y, KONG X Q, ZOU Y J. A novel approach for estimating crash modification factors: Jointly modeling crash counts and time intervals between crashes [C]// Transportation Research Board 98th Annual Meeting, Washington DC, 2019.

[20] LordD, GeedipallyS RSafety effects of the red-light camera enforcement program in Chicago, Illinois [R], 2014, College Station, Texas, Lord Consulting

[21] WuL T, GeedipallyS R, PikeA M. Safety evaluation of alternative audible lane departure warning treatments in reducing traffic crashes: an empirical Bayes observational before-after study [J]. Transportation Research Record, 2018, 2672(21): 30-40

[22] LordD, ManneringF. The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives [J]. Transportation Research Part A: Policy and Practice, 2010, 44(5): 291-305

[23] WuL T, ZouY J, LordD. Comparison of sichel and negative binomial models in hot spot identification [J]. Transportation Research Record, 2014, 2460(1): 107-116

[24] ZouY J, LordD, ZhangY L, PengY C. Comparison of sichel and negative binomial models in estimating empirical Bayes estimates [J]. Transportation Research Record, 2013, 2392(1): 11-21

[25] LordD, GeedipallyS R, GuikemaS D. Extension of the application of Conway-Maxwell-Poisson models: Analyzing traffic crash data exhibiting underdispersion [J]. Risk Analysis: An International Journal, 2010, 30(8): 1268-1276

[26] LordD, GuikemaS D, GeedipallyS R. Application of the Conway-Maxwell-Poisson generalized linear model for analyzing motor vehicle crashes [J]. Accident Analysis & Prevention, 2008, 40(3): 1123-1134

[27] ZouY J, AshJ E, ParkB J, LordD, WuL T. Empirical Bayes estimates of finite mixture of negative binomial regression models and its application to highway safety [J]. Journal of Applied Statistics, 2018, 45(9): 1652-1669

[28] ParkB J, LordD, WuL T. Finite mixture modeling approach for developing crash modification factors in highway safety analysis [J]. Accident Analysis & Prevention, 2016, 97: 274-287

[29] ZouY J, HenricksonK, WuLT, WangY H, ZhangZ R. Application of the empirical Bayes method with the finite mixture model for identifying accident-prone spots [J]. Mathematical Problems in Engineering, 2015, 2015(10): 1-10

[30] ParkB J, LordD, LeeC. Finite mixture modeling for vehicle crash data with application to hotspot identification [J]. Accident Analysis & Prevention, 2014, 71: 319-326

[31] YangX X, ZouY J, WuL T, ZhongX Z, WangY H, IjazM, PengY C. Comparative analysis of the reported animal-vehicle collisions data and carcass removal data for hotspot identification [J]. Journal of Advanced Transportation, 2019, 2019(3): 1-13

[32] HAUER E. Trustworthiness of safety performance functions [C]// The 93rd Annual Meeting of the Transportation Research Board. Washington DC, 2014.

[33] LordD. Modeling motor vehicle crashes using Poisson-gamma models: Examining the effects of low sample mean values and small sample size on the estimation of the fixed dispersion parameter [J]. Accident Analysis & Prevention, 2006, 38(4): 751-766

[34] ManneringF. Temporal instability and the analysis of highway accident data [J]. Analytic Methods in Accident Research, 2018, 17: 1-13

[35] HauerE, HarwoodD W, CouncilF M, GriffithM S. Estimating safety by the empirical Bayes method: A tutorial [J]. Transportation Research Record, 2002, 1784(1): 126-131

[36] YANG X, ZOU Y, TANG J, LIANG J, IJAZ M. Evaluation of short-term freeway speed prediction based on periodic analysis using statistical models and machine learning models [J]. Journal of Advanced Transportation, 2020. DOI: https://doi.org/10.1155/2020/9628957.

[37] DasS, Minjares-KyleL, WuL, HenkR. Understanding crash potential associated with teen driving: Survey analysis using multivariate graphical method [J]. Journal of Safety Research, 2019, 70: 213-222