Optimal Control of the False Discovery Exceedance in Large-Scale Directional Multiple Testing

Abstract

In large-scale multiple hypothesis testing, controlling the false discovery exceedance (FDX) is emerging as a compelling alternative to the widely employed false discovery rate (FDR) when the false discovery proportion (FDP) exhibits high variability. Existing methods for FDX control have mainly focused on nondirectional discovery within a two-group model framework. However, this approach can significantly compromise the reliability of FDX control when directional decisions are required, especially in cases when the signal-to-noise ratio is low. Furthermore, the theoretical optimality of FDX control has not been thoroughly explored. In this paper, we introduce an empirical Bayes approach tailored for directional FDX control within a three-group model. We demonstrate that an oracle decision rule, which ranks and thresholds a directional version of the local false discovery rate (lfdr), achieves optimality by maximizing power while adhering to the directional FDX constraint. We also propose a data-driven procedure to mimic the oracle rule in practical applications and establish its asymptotic optimality. Through extensive simulation studies and a real-data application, we show the superior performance of our method in directional FDX control.

Keywords

Directional decision-making / Empirical Bayes / False discovery exceedance / Multiple hypothesis testing / 62J15

References

[1] Ajello M, Arimoto M, Axelsson M, Baldini L, Barbiellini G, Bastieri D, Bellazzini R, Bhat P, Bissaldi E, Blandford R, et al. . A decade of gamma-ray bursts observed by fermi-lat: the second GRB catalog. Astrophys J, 2019, 878(1): 52

[2] Barras L, Scaillet O, Wermers R. False discoveries in mutual fund performance: measuring luck in estimated alphas. J. Financ., 2010, 65(1): 179-216

[3] Basu P, Fu L, Saretto A, Sun W. An empirical bayes approach to controlling the false discovery exceedance. J. Bus. Econ. Stat., 2024, 42(3): 1041-1052

[4] Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Stat. Soc. Ser. B (Methodol.), 1995, 57(1): 289-300

[5] Blanchard G, Neuvial P, Roquain E. Post hoc confidence bounds on false positives using reference families. Ann. Stat., 2020, 48(3): 1281-1303

[6] Delattre S, Roquain E. New procedures controlling the false discovery proportion via Romano–Wolf’s heuristic. Ann. Stat., 2015, 43(3): 1141-1177

[7] Efron B. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. Institute of Mathematical Statistics Monographs, 2010, Cambridge. Cambridge University Press

[8] Efron B, Tibshirani R, Storey JD, Tusher V. Empirical bayes analysis of a microarray experiment. J. Am. Stat. Assoc., 2001, 96(456): 1151-1160

[9] Farcomeni A. Generalized augmentation to control the false discovery exceedance in multiple testing. Scand. J. Stat., 2009, 36(3): 501-517

[10] Genovese CR, Wasserman L. Exceedance control of the false discovery proportion. J. Am. Stat. Assoc., 2006, 101(476): 1408-1417

[11] Harvey CR, Liu Y. False (and missed) discoveries in financial economics. J. Financ., 2020, 75(5): 2503-2553

[12] Heller R, Rosset S. Optimal control of false discovery criteria in the two-group model. J. R. Stat. Soc. Ser. B Stat. Methodol., 2021, 83: 133-155

[13] Hemerik J, Solari A, Goeman JJ. Flexible control of the median of the false discovery proportion. Biometrika, 2024, 111(4): 1129-1150

[14] Holte SE, Lee EK, Mei Y. Symmetric directional false discovery rate control. Stat. Methodol., 2016, 33: 71-82

[15] Javanmard A, Javadi H. False discovery rate control via debiased lasso. Electron. J. Stat., 2019, 13: 1212-1253

[16] Katsevich E, Ramdas A. Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings. Ann. Stat., 2020, 48(6): 3465-3487

[17] Kong W, Wei X, Wu Y, Shao M, Zhang Q, Sadowsky MJ, Ishii S, Reich PB, Wei G, Jiao S, Qiu L, Liu L. Afforestation can lower microbial diversity and functionality in deep soil layers in a semiarid region. Glob. Change Biol., 2022, 28(20): 6086-6101

[18] Leung D, Tran N. Adaptive procedures for directional false discovery rate control. Electron. J. Stat., 2024, 18(1): 706-741

[19] Li J, Maathuis MH, Goeman JJ. Simultaneous false discovery proportion bounds via knockoffs and closed testing. J. R. Stat. Soc. Ser. B Stat. Methodol., 2024, 86(4): 966-98

[20] Liang, W., Xiang, D., Mei, Y., Li, W.: Directional false discovery rate control in large-scale multiple comparisons. J. Appl. Stat. pp. 1–20 (2024)

[21] Lloyd-Price J, Arze C, Ananthakrishnan AN, Schirmer M, Avila-Pacheco J, Poon TW, Andrews E, Ajami NJ, Bonham KS, Brislawn CJ, et al. . Multi-omics of the gut microbial ecosystem in inflammatory bowel diseases. Nature, 2019, 569(7758): 655-662

[22] Luo D, Ebadi A, Emery K, He Y, Noble WS, Keich U. Competition-based control of the false discovery proportion. Biometrics, 2023, 79(4): 3472-3484

[23] Miecznikowski JC, Wang J. Exceedance control of the false discovery proportion via high precision inversion method of berk-jones statistics. Comput. Stat. Data Anal., 2023, 185 107758

[24] Romano JP, Wolf M. Control of generalized error rates in multiple testing. Ann. Stat., 2007, 35(4): 1378-1408

[25] Schott M, Grinberger AY, Lautenbach S, Zipf A. The impact of community happenings in openstreetmap—establishing a framework for online community member activity analyses. ISPRS Int. J. Geo Inf., 2021, 10(3): 164

[26] Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA, D’Amico AV, Richie JP. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell, 2002, 1(2): 203-209

[27] Souto D, Prieto CA, Cunha K, Pinsonneault M, Smith VV, Garcia-Dias R, Bovy J, García-Hernández DA, Holtzman J, Johnson JA, Jönsson H, Majewski SR, Shetrone M, Sobeck J, Zamora O, Pan K, Nitschelm C. Chemical abundances of main-sequence, turnoff, subgiant, and red giant stars from apogee spectra. II. Atomic diffusion in m67 stars. Astrophys. J., 2019, 874(1): 97

[28] Sun W, Cai TT. Oracle and adaptive compound decision rules for false discovery rate control. J. Am. Stat. Assoc., 2007, 102: 901-912

[29] Tian Z, Liang K, Li P. A powerful procedure that controls the false discovery rate with directional information. Biometrics, 2021, 77: 212-222

[30] van der Laan, M.J., Dudoit, S., Pollard, K.S.: Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Stat. Appl. Genetics Mol. Biol. 3(1) (2004)

[31] Xiang D, Li W, Tsung F, Pu X, Kang Y. Fault classification for high-dimensional data streams: a directional diagnostic framework based on multiple hypothesis testing. Naval Res. Logist. (NRL), 2021, 68(7): 973-987

[32] Xiang D, Zhao SD, Cai TT. Signal classification for the integrative analysis of multiple sequences of large-scale multiple tests. J. R. Stat. Soc. Ser. B Stat. Methodol., 2019, 81: 707-734

RIGHTS & PERMISSIONS

School of Mathematical Sciences, University of Science and Technology of China and Springer-Verlag GmbH Germany, part of Springer Nature