Outlier management in data analysis: a checklist for authors and reviewers

Abstract

The improper handling of outliers in the analysis of variance (ANOVA) presents a persistent challenge in forestry research, which may lead to biased results, inflated Type I error rates, and obscured scientific signals. The current practice is often an ad hoc method, potentially driven by a need to achieve statistical significance rather than principled scientific reasoning. This Editorial paper addresses this systemic issue by proposing a structured, step-by-step framework for the diagnosis and management of outliers. The framework guides researchers to first investigate the cause of an outlier (data error, measurement error, or genuine extreme value), then statistically assess its impact on ANOVA results and assumptions, and finally, make a transparent decision on its treatment. We strongly advise against the statistically problematic practice of replacing outliers with the mean of other replicates, as it violates data integrity and obscures true variability. Instead, we recommend robust alternatives, including data transformation, non-parametric tests, or the use of trimmed means. This approach aims to uphold statistical robustness and scientific integrity, thereby improving the rigor of forestry research and its publications.

Keywords

Journal editor / Peer review / Statistical analysis / Science communication / Scientific writing

References

[1] Agathokleous E, Yu L. Six statistical issues in scientific writing that might lead to rejection of a manuscript. J Forestry Res, 2022, 33: 731-739.

[2] Agathokleous E, Yu L. Effect size in papers published by the Journal of Forestry Research: a missing treasure?. J Forestry Res, 2023, 34: 297-299.

[3] Aguinis H, Gottfredson RK, Joo H. Best-practice recommendations for defining, identifying, and handling outliers. Organ Res Meth, 2013, 16(2): 270-301.

[4] Benhadi-Marín J. A conceptual framework to deal with outliers in ecology. Biodivers Conserv, 2018, 27(12): 3295-3300.

[5] Cook CN, Freeman AR, Liao JC, Mangiamele LA. The philosophy of outliers: reintegrating rare events into biological science. Integr Comp Biol, 2022, 61(6): 2191-2198.

[6] Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol, 2016, 31(4): 337-350.

[7] Hawkins DM. Identification of outliers, 1980, Netherlands. Springer.

[8] Knott JA, Liknes GC, Giebink CL, Oh S, Domke GM, McRoberts RE, Quirino VF, Walters BF. Effects of outliers on remote sensing-assisted forest biomass estimation: a case study from the United States national forest inventory. Meth Ecol Evol, 2023, 14(7): 1587-1602.

[9] Lakens D. Why P values are not measures of evidence. Trends Ecol Evol, 2022, 37(4): 289-290.

[10] Łopucki R, Kiersztyn A, Pitucha G, Kitowski I. Handling missing data in ecological studies: ignoring gaps in the dataset can distort the inference. Ecol Model, 2022, 468109964

[11] Nakagawa S, Cuthill IC. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev, 2007, 82(4): 591-605.

[12] NIST (2012) NIST/SEMATECH e-Handbook of Statistical Methods. http://www.itl.nist.gov/div898/handbook/

[13] Osborne JW, Overbay A. The power of outliers (and why researchers should ALWAYS check for them). Pract Assessment Res Eval, 2004, 96

[14] Siegel S, Castellan NJJr. Nonparametric statistics for the behavioral sciences, international Edition, 19882nNew York. McGraw-Hill Book Company

[15] Smiti A. A critical overview of outlier detection methods. Comput Sci Rev, 2020, 38100306

[16] Solla F, Tran A, Bertoncelli D, Musoff C, Bertoncelli CM. Why a P-value is not enough. Clin Spine Surg A Spine Publ, 2018, 31(9): 385-388.

[17] Sullivan JH, Warkentin M, Wallace L. So many ways for assessing outliers: what really works and does it matter?. J Bus Res, 2021, 132: 530-543.

[18] Thabane L, Mbuagbaw L, Zhang SY, Samaan Z, Marcucci M, Ye CL, Thabane M, Giangregorio L, Dennis B, Kosa D, Borg Debono V, Dillenburg R, Fruci V, Bawor M, Lee J, Wells G, Goldsmith CH. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med Res Methodol, 2013, 1392

[19] Zuur AF, Ieno EN, Elphick CS. A protocol for data exploration to avoid common statistical problems. Meth Ecol Evol, 2010, 1(1): 3-14.

RIGHTS & PERMISSIONS

Northeast Forestry University