Guidelines for the use and reporting of experimental statistics in additive manufacturing: An assessment of current practices

Abstract

Additive manufacturing (AM) creates three-dimensional objects using various approaches, typically layer-by-layer. One emerging method is laser-based powder bed fusion of metals (PBF-LB/M), which uses high-energy lasers to melt metallic powder into shape. AM processes are influenced by many factors, yet there is no standardized framework for quantifying their effects on final products. This guide introduces key principles of experimental design and statistics, outlining a roadmap for conducting rigorous experiments. We review the literature on AM generally and PBF-LB/M specifically to assess how well current practices align with standardized methodologies. In addition, we compare the evolution of experimental techniques in PBF-LB/M to those in a more regulated industry to explore potential cross-pollination. Our analysis reveals that most studies do not adhere to best practices in experimental design and statistical analysis. For example, randomization of run order is rarely mentioned, and statistical model assumptions are often unchecked. Even in tightly regulated fields, experimental designs and statistical methods remain basic and lack sophistication. To improve research quality, we provide recommendations for establishing standardized experimental and reporting practices in AM.

Keywords

Design of experiments / Response surface methodology / Additive manufacturing and 3D printing / Laser-based powder bed fusion of metals / Experimental statistics

Acknowledgments

None.

Funding

The review described here was performed at The University of Texas at El Paso (UTEP) within the W.M. Keck Center for 3D Innovation (Keck Center). This material is based on research sponsored by Air Force Research Laboratory under Agreement Number FA8650-20-2-5700. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes, notwithstanding any copyright notation thereon. Additional support was provided by strategic investments via discretionary UTEP Keck Center funds and the Mr. and Mrs. MacIntosh Murchison Chair I in Engineering Endowment at UTEP. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory or the U.S. Government.

Conflict of interest

The authors declare they have no competing interests.

Author contributions

Conceptualization: Colin Lynch, Ryan Wicker, Rene Villalobos Visualization: Colin Lynch, Rene Villalobos Writing-original draft: Colin Lynch, Rene Villalobos Writing-review & editing: All authors

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data

The dataset supporting this study is openly available on Zenodo at https://zenodo.org/records/17137612. The file statsAndDOEinAM.csv contains metadata from a systematic review of experimental designs and statistical analyses in AM, including fields for publication details, design type, analysis type, sample size, justification of sample size, model adequacy, model selection, randomization, and blocking. The repository also includes the R script, which will reproduce the figures and analyses in this manuscript.

References

[1] Wohlers T, Gornet T, Mostow N, Campbell I, Diegel O, Kowen J, et al. History of Additive Manufacturing. SSRN Journal. Published online 2023. doi: 10.2139/ssrn.4474824

[2] Beaman JJ, Bourell DL, Seepersad CC, Kovar D. Additive manufacturing review: Early past to current practice. J Manuf Sci Eng. 2020; 142(11):110812. doi: 10.1115/1.4048193

[3] Gu D, Shi X, Poprawe R, Bourell DL, Setchi R, Zhu J.Material-structure-performance integrated laser-metal additive manufacturing. Science. 2021; 372(6545):eabg1487. doi: 10.1126/science.abg1487

[4] ASTM. Standard F2792-09: Standard Terminology for Additive Manufacturing Technologies. Superseded. United States: ASTM; 2021.

[5] Srivatsan TS, Sudarshan TS. Additive Manufacturing: Innovations, Advances, and Applications. Boca Raton; CRC Press; 2015. doi: 10.1201/b19360

[6] Frazier WE.Metal additive manufacturing: A review. J Mater Eng Perform. 2014; 23:1917-1928. doi: 10.1007/s11665-014-0958-z

[7] Hasanov S, Suhas A, Mithila R, et al. Review on additive manufacturing of multi-material parts: Progress and challenges. J Manuf Mater Process. 2021; 6(1):4. doi: 10.3390/jmmp6010004

[8] ISO/ASTM 52900. Additive Manufacturing-General Principles-Fundamentals and Vocabulary. Switzerland: ISO/ ASTM; 2021.

[9] Blakey-Milner B, Gradl P, Snedden G, et al. Metal additive manufacturing in aerospace: A review. Mater Des. 2021;209:110008. doi: 10.1016/j.matdes.2021.110008

[10] Paolini A, Kollmannsberger S, Rank E. Additive manufacturing in construction: A review on processes, applications, and digital planning methods. Addit Manuf. 2019;30:100894. doi: 10.1016/j.addma.2019.100894

[11] Dilberoglu UM, Gharehpapagh B, Yaman U, Dolen M.The role of additive manufacturing in the era of industry 4.0. Procedia Manuf. 2017; 11:545-554. doi: 10.1016/j.promfg.2017.07.148

[12] Javaid M, Haleem A. Additive manufacturing applications in medical cases: A literature based review. Alexandria J Med. 2018; 54(4):411-422. doi: 10.1016/j.ajme.2017.09.003

[13] Prakash KS, Nancharaih T, Rao VS. Additive manufacturing techniques in manufacturing-an overview. Mater Today Proc. 2018; 5(2):3873-3882. doi: 10.1016/j.matpr.2017.11.642

[14] Wiese M, Kwauka A, Thiede S, Herrmann C. Economic assessment for additive manufacturing of automotive end-use parts through digital light processing (DLP). CIRP J Manuf Sci Technol. 2021; 35:268-280. doi: 10.1016/j.cirpj.2021.06.020

[15] Wong KV, Hernandez A.A review of additive manufacturing. ISRN Mechan Eng. 2012;2012:208760. doi: 10.5402/2012/208760

[16] Pinto JM, Cristobal A, Marcelo EA, et al. Sensitivity analysis of geometric errors in additive manufacturing medical models. Med Eng Phys. 2015; 37(3):328-334. doi: 10.1016/j.medengphy.2015.01.009

[17] Petruse RE, Puşcaşu S, Pascu A, Bondrea I. Key factors towards a high-quality additive manufacturing process with ABS material. Mater Today Proc. 2019; 12:358-366. doi: 10.1016/j.matpr.2019.03.136

[18] Zhang B, Raiyan S, Lei X, et al. An efficient framework for printability assessment in laser powder bed fusion metal additive manufacturing. Addit Manuf. 2021;46:102018. doi: 10.1016/j.addma.2021.102018

[19] Johnson L, Mohamad M, Bing Z, et al. Assessing printability maps in additive manufacturing of metal alloys. Acta Mater. 2019; 176:199-210. doi: 10.1016/j.actamat.2019.07.005

[20] Sun S, Brandt M, Easton MJLAM. Powder bed fusion processes: An overview. In: Laser Additive Manufacturing. Netherlands: Elsevier; 2017. p. 55-77. doi: 10.1016/B978-0-08-100433-3.00002-6

[21] Fisher RA.Design of experiments. BMJ. 1936; 1(3923):554. doi: 10.1136/bmj.1.3923.554-a

[22] Hicks CR. Fundamental Concepts in the Design of Experiments. New York: Holt, Rinehart and Winston; 1964.

[23] Montgomery DC. Design and Analysis of Experiments. United States: John Wiley & Sons; 2017.

[24] Khuri AI, Mukhopadhyay S.Response surface methodology. Wiley Interdiscip Rev Comput Stat. 2010; 2(2):128-149. doi: 10.1002/wics.73

[25] Myers RH, Montgomery DC, Anderson-Cook CM. Response Surface Methodology: Process and Product Optimization Using Designed Experiments. 4^th ed. United States: John Wiley & Sons; 2016.

[26] Lynch CM, Wicker RB, Villalobos JR. An Overview of the Design of Optimal Experiments and Statistical Practices for Scientific and Engineering Applications. ResearchGate; 2024. Available from: https://www.researchgate.net/ publication/383874030_An_overview_of_the_design_ of_optimal_experiments_and_statistical_practices_for_scientific_and_engineering_applications [Last accessed on 2025 Sep 27].

[27] ISO/ASTM International. ISO/ASTM 52901:2016 - Additive Manufacturing-General Principles-Requirements for Purchased AM Parts. United States: ASTM International; 2016.

[28] ISO/ASTM International. ISO/ASTM 52915:2016 - Specification for Additive Manufacturing File Format (AMF), Version 1.2. United States: ASTM International; 2016.

[29] Gijbels I, Prosdocimi I. Loess. Wiley Interdiscip Rev Comput Stat. 2010; 2(5):590-599. doi: 10.1002/wics.104

[30] Schmitz H, Nadvi K.Clustering and industrialization: Introduction. World Dev. 1999; 27(9):1503-1514. doi: 10.1016/S0305-750X(99)00072-8

[31] Wijuniamurti S, Nugroho S, Rachmawati R. Agglomerative nesting (AGNES) method and divisive analysis (DIANA) method for hierarchical clustering on some distance measurement concepts. J Stat Data Sci. 2022; 1(1):7-11. doi: 10.33369/jsds.v1i1.21009

[32] Rojo J, Rivero R, Romero-Morte J, et al. Modeling pollen time series using seasonal-trend decomposition procedure based on LOESS smoothing. Int J Biometeorol. 2017; 61:335-348. doi: 10.1007/s00484-016-1215-y

[33] Dickersin K, Mayo-Wilson E. Standards for design and measurement would make clinical research reproducible and usable. Proc Natl Acad Sci U S A. 2018; 115(11):2590-2594. doi: 10.1073/pnas.1708273114

[34] McNair L. Ethical and regulatory oversight of clinical research: The role of the Institutional Review Board. Exp Biol Med (Maywood). 2022; 247(7):561-566 doi: 10.1177/15353702221078216

[35] ASTM International. ASTM E2709-12: Standard Practice for Demonstrating Capability to Comply with an Acceptance Procedure. United States: ASTM International; 2012.

[36] ASTM International. ASTM F2924-14: Standard Specification for Additive Manufacturing Titanium-6 Aluminum-4 Vanadium with Powder Bed Fusion. United States: ASTM International; 2014.

[37] ASTM International. ASTM F3001-14: Standard Specification for Additive Manufacturing Titanium-6 Aluminum-4 Vanadium ELI (Extra Low Interstitial) with Powder Bed Fusion. United States: ASTM International; 2015.

[38] ASTM International. ASTM F2077-17: Standard Test Methods for Intervertebral Body Fusion Devices. United States: ASTM International; 2017.

[39] ASTM International. ASTM F382-19: Standard Specification and Test Method for Metallic Bone Plates. United States: ASTM International; 2019.

[40] Food and Drug Administration. Technical Considerations for Additive Manufactured Medical Devices: Guidance for Industry and Food and Drug Administration Staff. Washington, DC: US Department of Health and Human Services; 2017.

[41] Vilanova M, Escribano-García R, Guraya T, San Sebastian M. Optimizing laser powder bed fusion parameters for IN-738LC by response surface method. Materials (Basel). 2020; 13(21):4879. doi: 10.3390/ma13214879

[42] Pfaff A, Jäcklein M, Schlager M, et al. An empirical approach for the development of process parameters for laser powder bed fusion. Materials (Basel). 2020; 13(23):5400. doi: 10.3390/ma13235400

[43] Flores Ituarte I, Wiikinkoski O, Jansson A. Additive manufacturing of polypropylene: A screening design of experiment using laser-based powder bed fusion. Polymers (Basel). 2018; 10(12):1293. doi: 10.3390/polym10121293

[44] Tyagi B, Dubey D, Sahai A, Sharma RS. Mechanical properties evaluation of FFF-printed ABS samples based on different process parameters combined with ANOVA and regression analysis. Proc Inst Mech Eng C J Mech Eng Sci. 2023; 238(7):09544062231151540. doi: 10.1177/09544062231151540

[45] Roy RK. A Primer on the Taguchi Method. Michigan: Society of Manufacturing Engineers; 2010.

[46] Vorland CJ, Brown AW, Dawson JA, et al. Errors in the implementation, analysis, and reporting of randomization within obesity and nutrition research: A guide to their avoidance. Int J Obes (Lond). 2021; 45(11):2335-2346. doi: 10.1038/s41366-021-00909-z

[47] Kang M, Ragan BG, Park JH. Issues in outcomes research: An overview of randomization techniques for clinical trials. J Athl Train. 2008; 43(2):215-221. doi: 10.4085/1062-6050-43.2.215

[48] Lynch CM, Montgomery DC. Optimal experimental designs for hypothesis testing with multiple factors: Maximising power for the biological sciences. Int J Exp Des Process Optim. 2024; 7(2):105-114. doi: 10.1504/IJEDPO.2024.140455

[49] Lynch CM, Starkey M, Montgomery D, Pavlic TP, Mizumoto N. More individuals or more groups? Incorporating sampling effort, statistical power, and model accuracy when designing experiments. Behav Ecol Sociobiol. 2025; 79(11). doi: 10.1007/s00265-025-03640-1

[50] Tanco M, Viles E, Jesus Álvarez M, Ilzarbe L. Why is not design of experiments widely used by engineers in Europe? J Appl Stat. 2010; 37(12):1961-1977. doi: 10.1080/02664760903207308

[51] Hoerl RW, Snee RD. Statistical engineering: An idea whose time has come? Am Stat. 2017; 71(3):209-219. doi: 10.1080/00031305.2016.1247015

[52] Shrout PE, Rodgers JL. Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annu Rev Psychol. 2018;69:487-510. doi: 10.1146/annurev-psych-122216-011845

[53] Zhang W, Yan S, Tian B, Fei D. Statistical assumptions and reproducibility in psychology: Data mining based on open science. Front Psychol. 2022;13:905977. doi: 10.3389/fpsyg.2022.905977

[54] Frost J. Interpreting Correlation Coefficients. Statistics by Jim; 2019. Available from: https://statisticsbyjim.com/basics/correlations [Last accessed on 2025 Sep 27].

[55] Taguchi G. Introduction to Quality Engineering: Designing Quality into Products and Processes. Tokyo: Asian Productivity Organization; 1986.

[56] Taguchi G. System of Experimental Design: Engineering Methods to Optimize Quality and Minimize Costs. Vol. 1. Dearborn: UNIPUB/Kraus International Publications; 1987.

[57] Borror CM, Montgomery DC. Mixed resolution designs as alternatives to Taguchi inner/outer array designs for robust design problems. Qual Reliab Eng Int. 2000; 16(2):117-127. doi: 10.1002/(SICI)1099-1638(200003/04)16:2<117:AID-QRE309>3.0.CO;2-0

[58] Vining GG, Myers RH. Combining Taguchi and response surface philosophies: A dual response approach. J Qual Technol. 1990; 22(1):38-45. doi: 10.1080/00224065.1990.11979204

[59] Samuel S, König-Ries B. Understanding experiments and research practices for reproducibility: An exploratory study. PeerJ. 2021;9:e11140. doi: 10.7717/peerj.11140

[60] Moylan S, Brown CU, Slotwinski J. Recommended protocol for round robin studies in additive manufacturing. J Test Eval. 2016; 44(2):1009. doi: 10.1520/JTE20150317