Beyond p-values: Assessing clinical significance in acupuncture research

Abstract

In acupuncture randomized controlled trials (RCTs), the proper interpretation of results requires a thorough understanding of key statistical concepts such as p-value, effect size, and the minimal clinically important difference (MCID). This paper explores the relationships among these metrics and their implications for assessing the clinical significance of acupuncture interventions. Through case studies, the paper highlights the limitations of relying solely on p-values for determining the clinical relevance of outcomes. Although statistically significant results may be reported, these studies often fail to meet the prespecified MCID, raising concerns about their real-world applicability. This paper argues that p-values must be interpreted alongside effect size and MCID to ensure that reported benefits reflect meaningful improvements for patients. By examining both short- and long-term outcomes, this analysis emphasizes the need for a holistic approach when evaluating acupuncture RCTs, ensuring that statistical findings are translated into clinically significant improvements.

Keywords

acupuncture / clinical relevance / effect size / minimal clinically important difference (MCID) / p-value / randomized controlled trials (RCTs)

References

[1] Cai Y , Shen J , Zhong D , Li Y , Wu T . Status quo, issues, and challenges for acupuncture research evidence: an overview of clinical and fundamental studies. J Evid Based Med. 2012; 5 (1): 12- 24.

[2] Zhang YQ , Jiao RM , Witt CM , et al. How to design high quality acupuncture trials-a consensus informed by evidence. BMJ. 2022; 376: e067476. Erratum in: BMJ. 2022;377: o1046.

[3] Gang WJ , Gong CZ , Jing XH . Acupuncture randomized controlled trials: comparing China-based vs. Western-based studies. Zhongguo Zhen Jiu. 2022; 42 (1): 3- 7.

[4] Ostojic M , Winkler PW , Karlsson J , Becker R , Prill R . Minimal clinically important difference: don't just look at the "p-value". Knee Surg Sports Traumatol Arthrosc. 2023; 31 (10): 4077- 4079.

[5] Sullivan GM , Feinn R . Using effect size-or why the P value is not enough. J Grad Med Educ. 2012; 4 (3): 279- 282.

[6] Embry TW , Piccirillo JF . Minimal clinically important difference reporting in randomized clinical trials. JAMA Otolaryngol Head Neck Surg. 2020; 146 (9): 862- 863.

[7] Schulz KF , Altman DG , Moher D , CONSORT Group . CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340 (mar23 1): c332.

[8] Fritz CO , Morris PE , Richler JJ . Effect size estimates: current use, calculations, and interpretation. J Exp Psychol Gen. 2012; 141 (1): 2- 18. Erratum in: J Exp Psychol Gen. 2012;141(1): 30.

[9] Decleene KE , Fogo J . Publication manual of the American Psychological Association. Occup Ther Health Care. 2012; 26 (1): 90- 92.

[10] Zhu L , Sun Y , Kang J , et al. Effect of acupuncture on neurogenic claudication among patients with degenerative lumbar spinal stenosis: a randomized clinical trial. Ann Intern Med. 2024; 177 (8): 1048- 1057.

[11] Zhao L , Sun M , Yin Z , et al. Long-term effects of individualized acupuncture for chronic neck pain: a randomized controlled trial. Ann Intern Med. 2024; 177 (10): 1330- 1338.

[12] Ashar YK , Sun M , Knight K , et al. Open-label placebo injection for chronic back pain with functional neuroimaging: a randomized clinical trial. JAMA Netw Open. 2024; 7 (9): e2432427.

[13] Cohen J . Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum; 1988.

[14] Cherkin DC , Sherman KJ , Avins AL , et al. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009; 169 (9): 858- 866.

[15] Lv ZT , Shen LL , Zhu B , et al. Effects of intensity of electroacupuncture on chronic pain in patients with knee osteoarthritis: a randomized controlled trial. Arthritis Res Ther. 2019; 21 (1): 120.

[16] Zheng H , Xiao XJ , Shi YZ , et al. Efficacy of acupuncture for chronic spontaneous urticaria: a randomized controlled trial. Ann Intern Med. 2023; 176 (12): 1617- 1624. Erratum in: Ann Intern Med. 2024;177(2): 268.

[17] Xiao X , Cao W , Zou Z , et al. Minimal clinically important difference for acupuncture for patients with chronic spontaneous urticaria: secondary analysis from a multicentre randomised controlled trial in China. BMJ Open. 2024; 14 (10): e085041.

RIGHTS & PERMISSIONS

The Author(s). Advanced Chinese Medicine published by John Wiley & Sons Australia, Ltd on behalf of Higher Education Press.