Suboptimal reporting of randomized controlled trials on non-pharmacological therapies in Chinese medicine

Abstract

With the successive release of the CONSORT extensions for acupuncture, moxibustion, cupping, and Tuina/massage, this review aims to assess the reporting characteristics and quality of randomized controlled trials (RCTs) based on these specific guidelines. A comprehensive review was conducted by searching multiple databases, including Embase, Ovid MEDLINE(R), All EBM Reviews, AMED, CNKI, VIP Chinese Medical Journal Database, and Wanfang Data, for publications from January 1 to December 31, 2022. Two reviewers independently evaluated the eligibility of the records, extracted predetermined information, and assessed the reporting based on the STRICTA, STRICTOM, STRICTOC, and STRICTOTM checklists. Among the included 387 studies (acupuncture, 213; Tuina/massage, 85; moxibustion, 73; cupping, 16), the overall reporting compliance averaged 56.0%, with acupuncture leading at 62.6%, followed by cupping (60.2%), moxibustion (53.1%), and Tuina/massage (47.9%). About half of the evaluated items showed poor reporting (compliance rate < 65%). Notably, international journals demonstrated significantly higher reporting quality than Chinese journals (P < 0.05). Although acupuncture trials had relatively higher compliance rates, deficiencies persist in reporting non-pharmacological therapies of Chinese medicine, particularly in areas like treatment environment details and provider background information.

Keywords

CONSORT / Chinese medicine / reporting guideline / STRICTA / STRICTOM / STRICTOC / STRICTOTM

References

[1] Ninot G, Agier S, Bacon S, Berr C, Boulze I, Bourrel G, Carbonnel F, Clément V, David M, Gerazime A, Gomez A, Guerdoux-Ninot E, Laurent A, Lavoie K, Libourel T, Lognos B, Maffre F, Maitre J, Martin S, Mercier G, Nalpas B, Nguyen TL, Engberink AO, Pujol JL, Quantin X, Rapior S, Senesse P, Stoebner-Delbarre A, Trouillet R. La Plateforme CEPS: Une structure universitaire de réflexion sur l’évaluation des interventions non médicamenteuses (INM). Hegel 2017; 7(1): 53–56

[2] Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P; CONSORT NPT Group. CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann Intern Med 2017; 167(1): 40–47

[3] Fan H, Lu F, Yang A, Dong Y, Liu P, Wang Y. A review on the nonpharmacological therapy of traditional Chinese medicine with antihypertensive effects. Evid Based Complement Alternat Med 2019; 2019: 1317842

[4] Coyle ME, Smith C, Peat B. Cephalic version by moxibustion for breech presentation. Cochrane Database Syst Rev 2023; 5(5): CD003928

[5] Ernst E. The safety of massage therapy. Rheumatology (Oxford) 2003; 42(9): 1101–1106

[6] Xie Q, Yang QL, Wang ZX, Wang H, Zhao JJ, Li J, Wang RQ, Du J. A study of non-pharmacologic treatments for depression in Chinese medicine. J Basic Chin Med (Zhongguo Zhong Yi Ji Chu Yi Xue Za Zhi) 2022; 28(3): 491–494

[7] Castellano-Tejedor C. Non-pharmacological interventions for the management of chronic health conditions and non-communicable diseases. Int J Environ Res Public Health 2022; 19(14): 8536

[8] Qaseem A, Wilt TJ, McLean RM, Forciea MA; Clinical Guidelines Committee of the American College of Physicians; Denberg TD, Barry MJ, Boyd C, Chow RD, Fitterman N, Harris RP, Humphrey LL, Vijan S. Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American College of Physicians. Ann Intern Med 2017; 166(7): 514–530

[9] Deng H, Shen X. The mechanism of moxibustion: ancient theory and modern research. Evid Based Complement Alternat Med 2013; 2013: 379291

[10] Choi TY, Ang L, Ku B, Jun JH, Lee MS. Evidence map of cupping therapy. J Clin Med 2021; 10(8): 1750

[11] Lee MS, Kim JI, Ernst E. Is cupping an effective treatment? An overview of systematic reviews. J Acupunct Meridian Stud 2011; 4(1): 1–4

[12] Kim SK, Bae H. Acupuncture and immune modulation. Auton Neurosci 2010; 157(1-2): 38–41

[13] Wang LL. Characteristic of moxibustion and its warming-dredging effect. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2011; 31(10): 865–868

[14] Yang X, Zhao H, Wang J. Chinese massage (Tuina) for the treatment of essential hypertension: a systematic review and meta-analysis. Complement Ther Med 2014; 22(3): 541–548

[15] Pibouleau L, Boutron I, Reeves BC, Nizard R, Ravaud P. Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review. BMJ 2009; 339: b4538

[16] Choi TY, Ang L, Jun JH, Alraek T, Lee MS. Acupuncture and moxibustion for cancer-related fatigue: an overview of systematic reviews and meta-analysis. Cancers (Basel) 2022; 14(10): 2347

[17] Zhang X, Zhang L, Xiong W, Wang X, Zhou X, Zhao C, Tian G, Shang H, Wu T, Miao J, Bian Z. Assessment of the reporting quality of randomised controlled trials of massage. Chin Med 2021; 16(1): 64

[18] Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350: h1258

[19] MacPherson H, Altman DG, Hammerschlag R, Youping L, Taixiang W, White A, Moher D; STRICTA Revision Group. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. J Evid Based Med 2010; 3(3): 140–155

[20] Kang WL, Xiao XJ, Fan R, Zhong DL, Li YX, She J, Li J, Feng Y, Jin RJ. Acupuncture for tension-type headache: a systematic review and meta-analysis of randomized controlled trials. Front Neurol 2023; 13: 943495

[21] Dong Q, Li X, Yuan P, Chen G, Li J, Deng J, Wu F, Yang Y, Fu H, Jin R. Acupuncture for carpal tunnel syndrome: a systematic review and meta-analysis of randomized controlled trials. Front Neurosci 2023; 17: 1097455

[22] Cheng CW, Fu SF, Zhou QH, Wu TX, Shang HC, Tang XD, Liu ZS, Liu J, Lin ZX, Lao L, Lü AP, Zhang BL, Liu BY, Bian ZX. Extending the CONSORT statement to moxibustion. J Integr Med 2013; 11(1): 54–63

[23] Zhang X, Tian R, Lam WC, Duan Y, Liu F, Zhao C, Wu T, Shang H, Tang X, Lyu A, Bian Z. Standards for reporting interventions in clinical trials of cupping (STRICTOC): extending the CONSORT statement. Chin Med 2020; 15(1): 10

[24] Zhang X, Liang F, Lau CT, Chan JC, Wang N, Deng J, Wang J, Ma Y, Zhong LLD, Zhao C, Yao L, Wu T, Lyu A, Tian G, Shang H, Miao J, Bian Z; STRICTOTM Working Group. STandards for Reporting Interventions in Clinical Trials Of Tuina/Massage (STRICTOTM): extending the CONSORT statement. J Evid Based Med 2023; 16(1): 68–81

[25] Yang J XJ, Yuan T, Wang X. Analysis of application of acupuncture and moxibustion RCT in CONSORT series report standard evaluation in recent 10 years. Chin Arch Tradit Chin Med (Zhonghua Zhong Yi Yao Xue Kan) 2021; 39(1): 250–253,17

[26] Ma B, Chen ZM, Xu JK, Wang YN, Chen KY, Ke FY, Niu JQ, Li L, Huang CB, Zheng JX, Yang JH, Zhu QG, Wang YP. Do the CONSORT and STRICTA checklists improve the reporting quality of acupuncture and moxibustion randomized controlled trials published in Chinese journals? A systematic review and analysis of trends. PLoS One 2016; 11(1): e0147244

[27] Kim SY, Lee EJ, Jeon JH, Kim JH, Jung IC, Kim YI. Quality assessment of randomized controlled trials of moxibustion using STandards for Reporting Interventions in Clinical Trials of Moxibustion (STRICTOM) and risk of bias (ROB). J Acupunct Meridian Stud 2017; 10(4): 261–275

[28] Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 2020; 18(10): 2119–2126

[29] Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, Macdonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, TunçalpÖ, Straus SE. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169(7): 467–473

[30] Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005; 37(5): 360–363

[31] Stevanovic A, Schmitz S, Rossaint R, Schürholz T, Coburn M. CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: a retrospective analysis. PLoS One 2015; 10(5): e0128061

[32] Yun X, Yaolong C, Zhao Z, Qi Z, Yangyang W, Runshen X, Xiuli X, Hui L. Using the RIGHT statement to evaluate the reporting quality of clinical practice guidelines in traditional Chinese medicine. PLoS One 2018; 13(11): e0207580

[33] Severin A, Strinzel M, Egger M, Barros T, Sokolov A, Mouatt JV, Müller S. Relationship between journal impact factor and the thoroughness and helpfulness of peer reviews. PLoS Biol 2023; 21(8): e3002238

[34] Li H, Zheng X, Zhang QC, Chen YM, Wang TF. Leading role of channel-based syndrome differentiation in acupuncture practice. Acupunct Res (Zhen Ci Yan Jiu) 2010; 35(2): 142–145

[35] Chen C, Xu B. Acupuncture-moxibustion syndrome differentiation based on differences between jing-jin diseases and zangfu diseases. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2017; 37(10): 1105–1107

[36] Liu B. Study of the multidimensional perspectives on the theory of syndrome differentiation and treatment with acupuncture and moxibustion. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2017; 37(6): 653–657

[37] Zhang X, Tian R, Zhao C, Birch S, Lee JA, Alraek T, Bovey M, Zaslawski C, Robinson N, Kim TH, Lee MS, Bian ZX. The use of pattern differentiation in WHO-registered traditional Chinese medicine trials —a systematic review. Eur J Integr Med 2019; 30: 100945

[38] Li J, Li YX, Luo LJ, Ye J, Zhong DL, Xiao QW, Zheng H, Geng CM, Jin RJ, Liang FR. The effectiveness and safety of acupuncture for knee osteoarthritis: an overview of systematic reviews. Medicine (Baltimore) 2019; 98(28): e16301

[39] Vickers AJ, Vertosick EA, Lewith G, MacPherson H, Foster NE, Sherman KJ, Irnich D, Witt CM, Linde K; Acupuncture Trialists’ Collaboration. Acupuncture for chronic pain: update of an individual patient data meta-analysis. J Pain 2018; 19(5): 455–474

[40] Ifrim Chen F, Antochi AD, Barbilian AG. Acupuncture and the retrospect of its modern research. Rom J Morphol Embryol 2019; 60(2): 411–418

[41] Yao JP, Chen LP, Xiao XJ, Hou TH, Zhou SY, Xu MM, Wang K, Hou YJ, Zhang L, Li Y. Effectiveness and safety of acupuncture for treating functional constipation: an overview of systematic reviews. J Integr Med 2022; 20(1): 13–25

[42] Wu Q, Wang J, Han D, Hu H, Gao H. Efficacy and safety of acupuncture and moxibustion for primary tinnitus: a systematic review and meta-analysis. Am J Otolaryngol 2023; 44(3): 103821

[43] Xiao XJ, Zhang LX, Shi YZ, Yao JP, Cao W, Liu Y, Zou ZH, Zhou SY, Chen ML, Li CX, Zheng QH, Li Y. Cupping therapy for patients with chronic urticaria: a systematic review and meta-analysis. J Integr Med 2020; 18(4): 303–312

[44] Liu Z, Gang L, Yunwei M, Lin L. Clinical efficacy of infantile massage in the treatment of infant functional constipation: a meta-analysis. Front Public Health 2021; 9: 663581

[45] Won J, Lee JH, Bang H, Lee H. Safety of acupuncture by Korean Medicine Doctors: a prospective, practice-based survey of 37, 490 consultations. BMC Complement Med Ther 2022; 22(1): 300

[46] Yin P, Gao N, Wu J, Litscher G, Xu S. Adverse events of massage therapy in pain-related conditions: a systematic review. Evid Based Complement Alternat Med 2014; 2014: 480956

[47] Liu CY, Wang J, Zeng YL, Li Y, Xin QL, Xu WN. Analysis of the academic features of the moxibustion of Xin’an physicians. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2019; 39(2): 203–206

[48] Fan XY, Wen YY, Yang CY, Luo NS, Lin Q, Fan GQ. Analysis of the relationship between migraine syndrome differentiation and acupuncture effect. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2020; 40(1): 96–102

[49] Kim SH, Jeong JH, Lim JH, Kim BK. Acupuncture using pattern-identification for the treatment of insomnia disorder: a systematic review and meta-analysis of randomized controlled trials. Integr Med Res 2019; 8(3): 216–226

[50] Tang C, Duan Y, Zhang Y, Zhang Y, Chen Z, Tang X, Zeng J, Riley D, Lee MS, Kim YS, Zhao H, Marrone G, Zhu X, Ge S, Wen H, Zhao W, Zheng X, Solos I, Chen Y, Guyatt G, Lu L, Xu N; RIGHT (Reporting Items for Practice Guidelines in Healthcare) for Acupuncture Working Group. RIGHT for acupuncture: an extension of the RIGHT statement for clinical practice guidelines on acupuncture. J Clin Epidemiol 2021; 139: 330–339

[51] Xie R, Xia Y, Chen Y, Li H, Shang H, Kuang X, Xia L, Guo Y. The RIGHT Extension statement for traditional Chinese medicine: development, recommendations, and explanation. Pharmacol Res 2020; 160: 105178

[52] Lee YS, Kim SY, Lee H, Chae Y, Lee MS. ACURATE: a guide for reporting sham controls in trials using acupuncture. J Evid Based Med 2023; 16(1): 82–90

[53] De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden; International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Lancet 2004; 364(9438): 911–912

[54] Su XT, Sun N, Zhang N, Wang LQ, Zou X, Li JL, Yang JW, Shi GX, Liu CZ. Effectiveness and safety of acupuncture for vascular cognitive impairment: a systematic review and meta-analysis. Front Aging Neurosci 2021; 13: 692508

[55] Yang J, Yu F, Lin K, Qu H, He Y, Zhao J, Feng F, Pan L, Kui Y. Assessment of reporting quality in randomized controlled trials of acupuncture for primary insomnia with CONSORT statement and STRICTA guidelines. Evid Based Complement Alternat Med 2022; 2022: 5157870

[56] Wu XLX. Analysis of the impact of acupuncture manipulation details for the objectivity of acupuncture clinical efficacy—take acupuncture RCT published on the four top medical journals in recent 10 Years for instance. J Liaoning Univ Tradit Chin Med (Liaoning Zhong Yi Yao Da Xue Xue Bao) 2017; 19(1): 117–121

[57] Wei C, Xu Y, Deng X, Gao S, Wan X, Chen J. Assessment of reporting completeness in acupuncture studies on patients with functional constipation using the STRICTA guidelines. Complement Ther Med 2022; 70: 102849

[58] Fernández-Jané C, Solà-Madurell M, Yu M, Liang C, Fei Y, Sitjà-Rabert M, Úrrutia G. Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: review of adherence to the STRICTA statement. F1000 Res 2020; 9: 226

[59] Chen D, Ni XX, Wang LJ, Zeng Q, Xie YJ, Zhao L. Literature quality analysis of RCTs regarding acupuncture for chronic neck pain. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2019; 39(8): 889–895

[60] Ruan H, Zhang H, Feng Z, Li X, Han W, Si Y, Li J. Assessment of completeness of reporting in randomized controlled trials of acupuncture therapy for chronic obstructive pulmonary disease. Int J Gen Med 2022; 15: 5335–5348

[61] Hughes JG, Lewith G, MacPherson H, Witt CM, Cummings M, Fisher P. Assessment of the quality of reporting in studies of acupuncture for patients with cancer using the STRICTA guidelines. Acupunct Med 2019; 37(4): 223–227

[62] Kim KH, Kang JW, Lee MS, Lee JD. Assessment of the quality of reporting in randomised controlled trials of acupuncture in the Korean literature using the CONSORT statement and STRICTA guidelines. BMJ Open 2014; 4(7): e005068

[63] Svenkerud S, MacPherson H. The impact of STRICTA and CONSORT on reporting of randomised control trials of acupuncture: a systematic methodological evaluation. Acupunct Med 2018; 36(6): 349–357

[64] Aboushanab TS, AlSanad S. Cupping therapy: an overview from a modern medicine perspective. J Acupunct Meridian Stud 2018; 11(3): 83–87

[65] Kim TH, Kim KH, Choi JY, Lee MS. Adverse events related to cupping therapy in studies conducted in Korea: a systematic review. Eur J Integr Med 2014; 6(4): 434–440

[66] Deng RY, Chen RX, Xing J. Evaluation of RCT report quality of moxibustion therapy for postherpetic neuralgia based on CONSORT statement and STRICTOM. China J Tradit Chin Med Pharm (Zhonghua Zhong Yi Yao Za Zhi) 2017; 32(4): 1715–1718

[67] Gong HB, Fu Y, Xiao XF, Jin LQ, Xie FS. CONSORT combined STRICTOM on the rule of law of moxibustion evaluation of the quality of RCT test for treatment of allergic rhinitis. Jiangxi J Tradit Chin Med (Jiangxi Zhong Yi Yao) 2016; 47(4): 56–60

[68] Cao QA, Fu Y, Dong ZW, Li XQ, Xiong J, Zhang HF, Jin LQ, Zhou X. Methodology and reporting quality research of moxibustion clinical study. J Liaoning Univ Tradit Chin Med (Liaoning Zhong Yi Yao Da Xue Xue Bao) 2022; 49(9): 217–220

[69] Bervoets DC, Luijsterburg PA, Alessie JJ, Buijs MJ, Verhagen AP. Massage therapy has short-term benefits for people with common musculoskeletal disorders compared to no treatment: a systematic review. J Physiother 2015; 61(3): 106–116

[70] Baroncini A, Maffulli N, Eschweiler J, Molsberger F, Klimuch A, Migliorini F. Acupuncture in chronic aspecific low back pain: a Bayesian network meta-analysis. J Orthop Surg Res 2022; 17(1): 319

[71] Field T. Massage therapy research review. Complement Ther Clin Pract 2014; 20(4): 224–229

[72] Field TM. Massage therapy effects. Am Psychol 1998; 53(12): 1270–1281

[73] Yu X, Zhang L, Yao JP, Zou ZH, Wen X. LI Y. Quality evaluation of Chinese and English literature reports of randomized controlled trials of acupuncture for the treatment of simple obesity. World Chin Med (Shi Jie Zhong Yi Yao) 2022; 17(20): 2873–2880

[74] Xiu WC, Meng X, Hu XY, Shi LJ, Gang WJ, Jing XH. Evaluation of the report quality of Chinese and English randomized controlled trials of acupuncture based on CONSORT statement and STRICTA checklist. Chin Acupunct Moxibustion (Zhongguo Zhen Jiu) 2023; 43(3): 355–361

[75] Yang Y, Wang T, Dong J, Tang L, Wang Y, Li N, Zhao L. Dynamic pain-related changes in pulse-graph measurements in patients with primary dysmenorrhea before and after electroacupuncture intervention and its correlation with TCM pattern. Evid Based Complement Alternat Med 2022; 2022: 3518179

[76] Fan XY, Chen J, Yue H, Dong YH, Xiao J, Chen M. Chinese medicine individualized treatment and precision medicine for hemorrhoid disease. Acta Chin Med Pharm (Zhong Yi Yao Xue Bao) 2020; 48(8): 40–43

[77] Ren LL, Dai GH, Gao WL, Guan H, Wei M, Dai AQ. Evaluation index of individualized curative effect based on TCM syndrome differentiation Construction ideas and methods. World Sci Tech—Mod Tradit Chin Med (Shi Jie Ke Xue Ji Shu—Zhong Yi Yao Xian Dai Hua) 2022; 24(2): 789–794

[78] Huang S, Hu M, Ng EHY, Stener-Victorin E, Zheng Y, Wen Q, Wang C, Lai M, Li J, Gao X, Wang X, Hu Z, Xia T, Hu R, Liu J, Wen X, Li S, Quan K, Liang X, Shang H, Ma H, Qiao J. A multicenter randomized trial of personalized acupuncture, fixed acupuncture, letrozole, and placebo letrozole on live birth in infertile women with polycystic ovary syndrome. Trials 2020; 21(1): 239

[79] Ko SJ, Kuo B, Kim SK, Lee H, Kim J, Han G, Kim J, Kim SY, Jang S, Son J, Kim M, Lee H, Yeo I, Joo KR, Park JW. Individualized acupuncture for symptom relief in functional dyspepsia: a randomized controlled trial. J Altern Complement Med 2016; 22(12): 997–1006

[80] Ioannidis JPA, Evans SJW, Gøtzsche PC, O’Neill RT, Altman DG, Schulz K, Moher D; CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004; 141(10): 781–788

[81] Gao Y. A review on TCM pattern standardization research. J Beijing Univ Tradit Chin Med (Beijing Zhong Yi Yao Da Xue Xue Bao) 1995; 18(5): 51–53

[82] Qiu RJ, Li M, Hu JY, Chen J, Shang HC. Methods for development of a core outcome set for clinical trials integrating traditional Chinese medicine and Western medicine. J Integr Med 2021; 19(5): 389–394

[83] EQUATORNetwork. Reporting checklists for diagnostic criteria for TCM syndromes. Registered 10 March 2021. Available at the website of EQUATOR Network

[84] ChineseEQUATOR Centre. Application research on reporting guidelines of TCM clinical trials has been successfully conducted. 2023. Available at the website of Chinese EQUATOR Centre

RIGHTS & PERMISSIONS

Higher Education Press