To enhance the clarity and transparency of reporting in trials involving Chinese herbal medicine (CHM) formulas, the CONSORT extension specifically tailored for CHM formulas (CONSORT-CHM Formulas 2017) was introduced in 2017 [
1]. This statement, along with its corresponding checklist, outlines key elements essential for reporting in randomized controlled trials (RCTs) related to the rationale, design, analysis, and interpretation of interventions using CHM formulas. The CONSORT-CHM Formulas 2017 elaborated on 7 of 25 CONSORT checklist items, namely title, abstract and keywords, backgrounds and objectives, participants, interventions, outcomes, generalizability, and interpretation. It is intended to facilitate the principles of transparency, consistency, and full disclosure in the use of CHM formulas in RCTs and in turn aid in their critical appraisal and interpretation, to improve the design and execution of clinical trials involving CHM formulas and contribute to the advancement of TCM [
1]. However, there has been a deficiency of reviews on the reporting quality assessment of RCTs involving CHM formulas interventions since the CONSORT-CHM Formulas 2017 was published. Specific flaws identified in several recent reviews include: (1) inadequate reporting on study design (such as the method of randomization), and (2) a lack of awareness among authors regarding the detailed description of CHM formulas interventions, which easily arise a risk of bias that could impact the reliability and validity of the findings [
2–
7]. Nevertheless, these findings should be interpreted with caution, as they focused on a specific disease along with no comprehensive rule for reporting quality assessment. On the other hand, over five years have passed since the CONSORT-CHM Formulas 2017 was carried out and a considerable number of RCTs involving CHM formulas have been published. For readers looking to know whether CONSORT-CHM Formulas 2017 endorsement influences the completeness and transparency of reporting, it is necessary to incorporate the most comprehensive corpus of literature on this topic. Thus, this study was performed to identify: (1) the reporting characteristics and adherence to the CONSORT-CHM Formulas 2017 checklist; and (2) relevant factors that could impact the reporting quality during the past five years.
This study was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement (PRISMA) 2020 [
8]. The study has been registered on the Open Science Framework, and the protocol has been pre-designed, which specified the detailed methodology for this study [
9]. The detailed search strategies were shown in Additional file 1 of supplementary material. Notably, we assessed the reporting quality of each study using the checklist of CONSORT-CHM Formulas 2017, including 22 extended items. To streamline the process, a customized quality assessment form with 42 specific sub-questions (Additional file 2) has been developed by the core researchers (Xuan Zhang, Chung Wah Cheng, and Zhaoxiang Bian). Each item has been reframed into one or more questions to facilitate a more intricate evaluation. Particularly, we compared the reporting quality of three types of CHM formulas according to the 27 sub-questions. Moreover, we designed subgroup analyses of four factors in the protocol, including distributions of study countries, years of publication, published languages, and different types of CHM formulas. However, since all trials were found to be conducted in China, we failed to analyze the subgroup comparisons regarding study locations in this final report. The Fisher’s exact test and binary logistic regression (SPSS 23.0, IBM Crop.) were used for data analysis. Statistical significance was set at
P < 0.05.
From the electronic search, a total of 24 687 citations were initially retrieved, including 8662 from Ovid database and 112 425 from four Chinese databases. After utilizing Endnote software to remove duplicates, 14 092 articles remained. Following title and abstract screening, 10 292 studies were excluded, and an additional 535 studies were removed after dull-text screening. Finally, a total of 3265 eligible RCTs were finally included. A complete flow diagram is displayed in Additional file 3. The baseline characteristics of 3265 studies were summarized in Additional file 4. All trials were conducted in China, and 3246 of them were published in Chinese along with 19 in English. Around half of trials (54.5%) were aimed at identifying the clinical efficacy of CHM formulas, while 45.4% targeted both the efficacy and safety of the intervention. The common study design was the single center (94.7%), two arms (93.0%), and simple randomization (59.8%). In comparison, only 4.4% of trials adopted blinding. The sample sizes of the included RCTs ranged from 14 to 22 605, with 69.9% fewer than 100 participants. For targeted diseases (Additional file 5), the top 3 were the digestive system (14.7%), genitourinary system (13.2%), and musculoskeletal system or connective tissue (10.8%). For intervention, 61.1% of trials adopted fixed CHM formulas, 21.7% chose individualized prescriptions, and 15.7% used Chinese patent medicines. Active control was predominantly employed (50.5%), followed by the add-on design (44.2%), placebo control (4.7%), and blank control (0.6%). The decoction (63.7%) and oral administration (70.7%) emerged as the predominant dosage form and administration route of CHM formulas (Additional file 6). The treatment duration ranged from 1 day to 86 weeks, of which 42.4% reported 1–4 weeks. Only 307 (9.4%) studies reported a 0.5–240 weeks follow-up design after the treatment. 30.8% of studies reported TCM-related outcomes, including TCM patterns (86.1%) and TCM symptoms (13.9%). 50.8% of the included studies reported the collection of human biological samples. 40.7% of studies reported sources of funding for the trial. Regarding the safety assessment of CHM formulas (Additional file 7), 1264 RCTs (39.3%) reported adverse effects (AEs), of which 820 identified the occurrence of AEs and 444 with no AEs in the trial. The most common (67.3%) measurement time of AEs was during the treatment period, while only 2.4% assessed AEs during follow-up period. The types of measurement methods for safety assessment included physical examination (34.1%), physicochemical test (14.7%), or both (27.6%). 81.7% of trials reported the occurrences of AEs in both intervention and control groups, and 7.1% identified AEs only in the intervention group. Specifically, the common AEs that occurred in the intervention groups included the digestive system damage and the systemic damage. Among them, only 17 studies reported that the AEs were definitely related to CHM formulas.
In terms of reporting quality, the average reporting compliance rate (CR) of 42 sub-questions in 3265 RCTs was 52.1%, with a range of 95.4% to 0.0% (Additional file 2). Of 42 specific questions, six presented a good CR of > 80%, while 11 showed a poor CR of < 30%. The missing and inadequate reporting were distributed as follows: the keywords of “Chinese herbal medicine formulas”(Q5, 8.4%) and “randomized controlled trials” (Q6, 8.3%); the TCM-related background and explanation of the disease or the TCM pattern (Q7, 23.7%); the biomedical science explanation and/or TCM rationale about the CHM formulas (Q8, 21.8%); the source, safety assessment of the CHM formulas (Q12, 24.7%; Q24, 0.0%); the source, quality control, and authentication method of each medical substance (Q15,16.3%; Q18, 0.0%; Q23, 0.1%); the quality control and safety assessment of the placebo (Q37, 15.3%); and the discussion on how the formula works on different TCM patterns or diseases (Q41, 2.3%) (Additional file 8). Notably, several obvious improvements were identified over the past five years (Fig.1), including “Introduction-background and objectives” (Q7–Q9); “Methods-interventions for patent proprietary CHM formulas” (Q29–Q34); “Methods-outcomes” (Q30); and the following four sub-items under the “Methods-interventions for fixed CHM formulas”: “pharmacologic study results of the formula” (Q21); “dosage of the formula” (Q25); “name of each medical substance” (Q14); “dosage of each medical substance” (Q17). In comparison, the “quality control of each ingredient and of the product or formula” (Q23), the “safety assessment of the formula” (Q24), and the “authentication method of each ingredient” (Q18) have not been reported, with the reporting rates were always as 0.0% during the past five years.
For three types of CHM formulas among 27 sub-questions (Additional file 9), the section of “Title, abstract, and keywords” (Q1–Q6) presented higher reporting scores in RCTs with patent proprietary CHM formulas or individualized CHM formulas than those in RCTs with fixed CHM formulas (P < 0.05). Furthermore, five sub-questions (Q35–Q39) within the “Methods-placebo control” section showed better reporting in RCTs with patent proprietary CHM formula interventions than those in RCTs with fixed CHM formula interventions (P < 0.001). In comparison, the sections of “Methods-participants” (Q10) and “Methods-outcomes” (Q40) presented higher scores in RCTs with fixed CHM formulas than those in RCTs with patent proprietary CHM formulas (P < 0.05). Regarding nine sub-questions (Q11–Q13, Q19–Q21, Q25–Q27) within “Methods-interventions,” a total of six showed better reporting in RCTs with patent proprietary CHM formulas than those in fixed CHM formulas studies (P < 0.05). Moreover, the overall reporting rate of these three types of CHM formulas presented significantly different (P < 0.05). In comparison, factors of publication languages and years showed no significant impacts on the reporting quality (P = 0.145). Details of subgroup analyses were shown in Fig.1.
In summary, on the one hand, the methodology-related reporting was inadequate in RCTs with CHM formulas, involving deficiency in randomization, sample size, blinding, and recruitment location diversity, which is quite similar to previous studies [
10]. Although the CHM formulas are applied in China extensively in treating various diseases according to the ICD-11 MMS categories in China, particularly in diseases of the digestive system (14.7%), genitourinary system (13.2%), and musculoskeletal system or connective tissue (10.8%), the conduction of relevant clinical trials legally international to validate their efficacy appears to be few [
11–
15]. The paucity and limited reports (39.3%) of safety outcomes of CHM formulas make it difficult to assign apposite roles to CHM formulas with undue respectability and credibility as well [
16].
On the other hand, deficiency and improvement coexist simultaneously with regard to the adherence of RCTs involving CHM formulas to the CONSORT-CHM Formulas 2017. First, the results showed that the reporting rates of three extended items (Q18, Q23, and Q24) involving the safety administration and quality control of CHM formulas were extremely low, nearly at 0.0%. CHMs are often derived from complex systems, which may be one of the prime reasons to lead to the difficulty in reporting detailed quality control and authentication methods of each medical substance of CHM formulas [
17]. The paucity and limited knowledge of the quality control of CHM formulas lead to the identification of the safest and most effective therapies as well as the promotion of their rational use more difficult [
18]. The detailed and standard quality control and authentication of each substance and the whole CHM formulas indeed contribute to the stability and safety of the clinical application. In addition, less than half of studies that targeted TCM patterns reported the name of TCM pattern in the title and abstract, which could lead to confusion or misinterpretation of the indications of the CHM formulas, and the difficulty of evidence searching for readers and reviewers. Taking the TCM patterns into consideration in the study protocol, conduction, and reporting of RCTs facilitates the elucidation of characteristics of pathologic changes at a certain stage of disease, which state the specific conditions where CHM formulas were applied [
19].
Although more time is necessary to adopt recommendations from guidelines, we strongly suggest following the checklist of CONOSRT-CHM Formulas 2017 on RCTs, and the training courses and tools may be necessary to help researchers comprehend and use the reporting guideline. As suggested by some scholars, when and how the reporting guideline is implemented within the editorial process and who takes responsibility for ensuring adherence to reporting guidelines could impact on RCT reporting [
20]. And another review indicated that journal endorsement of the CONSORT Statement significantly influenced the completeness of reporting of trials published in medical journals [
21]. Based on this study, we advocate for continued, concerted, and coordinated efforts to be initiated by journals, editors, reviewers, and investigators together to address these issues, especially for an emphasis on promoting and endorsing the CONSORT-CHM Formulas 2017 reporting guideline effectively. Detailed discussion about the results was shown in Additional file 10.
PDF
(824KB)
