How to use quantitative and qualitative frameworks in whole systems research: detailed examples from two mixed-methods case studies

Abstract

Introduction Evaluating complex interventions such as complementary and alternative medicine (CAM), including traditional Chinese medicine (TCM), requires approaches that capture both context and outcomes. Whole systems research (WSR) employs mixed-methods designs to study these systems in real-world settings. Methods We highlight key methodological considerations for mixed-methods data collection and analysis, illustrated by two case studies: (1) a randomized controlled trial with an embedded process evaluation of spiritual healing for moderate depression; and (2) a cross-sectional survey of oncology providers with written comments (n = 157). Results In Case 1, spiritual healing showed potential benefit for adults with moderate depression. Adherence was high, with minimal dropout and timely questionnaire completion; the process evaluation linked this to close follow-up by the study nurse. In Case 2, communication about CAM differed across provider groups: CAM providers promoted integration with conventional care; physicians were more skeptical, citing limited efficacy evidence; and nurses expressed more positive attitudes. CAM providers and nurses favored strengthening communication between CAM and conventional providers, whereas physicians were less supportive. Qualitative comments enriched and contextualized these patterns. Conclusion Integrating qualitative contextual insights with quantitative outcomes yielded more comprehensive evidence than either method alone, enabling the operationalization of WSR. The detailed procedures for data collection and analysis provide a practical template for students and researchers new to mixed-methods research.

Keywords

whole systems research / mixed-methods / complementary and alternative medicine / traditional Chinese medicine / process evaluation

Funding

This study was funded inhouse by NAFKAM - Norway’s National Research Center in Complementary and Alternative Medicine. UiT, the Arctic University of Norway, Tromsø, Norway.

Author Contribution Statement

Trine Stub: Conceptualization, data curation, methodology, software, formal analysis, visualization, writing - original draft, and writing - review and editing. Terje Alræk: Data curation and writing - review and editing. Agnete E. Kristoffersen: Writing - review and editing.

Declaration of Competing Interest

The first two authors are editorial board members of Evidence-Based Chinese Medicine and Technology Assessment. They have not participated in the peer-review process or the final decision-making for this manuscript, and they declare no other competing interests.

Data Availability

Additional data related to this paper may be obtained from the corresponding author upon reasonable request.

Use of AI Statement

None.

References

[1] Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

[2] Verhoef MJ, Lewith G, Ritenbaugh C, Thomas K, Boon H, Fønnebø V. Whole systems research: moving forward. Focus Altern Complementary Ther. 2004; 9(2):87-90.

[3] Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A. Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT. Complementary Ther Med. 2005; 13(3):206-212.

[4] National Cancer Institute, National Institutes of Health. Complementary and alternative medicine. https://www.cancer.gov/about-cancer/treatment/cam. Accessed February 20, 2026.

[5] MacPherson H, Hammerschlag R, Lewith G, Schnyer R. Acupuncture Research Strategies for Establishing an Evidence Base. London, UK: Churchill Livingstone; 2007: 260.

[6] Verhoef MJ, Casebeer AL, Hilsden RJ. Assessing efficacy of complementary medicine: adding qualitative research methods to the “gold standard”. J Altern Complementary Med. 2002; 8(3):275-281.

[7] Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs: principles and practices. Health Serv Res. 2013; 48(6 Pt2):2134-2156.

[8] O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008; 13(2):92-98.

[9] O’Cathain A, Murphy E, Nicholl J. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res. 2007; 7(1):85.

[10] Patton MQ. Qualitative Research & Evaluation Methods: Integrating Theory and Practice. Thousand Oaks, CA: Sage Publications, Inc.; 2015.

[11] Zheng M. Conceptualization of cross-sectional mixed methods studies in health science: a methodological review. Int J Quant Qual Res Methods. 2015; 3(2):66-87.

[12] French C, Pinnock H, Forbes G, Skene I, Taylor SJC. Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it? a systematic review. Trials. 2020; 21(1):916.

[13] Stephenson J, Imrie J. Why do we need randomised controlled trials to assess behavioural interventions? BMJ. 1998; 316(7131):611-613.

[14] Jekel JF, Katz DL, Elmore JG, Wild DMG. Epidemiology, Biostatistics, and Preventive Medicine. 3rd ed. Philadelphia, PA: Saunders; 2007.

[15] Clement C, Edwards SL, Rapport F, Russell IT, Hutchings HA. Exploring qualitative methods reported in registered trials and their yields (EQUITY): systematic review. Trials. 2018; 19(1):589.

[16] Sackett DL, Rosenberg WMC, Muir Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996; 312(7023):71-72.

[17] Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ. 1995; 311(6996):42-45.

[18] O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open. 2013; 3(6):e002889.

[19] O’Cathain A, Hoddinott P, Lewin S, et al. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot Feasibility Stud. 2015; 1:32.

[20] Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009;339:b3496.

[21] Donovan JL, Rooshenas L, Jepson M, et al. Optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the Quintet Recruitment Intervention (QRI). Trials. 2016; 17(1):283.

[22] Blencowe NS, Blazeby JM, Donovan JL, Mills N. Novel ways to explore surgical interventions in randomised controlled trials: applying case study methodology in the operating theatre. Trials. 2015; 16:589.

[23] Clement C, Rapport F, Seagrove A, Alrubaiy L, Williams J. Healthcare professionals’ views of the use and administration of two salvage therapy drugs for acute ulcerative colitis: a nested qualitative study within the CONSTRUCT trial. BMJ Open. 2017; 7(2):e014512.

[24] Sebire SJ, Edwards MJ, Kesten JM, et al. Process evaluation of the Bristol girls dance project. BMC Public Health. 2016; 16:349.

[25] Mann C, Shaw A, Guthrie B, et al. Protocol for a process evaluation of a cluster randomised controlled trial to improve management of multimorbidity in general practice: the 3D study. BMJ Open. 2016; 6(5):e011260.

[26] Guetterman TC, Fetters MD, Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med. 2015; 13(6):554-561.

[27] Creswell JW, Clark VLP. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage Publications; 2017.

[28] Stub T, Quandt SA, Arcury TA, Sandberg JC, Kristoffersen AE. Complementary and conventional providers in cancer care: experience of communication with patients and steps to improve communication with other providers. BMC Complementary Altern Med. 2017; 17(1):301.

[29] Stub T, Hansen AH, Irgens AC, Knudsen-Baas O, Kristoffersen AE. Assessing the feasibility of a spiritual healing intervention for adults with moderate depression: a pilot randomized controlled trial. Complement Ther Med. 2025; 89:103133.

[30] Van Aken R, Taylor B. Emerging from depression: the experiential process of Healing Touch explored through grounded theory and case study. Complement Ther Clin Pract. 2010; 16(3):132-137.

[31] Steer RA, Cavalieri TA, Leonard DM, Beck AT. Use of the Beck Depression Inventory for Primary Care to screen for major depression disorders. Gen Hosp Psychiatry. 1999; 21(2):106-111.

[32] Arifin WN. Random sampling and allocation using SPSS. Educ Med J. 2012; 4(1):e129-e143.

[33] Patton MQ. Qualitative Research & Evaluation Methods. 3rd ed. Thousand Oaks, CA: Sage Publications; 2002.

[34] Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005; 15(9):1277-1288.

[35] Richards L. Handling Qualitative Data: A Practical Guide. London, UK: Sage Publications; 2005.

[36] Stub T, Quandt SA, Arcury TA, et al. Perception of risk and communication among conventional and complementary health care providers involving cancer patients’ use of complementary therapies: a literature review. BMC Complement Altern Med. 2016; 16(1):353.

[37] de Leeuw ED, Hox JJ, Dillman DA. International Handbook of Survey Methodology. Hove, UK: Psychology Press; 2008.

[38] Morse JM, Richards L. ReadMe First for a User´s Guide to Qualitative Methods. Thousand Oaks, CA: Sage Publications; 2002.

[39] Stöckigt DMB, Kirschbaum B, Carstensen DMM, Witt DMCM, Brinkhaus DMB. Prophylactic acupuncture treatment during chemotherapy in patients with breast cancer-results of a qualitative study nested in a randomized pragmatic trial. Integr Cancer Ther. 2021; 20: 15347354211058207.

[40] Alraek T, Malterud K. Acupuncture for menopausal hot flashes: a qualitative study about patient experiences. J Altern Complement Med. 2009; 15(2):153-158.

[41] Bolster M, Dietzel J, Habermann IV, Hörder S, Brinkhaus B, Stöckigt B. Patients’ experiences with acupuncture for diabetic polyneuropathy as part of a randomized controlled trial (ACUDPN)-a qualitative study. BMC Complement Med Ther. 2025; 25(1):317.

[42] Liu JP, Chen KJ. Methodology guideline for clinical studies investigating traditional Chinese medicine and integrative medicine. Complement Ther Med. 2015; 23(5):751-756.

[43] Bishop FL, Amos N, Yu H, Lewith GT. Health-care sector and complementary medicine: practitioners’ experiences of delivering acupuncture in the public and private sectors. Prim Health Care Res Dev. 2012; 13(3):269-278.

[44] Leng J, Lee T, Sarpel U, et al. Identifying the informational and psychosocial needs of Chinese immigrant cancer patients: a focus group study. Support Care Cancer. 2012; 20(12):3221-3229.

[45] Malterud K. Kvalitative metoder i medisinsk forskning. En innføring. [Qualitative Methods in Medical Research. An Introduction. ] (3rd ed.). Oslo, Norway: Universitetsforlaget; 2011.

[46] Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001; 358(9280):483-488.

[47] Warne M. Med våra gögon. En methodbok om Photovoice. [With Our Eyes. A Method Book about Photovoice]. Sundsvall, Sweden: Mid Sweden University; 2012.