Highly Cited Articles in Evolutionary Psychiatry: Assessment With a Quality and Error Rating Scale

Abstract

Background:

Evolutionary psychiatry is a growing field that emphasizes the value of evolutionary explanations for traits that make individuals vulnerable to mental disorders. Some articles that apply evolutionary theory in psychiatry make errors, such as viewing a disease as an adaptation. We assessed article quality and error quantity in the most cited articles on evolutionary psychiatry and examined the relationship of these measures to citation number.

Methods:

PubMed, Web of Science, and Google Scholar were searched in 2023 and again in 2025 using specific terms related to “evolution” and “psychiatry”, to find the most highly cited articles in the field. Based on the work of Nesse, we developed a measure for assessing overall article quality and error quantity in evolutionary psychiatry articles. We applied the measure to the 20 most highly cited articles, and calculated the correlations of article quality and error quantity with number of citations.

Results:

Twenty highly cited articles, with a mean citation count of 413.30 and publication year from 1964 to 2011, were rated. While the most highly cited articles had good quality on average, they also made important errors. There was no significant correlation of article quality or error quantity with citation count.

Conclusion:

Highly cited articles in evolutionary psychiatry demonstrated strengths but also weaknesses. The lack of a relationship of article quality and error quantity to citation rates suggests that other factors influence such citations. Future research should focus on achieving consensus on how best to assess the quality of evolutionary psychiatry articles and what errors should be avoided.

Graphical abstract

Keywords

evolutionary psychiatry / systematic review / citation analysis / article quality / common errors / rating scale

Main Points

1. The most highly cited articles in evolutionary psychiatry demonstrated overall good quality, particularly in defining research questions with precision.

2. Important conceptual errors were nevertheless common, with half of the top-cited articles failing to consider alternative hypotheses or presenting unbalanced evidence.

3. No significant relationship was found between article quality, error quantity, and citation count, suggesting citation frequency is driven by factors other than scientific rigor.

4. The rating system derived from Nesse’s quality and error criteria showed reliable application, supporting its potential utility as a standardised evaluative tool for the field.

5. Findings highlight a need for consensus on quality standards in evolutionary psychiatry, as well as improved methodological rigour in specifying and evaluating evolutionary hypotheses.

1. Introduction

The seminal text, “Why We Get Sick” argued that evolutionary theory is useful in understanding traits that leave members of a species vulnerable to disease [1]. Evolutionary medicine and evolutionary psychiatry subsequently emerged as new disciplines, with specialized conferences, textbooks, and journals [2]. A recent review of evolutionary psychiatry updated the range of possible explanations for vulnerability to mental disorders including mismatch of evolutionary history with current environments and evolutionary tradeoffs that have both benefits and costs [3]. Such explanations complement proximal biological explanations that describe mechanisms underlying disease with more distal evolutionary accounts of the origins of vulnerability to failure [3].

Evolutionary medicine and psychiatry have grappled with the conceptual questions of what makes for a high quality publication in the field, and of what conceptual errors continue to be made by contributors [4]. It has been suggested that good articles in the field define their questions with precision and consider multiple possible explanations of observed phenomena [5]. In contrast, errors include viewing disorders as adaptations, and providing explanations based on what is good for the species [4]. The difficulty in delineating normality from psychopathology is a particular problem in psychiatry and evolutionary psychiatry, where the drawing of a line between normal and excessive activation of an adaptive defence often becomes blurry [6]. This article is an effort towards providing some degree of quality standardization for the field.

We developed a measure of the overall quality and error number in evolutionary psychiatry articles, based on Nesse’s work on how to test evolutionary hypotheses, and on errors in the field [4]. Nesse’s work on quality standards is in turn based on extensive engagement with and review of the field. We applied these measures to the most highly cited articles in the field. We assessed article quality and error quantity in the most cited articles on evolutionary psychiatry and examined the relationship of these measures to citation frequency.

2. Methods

2.1 Selection of Studies

Two reviewers (CB and CR) searched Pubmed (https://pubmed.ncbi.nlm.nih.gov/), Web of Science (https://www.webofscience.com) and Google Scholar (https://scholar.google.com/) on 8 September 2023 and repeated the search on 7 January 2025 for relevant articles using search terms curated for each database, covering “evolution” and “psychiatry”. MeSH terms (https://www.ncbi.nlm.nih.gov/mesh/) were primarily used, however, as most databases do not allow ordering by citation count, this method was augmented by searching reference lists of review articles.

Two authors (CB and CR) reviewed the titles and abstracts of articles obtained by this search strategy. Included in this review were all articles that made evolutionary claims about particular mental disorders, excluding review articles, unless they made specific claims, and those focused on non-human research. The 20 most highly cited studies were included in this study.

2.2 Rating of Studies

Articles were assessed for quality based on the article “How to test an evolutionary hypothesis about disease” [4]. This article lists 4 main objectives, namely: F1—Define the object of explanation with great specificity; F2—Specify all possible alternative hypotheses for why the trait is apparently suboptimal; F3—Make explicit predictions from each hypothesis; F4—Use all available evidence to test the predictions from all alternative hypotheses to arrive at a judgment about the contributions of different factors.

To generate a total quality score (TQS), each article was rated on each of the four objectives using a Likert scale ranging from 1 to 5: to what extent do you believe this paper adequately addresses the following, 1—Strongly disagree; 2—Disagree; 3—Neither agree nor disagree (uncertain); 4—Agree; 5—Strongly agree. The four scores were summed to give a TQS of 4 to 20. This score was then divided by 4 and anchored as follows 1–1.8 = very poor; 1.8–2.6 = poor; 2.6–3.4 = average; 3.4–4.2 = good; 4.2–5.0 = very good.

Articles were then assessed for errors based on the article “Some common mistakes in testing evolutionary hypotheses about disease” [4]. These errors are paraphrased here: FN1—Attempting to explain a disease as if it is an adaptation; FN2—Proposing an explanation based on what is good for the species; FN3—Proposing adaptive functions for rare genetic conditions; FN4—Confusing proximate and evolutionary explanations; FN5—Thinking that evidence for learning influencing a trait indicates that no evolutionary explanation is needed; FN6—Thinking that evidence for environmental or cultural differences in a trait is evidence against evolutionary influences; FN7—Confusing genetic explanations, especially behavioural genetic explanations, with evolutionary explanations; FN8—Failing to consider all of the alternative hypotheses; FN9—Assuming that evidence for one hypothesis is evidence against another; FN10—Presenting all the evidence in favour of a pet hypothesis and all the evidence against other hypotheses, instead of offering a balanced consideration of all evidence for and against all hypotheses.

We assessed each article for evidence of whether each of these errors was made. As points 1–4 above were viewed as more fundamental errors than points 5–10, errors from points 1–4 were each given an error score of 1, while errors from points 5–10 were each given an error score of 0.5. Thus the total error score (TES) could range from 2 to 7.

Finally, each article was assigned an Overall Impression (or global rating) that ranged from 1 (Poor) to 5 (Excellent) (Table 1).

All articles were scored independently by 2 reviewers (CB, CR). Differences were resolved by discussion or by bringing in a third reviewer (DJS).

2.3 Approach

Article quality (as measured by TQS), error quantity (as measured by TES), and Overall Impression scores were subjected to Cohen’s Kappa to assess inter-rater reliability (IRR), prior to resolution by discussion or the third reviewer (DJS). These results were interpreted as follows: values \le0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement.

Article quality (as measured by TQS), error quantity (as measured by TES), and Overall Impression scores for each article were correlated with citation count, with significance assessed using a two-tailed t-test. Spearman correlation was used for data that were non-normally distributed, of ordinal scale (i.e., publication year), or when the relationship between variables was non-linear. Statistical analysis was performed using IBM SPSS Statistics (version 26.0, IBM, Armonk, NY, USA) and differences were considered significant where p 0.05.

3. Results

3.1 Included Studies

We analyzed the 20 most highly cited articles in evolutionary psychiatry [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. The lowest cited article had 130 citations, while the highest cited article had 1000 citations, with a mean of 413.30, median of 362, standard deviation of 250.52, and range of 870. The earliest published article was from 1964, with the most recent article from 2011, with a mean publication year of 2000 and a standard deviation of 10.01. Table 2 (Ref. [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]) shows the included articles, with study ID, Scopus citation count and Field-Weighted Citation Impact (FWCI) where available.

3.2 Study Ratings

Inter-rater reliability was assessed using Cohen’s Kappa, with selected values reported for key scoring domains. Across all domains, there was an overall agreement rate of 68.5%.

Fig. 1 shows the distributions of positive scores (F1–F4) and TQS. Histograms display the frequency of reviewer scores assigned to each quality domain (F1–F4) and the TQS across the 20 included articles. Scores for F1–F4 were recorded on a 1–5 scale; TQS is the mean of the four domain scores. The overall distribution of TQS shows clustering toward both moderate (\approx3) and higher (\approx4+) values. The overall TQS showed a mean of 3.6, standard deviation of 0.6. This falls on the low end of the good score of 3.4–4.2. Notably, the highest scoring section was F1 (M = 3.85, SD = 0.6). There were varying degrees of IRR, with F4 showing the highest level (k = 0.4).

Table 3 shows the results of the error scoring, with number of times the error was made and the percentage. Notably, FN8 and FN10 were found to be the most common errors, each made by ten of the articles assessed (50%). FN5 and FN6 were not found to be made in the assessed articles. The item with the highest IRR was FN1 (k = 0.857).

The mean Overall Impression score was 3.0, with a standard deviation of 1.5. There was moderate agreement in terms of IRR (k = 0.6).

3.3 Results

There was no significant correlation between TQS and either citation count (\rho = –0.123, p = 0.605, n = 20) or field weighted citation impact (FWCI) (\rho = –0.204, p = 0.465, n = 15). The error quantity was not significantly related to citation count (\rho = 0.332, p = 0.152, n = 20) or FWCI (\rho = 0.254, p = 0.360, n = 15). The Overall Impression score was likewise unrelated to citation count (\rho = –0.168, p = 0.478, n = 20) or FWCI (\rho = –0.137, p = 0.627, n = 15). Fig. 2 is a scatterplot which depicts the relationship TQS and citation count, while Fig. 3 is a scatterplot which depicts the relationship between TES and citation count.

The reviewer rating scales were strongly inter‑correlated: TQS was associated with lower TES (\rho = –0.700, p = 0.001, n = 20) and with better Overall Impression (\rho = 0.770, p 0.001, n = 20). Higher TQS was associated with poorer Overall Impression (\rho = –0.726, p 0.001, n = 20).

Publication year did not correlate with citation number (\rho = –0.187, p = 0.431, n = 20) or FWCI (\rho = 0.232, p = 0.405, n = 15).

As expected, citation number and FWCI were strongly positively related (\rho= 0.800, p 0.001, n = 15).

4. Discussion

The main findings of this research were (1) the most highly cited articles have good quality on average, (2) the most highly cited articles make important errors, (3) there was no significant relationship between article quality or error quantity and citation number.

Overall, the top 20 mostly highly cited articles achieved a mean on the low end of the “good” rating for each of the 4 positive scoring metrics (3.6–3.9; with good considered 3.4–4.2). For example, the TQS had a rating of 3.64 (SD 0.6). The included articles scored a mean of 3.85 on F1 (Define the object of explanation with great specificity), indicating that this area represents a strength of the included articles. Conversely, F3 (Make explicit predictions from each possible hypothesis) had a mean of 3.60, while F2 (Specify all possible alternative hypotheses for why the trait is apparently suboptimal) and F4 (Use all available evidence to test the predictions from all alternative hypotheses) had the lowest mean score of included articles (3.55), indicating that these are areas that future articles can improve upon.

Of the errors made, the most common were failing to consider all of the alternative hypotheses and presenting all the evidence in favour of a pet hypothesis and all the evidence against other hypotheses, instead of offering a balanced consideration of all evidence for and against all hypotheses. 50% of the assessed articles made one or both of these errors.

None of the top 20 articles most highly cited articles made the errors of FN5 (Thinking that evidence for learning influencing a trait indicates that no evolutionary explanation is needed) or FN6 (Thinking that evidence for environmental or cultural differences in a trait is evidence against evolutionary influences). These errors may, however, be more prevalent in less cited evolutionary psychiatry articles.

Article quality and error quantity were not significantly associated with citation number. This is consistent with previous work in many fields suggesting that quality and methodological rigor are not necessarily correlated with citation number [27, 28]. In neuroscience, orthopaedics and plastic surgery citation numbers are influenced by factors including age of the article, study design, level of evidence, conflict of interest disclosures, and number of authors, but our study was not designed to investigate such influence [27, 29, 30].

This article indicates that the quality standards developed by Nesse can be assessed reliably, and suggests that their routine application could assist the field. Nesse’s quality standards have face validity insofar as they align with general scientific procedures, and our assessment measure now adds procedural validity insofar as it potentially allows different articles to be rated in a standard way. We cannot provide external validity of this measure against a pre-existing measure as this is the first such measure in the field, and indeed the question of how best to define the optimal “gold standard” for assessing the quality of work in evolutionary psychiatry is perhaps an “essentially contested” question that deserves ongoing attention.

Several additional limitations deserve emphasis. First, we assessed only the top 20 highly cited articles, which limits generalizability to the broader field of evolutionary psychiatry; it is possible that less highly cited articles would receive quite different quality and error scores. Second, we relied solely on Scopus for citation numbers; different databases provide different citation numbers, and may have yielded a different selection of articles. Third, some of the scoring items, particularly FN 1–4 and Overall Impression, involve a degree of subjective judgment; more objective anchoring of scores may be possible in the future. Fourth, we used one particular weighting system for summing error quantity; other weightings could be more useful and may be worth investigating. Fifth, the raters are both physicians rather than experts in evolutionary theory or medicine, and the high kappas obtained may not be generalizable to other raters. Sixth, we relied heavily on the work of Nesse; additional or different criteria may also be useful.

5. Conclusion

Our results suggest that the most highly cited articles in evolutionary psychiatry have both strengths and weaknesses. Strengths include defining the research question with precision and predicting the answers based on a specific hypothesis. Weaknesses include failing to consider and specify alternate hypotheses, and presenting evidence in favour of a pet hypothesis while not using all available evidence. The lack of an association of article quality and error quantity with citation number suggests that other factors influence such citations. While the field has significant promise, further work is needed to achieve consensus on how best to assess the quality of evolutionary psychiatry articles and on what errors should be avoided.

Availability of Data and Materials

Data is available from the authors.

References

[1] Nesse RM, Williams GC. Why we Get Sick: the New Science of Darwinian Medicine. Knopf Doubleday Publishing Group: New York. 2012.

[2] Natterson-Horowitz B, Aktipis A, Fox M, Gluckman PD, Low FM, Mace R, et al. The future of evolutionary medicine: sparking innovation in biomedicine and public health. Frontiers in Science. 2023; 1: 997136. https://doi.org/10.3389/fsci.2023.997136.

[3] Nesse RM. Evolutionary psychiatry: foundations, progress and challenges. World Psychiatry: Official Journal of the World Psychiatric Association (WPA). 2023; 22: 177–202. https://doi.org/10.1002/wps.21072.

[4] Nesse RM. The importance of evolution for medicine.In: Trevathan WR, McKenna JJ, Smith EO (eds.) Evolutionary Medicine (pp. 416–432). 2nd ed. Oxford University Press: New York. 2007

[5] Nesse RM. Ten questions for evolutionary studies of disease vulnerability. Evolutionary Applications. 2011; 4: 264–277. https://doi.org/10.1111/j.1752-4571.2010.00181.x.

[6] Stein DJ, Nesse RM. Tacit Creationism Encourages Oversimplified Views of Functions and Dysfunctions. The European Journal of Neuroscience. 2025; 61: e70028. https://doi.org/10.1111/ejn.70028.

[7] Allen NB, Badcock PBT. The social risk hypothesis of depressed mood: evolutionary, psychosocial, and neurobiological perspectives. Psychological Bulletin. 2003; 129: 887–913. https://doi.org/10.1037/0033-2909.129.6.887.

[8] Andrews PW, Thomson JA, Jr. The bright side of being blue: depression as an adaptation for analyzing complex problems. Psychological Review. 2009; 116: 620–654. https://doi.org/10.1037/a0016242.

[9] Bateson M, Brilot B, Nettle D. Anxiety: an evolutionary approach. Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie. 2011; 56: 707–715. https://doi.org/10.1177/070674371105601202.

[10] Brüne M. “Theory of mind” in schizophrenia: a review of the literature. Schizophrenia Bulletin. 2005; 31: 21–42. https://doi.org/10.1093/schbul/sbi002.

[11] Crespi B, Badcock C. Psychosis and autism as diametrical disorders of the social brain. The Behavioral and Brain Sciences. 2008; 31: 241–261; discussion 261–320. https://doi.org/10.1017/S0140525X08004214.

[12] Crow TJ. Schizophrenia as the price that homo sapiens pays for language: a resolution of the central paradox in the origin of the species. Brain Research. Brain Research Reviews. 2000; 31: 118–129. https://doi.org/10.1016/s0165-0173(99)00029-6.

[13] Gilbert P, Allan S. The role of defeat and entrapment (arrested flight) in depression: an exploration of an evolutionary view. Psychological Medicine. 1998; 28: 585–598. https://doi.org/10.1017/s0033291798006710.

[14] Hagen EH. The Functions of Postpartum Depression. Evolution and Human Behavior. 1999; 20: 325–359. https://doi.org/10.1016/S1090-5138(99)00016-1.

[15] Huxley J, Mayr E, Osmond H, Hoffer A. Schizophrenia as a genetic morphism. Nature. 1964; 204: 220–221. https://doi.org/10.1038/204220a0.

[16] Jonason PK, Li NP, Webster GD, Schmitt DP. The dark triad: Facilitating a short‐term mating strategy in men. European Journal of Personality. 2009; 23: 5–18. https://psycnet.apa.org/doi/10.1002/per.698.

[17] Klein DF. False suffocation alarms, spontaneous panics, and related conditions. An integrative hypothesis. Archives of General Psychiatry. 1993; 50: 306–317. https://doi.org/10.1001/archpsyc.1993.01820160076009.

[18] Marks IF, Nesse RM. Fear and fitness: an evolutionary analysis of anxiety disorders. Ethology and Sociobiology. 1994; 15: 247–261. https://doi.org/10.1016/0162-3095(94)90002-7.

[19] Nesse R. Emotional disorders in evolutionary perspective. The British Journal of Medical Psychology. 1998; 71: 397–415. https://doi.org/10.1111/j.2044-8341.1998.tb01000.x.

[20] Nesse RM. Is depression an adaptation? Archives of General Psychiatry. 2000; 57: 14–20. https://doi.org/10.1001/archpsyc.57.1.14.

[21] Nesse RM. Natural selection and the regulation of defenses: A signal detection analysis of the smoke detector principle. Evolution and Human Behavior. 2005; 26: 88–105. https://doi.org/10.1016/j.evolhumbehav.2004.08.002.

[22] Mealey L. The sociobiology of sociopathy: an integrated evolutionary model. Behavioral and Brain Sciences. 1995; 18: 523–541. https://psycnet.apa.org/doi/10.1017/S0140525X00039595.

[23] Nettle D, Clegg H. Schizotypy, creativity and mating success in humans. Proceedings. Biological Sciences. 2006; 273: 611–615. https://doi.org/10.1098/rspb.2005.3349.

[24] Price J, Sloman L, Gardner R, Jr, Gilbert P, Rohde P. The social competition hypothesis of depression. The British Journal of Psychiatry: the Journal of Mental Science. 1994; 164: 309–315. https://doi.org/10.1192/bjp.164.3.309.

[25] Sloman L, Gilbert P, Hasey G. Evolved mechanisms in depression: the role and interaction of attachment and social rank in depression. Journal of Affective Disorders. 2003; 74: 107–121. https://doi.org/10.1016/s0165-0327(02)00116-7.

[26] Watson PJ, Andrews PW. Toward a revised evolutionary adaptationist analysis of depression: the social navigation hypothesis. Journal of Affective Disorders. 2002; 72: 1–14. https://doi.org/10.1016/s0165-0327(01)00459-1.

[27] Lopez J, Calotta N, Doshi A, Soni A, Milton J, May JW, Jr, et al. Citation Rate Predictors in the Plastic Surgery Literature. Journal of Surgical Education. 2017; 74: 191–198. https://doi.org/10.1016/j.jsurg.2016.08.005.

[28] Mackinnon S, Drozdowska BA, Hamilton M, Noel-Storr AH, McShane R, Quinn T. Are methodological quality and completeness of reporting associated with citation-based measures of publication impact? A secondary analysis of a systematic review of dementia biomarker studies. BMJ Open. 2018; 8: e020331. https://doi.org/10.1136/bmjopen-2017-020331.

[29] Jamjoom HZ, Gahtani AY, Jamjoom AB. Predictors of citation rates for research publications in Neurosciences. Neurosciences (Riyadh, Saudi Arabia). 2022; 27: 116–120. https://doi.org/10.17712/nsj.2022.2.20210145.

[30] Okike K, Kocher MS, Torpey JL, Nwachukwu BU, Mehlman CT, Bhandari M. Level of evidence and conflict of interest disclosure associated with higher citation rates in orthopedics. Journal of Clinical Epidemiology. 2011; 64: 331–338. https://doi.org/10.1016/j.jclinepi.2010.03.019.

Funding

South African Medical Research Council