Skip navigation
Skip navigation

Publication bias in ecology and evolution: an empirical assessment using the trim and fill method

Jennions, Michael; Moller, Anders Pope

Description

Recent reviews of specific topics, such as the relationship between male attractiveness to females and fluctuating asymmetry or attractiveness and the expression of secondary sexual characters, suggest that publication bias might be a problem in ecology and evolution. In these cases, there is a significant negative correlation between the sample size of published studies and the magnitude or strength of the research findings (formally the 'effect size'). If all studies that are conducted are...[Show more]

dc.contributor.authorJennions, Michael
dc.contributor.authorMoller, Anders Pope
dc.date.accessioned2015-12-10T23:00:54Z
dc.date.available2015-12-10T23:00:54Z
dc.identifier.issn1464-7931
dc.identifier.urihttp://hdl.handle.net/1885/61540
dc.description.abstractRecent reviews of specific topics, such as the relationship between male attractiveness to females and fluctuating asymmetry or attractiveness and the expression of secondary sexual characters, suggest that publication bias might be a problem in ecology and evolution. In these cases, there is a significant negative correlation between the sample size of published studies and the magnitude or strength of the research findings (formally the 'effect size'). If all studies that are conducted are equally likely to be published, irrespective of their findings, there should not be a directional relationship between effect size and sample size; only a decrease in the variance in effect size as sample size increases due to a reduction in sampling error. One interpretation of these reports of negative correlations is that studies with small sample sizes and weaker findings (smaller effect sizes) are less likely to be published. If the biological literature is systematically biased this could undermine the attempts of reviewers to summarise actual biology relationships by inflating estimates of average effect sizes. But how common is this problem? And does it really affect the general conclusions of literature reviews? Here, we examine data sets of effect sizes extracted from 40 peer-reviewed, published meta-analyses. We estimate how many studies are missing using the newly developed 'trim and fill' method. This method uses asymmetry in plots of effect size against sample size ('funnel plots') to detect 'missing' studies. For random-effect models of meta-analysis 38% (15/40) of data sets had a significant number of 'missing' studies. After correcting for potential publication bias, 21% (8/38) of weighted mean effects were no longer significantly greater than zero, and 15% 5/34) were no longer statistically robust when we used random-effects models in a weighted meta-analysis. The mean correlation between sample size and the magnitude of standardised effect size was also significantly negative (rs= -0.20, P < 0.0001). Individual correlations were significantly negative (P < 0.10) in 35% (14/40) of cases. Publication bias may therefore affect the main conclusions of at least 15-21% of meta-analyses. We suggest that future literature reviews assess the robustness of their main conclusions by correcting for potential publication bias using the 'trim and fill' method.
dc.publisherCambridge University Press
dc.sourceBiological Reviews
dc.subjectKeywords: ecology; evolution; meta analysis; publication; review; scientific literature; sex difference; statistical analysis; animal; ecology; female; male; meta analysis; publishing; sample size; Animal; Ecology; Evolution; Female; Male; Meta-Analysis; Publicatio Effect size; Fail-safe number; Fluctuating asymmetry; Funnel plots; Meta-analysis; Publication bias; Trim and fill
dc.titlePublication bias in ecology and evolution: an empirical assessment using the trim and fill method
dc.typeJournal article
local.description.notesImported from ARIES
local.description.refereedYes
local.identifier.citationvolume77
dc.date.issued2002
local.identifier.absfor060311 - Speciation and Extinction
local.identifier.ariespublicationMigratedxPub619
local.type.statusPublished Version
local.contributor.affiliationJennions, Michael, College of Medicine, Biology and Environment, ANU
local.contributor.affiliationMoller, Anders Pope, Universite Pierre et Marie Curie
local.bibliographicCitation.startpage211
local.bibliographicCitation.lastpage222
local.identifier.doi10.1017/S1464793101005875
dc.date.updated2015-12-10T08:25:03Z
local.identifier.scopusID2-s2.0-0036014507
CollectionsANU Research Publications

Download

There are no files associated with this item.


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator