P value-driven methods were underpowered to detect publication bias: analysis of Cochrane review meta-analyses

Date

2020

Authors

Furuya-Kanamori, Luis
Xu, Chang
Lin, Lifeng
Doan, Tinh
Chu, Haitao
Thalib, Lukman
Doi, Suhail A R

Journal Title

Journal ISSN

Volume Title

Publisher

Elsevier

Abstract

Objectives The aim of the study was to investigate the effect of number of studies in a meta-analysis on the detection of publication bias using P value-driven methods. Methods The proportion of meta-analyses detected by Egger's, Harbord's, Peters', and Begg's tests to have asymmetry suggestive of publication bias were examined in 5,014 meta-analyses from Cochrane reviews. P values were also assessed in meta-analyses with varying number of studies, whereas symmetry was held constant. A simulation study was conducted to investigate if the above tests underestimate or overestimate the presence of publication bias. Results The proportion of meta-analyses detected as asymmetrical via Egger's, Harbord's, Peters', and Begg's tests decreased by 42.6%, 41.1%, 29.3%, and 28.3%, respectively, when the median number of studies in the meta-analysis decreased from 87 to 14. P values decreased as the number of studies increased in the meta-analysis, despite the level of symmetry remaining constant. The simulation study confirmed that when publication bias is present, P value tests underestimate the presence of publication bias, particularly when study numbers are small. Conclusion P value-based tests used for the detection of publication bias-related asymmetry in meta-analysis require careful examination, as they underestimate asymmetry. Alternative methods not dependent on the number of studies are preferable.

Description

Keywords

Citation

Source

Journal of Clinical Epidemiology

Type

Journal article

Book Title

Entity type

Access Statement

License Rights

DOI

10.1016/j.jclinepi.2019.11.011

Restricted until

2037-12-31