Academic Commons

Articles

A test for reporting bias in trial networks: simulation and case studies

Trinquart, Ludovic; Ioannidis, John; Chatellier, Gilles; Ravaud, Philippe

Networks of trials assessing several treatment options available for the same condition are increasingly considered. Randomized trial evidence may be missing because of reporting bias. We propose a test for reporting bias in trial networks. We test whether there is an excess of trials with statistically significant results across a network of trials. The observed number of trials with nominally statistically significant p-values across the network is compared with the expected number. The performance of the test (type I error rate and power) was assessed using simulation studies under different scenarios of selective reporting bias. Examples are provided for networks of antidepressant and antipsychotic trials, where reporting biases have been previously demonstrated by comparing published to Food and Drug Administration (FDA) data. In simulations, the test maintained the type I error rate and was moderately powerful after adjustment for type I error rate, except when the between-trial variance was substantial. In all, a positive test result increased moderately or markedly the probability of reporting bias being present, while a negative test result was not very informative. In the two examples, the test gave a signal for an excess of statistically significant results in the network of published data but not in the network of FDA data. The test could be useful to document an excess of significant findings in trial networks, providing a signal for potential publication bias or other selective analysis and outcome reporting biases.

Subjects

Files

  • thumnail for 1471-2288-14-112-S1.docx 1471-2288-14-112-S1.docx binary/octet-stream 38.4 KB Download File
  • thumnail for 1471-2288-14-112-S2.docx 1471-2288-14-112-S2.docx binary/octet-stream 443 KB Download File
  • thumnail for 1471-2288-14-112-S3.docx 1471-2288-14-112-S3.docx binary/octet-stream 88.7 KB Download File
  • thumnail for sword-433413914737632947740.zip sword-433413914737632947740.zip binary/octet-stream 1.78 MB Download File

Also Published In

Title
BMC Medical Research Methodology
DOI
https://doi.org/10.1186/1471-2288-14-112

More About This Work

Academic Units
Epidemiology
Publisher
BioMed Central
Published Here
October 7, 2014
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.