Academic Commons

Articles

Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial

Yavchitz, Amélie; Ravaud, Philippe; Hopewell, Sally; Baron, Gabriel; Boutron, Isabelle

To allow an accurate evaluation of abstracts of systematic reviews, the PRISMA Statement recommends that the limitations of the evidence (e.g., risk of bias, publication bias, inconsistency, imprecision) should be described in the abstract. We aimed to evaluate the impact of adding such limitations sections on reader’s interpretation. We performed a two-arm parallel group randomized controlled trial (RCT) using a sample of 30 abstracts of systematic reviews evaluating the effects of healthcare intervention with conclusions favoring the beneficial effect of the experimental treatments. Two formats of these abstracts were derived: one reported without and one with a standardized limitations section written according to the PRISMA statement for abstracts. The primary outcome was readers’ confidence in the results of the systematic review as stated in the abstract assessed by a Likert scale from 0, not at all confident, to 10, very confident. In total, 300 participants (corresponding authors of RCT reports indexed in PubMed) were randomized by a web-based randomization procedure to interpret one abstract with a limitations section (n = 150) or without a limitations section (n = 150). Participants were blinded to the study hypothesis. Adding a limitations section did not modify readers’ interpretation of findings in terms of confidence in the results (mean difference [95% confidence interval] 0.19 [−0.37–0.74], p = 0.50), confidence in the validity of the conclusions (0.07 [−0.49–0.62], p = 0.80), or benefit of the experimental intervention (0.12 [−0.42–0.44], p = 0.65). This study is limited because the participants were expert-readers and are not representative of all systematic review readers. Adding a limitations section to abstracts of systematic reviews did not affect readers’ interpretation of the abstract results. Other studies are needed to confirm the results and explore the impact of a limitations section on a less expert panel of participants. Trial registration: ClinicalTrial.gov (NCT01848782).

Subjects

Files

  • thumnail for 1471-2288-14-123-S1.docx 1471-2288-14-123-S1.docx binary/octet-stream 56.3 KB Download File
  • thumnail for sword-563510540289558591070.zip sword-563510540289558591070.zip binary/octet-stream 340 KB Download File

Also Published In

Title
BMC Medical Research Methodology
DOI
https://doi.org/10.1186/1471-2288-14-123

More About This Work

Academic Units
Epidemiology
Publisher
BioMed Central
Published Here
November 26, 2014
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.