This text is
based on pp 32-33 of Hemilä
(2006)
These documents have up to date links to documents that are available
via
the net.
Harri Hemilä
Department of Public Health
University of Helsinki,
Helsinki, Finland
harri.hemila@helsinki.fi
In addition to the meta-analysis of intravenous magnesium in acute
myocardial infarction, there are several other examples of misleading
conclusions from meta-analyses. LeLorier et al. (1997) compared the
results of 12 large trials with meta-analyses published earlier on the
same topic, finding that the meta-analyses would have led to the
adoption of an inefficient treatment in 32% of cases, and to the
rejection of a useful treatment in 33% of cases.
When large trials were compared with other large trials on the same
topic, agreement among them was approximately as low as that reported
by LeLorier et al. between meta-analyses and large trials, and thus
Furukawa et al. (2000) concluded that taking large randomized trials as
the ‘gold standard’ can be problematic, and there is "no substitute for
clear and hard thinking for a study, be it meta-analysis or a
megatrial." Horwitz (1987), who analyzed 36 topics on which controlled
trials led to discrepant results, also considered that the ‘gold
standard’ status of randomized trials is misleading as the results
depend substantially on the settings, such as clinical heterogeneity
among patients enrolled in the trials, varying protocols across the
trials, etc. When Poynard et al. (2002) compared the ‘survival of
conclusions’ from meta-analyses with the conclusions from
non-randomized studies and randomized trials, they found that
meta-analysis was conclusively the poorest of the three for producing
viable conclusions. It seems possible that meta-analysis is used on
average less on topics on which large studies have shown clearly
negative or positive findings, and in this respect the poor ‘survival
of conclusions’ may reflect the topics that are studied better than the
method per se.
LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F (1997)
Discrepancies between meta-analyses and subsequent large randomized
controlled trials. N Engl J Med
337:536-42 * comments in: (1997);337:559-61 (1998);338:59-62 and
JAMA (1998);280:518-9
Poynard T, Munteanu M, Ratziu V, et al. (2002) Truth survival in
clinical research: an evidence-based requiem? Ann Intern Med
136:888-95 * comments in: (2002):137:932
