This text is
based on p 32 of Hemilä
(2006)
These documents have up to date links to documents that are available
via
the net.
Harri Hemilä
Department of Public Health
University of Helsinki,
Helsinki, Finland
harri.hemila@helsinki.fi
The various potential limitations of the experimental data should make
a meta-analyst cautious in drawing conclusions, but sometimes the
conclusions are extraordinarily comprehensive considering the kind of
small trials on which they are based.
A particularly bold general proposal related to drawing conclusions
from meta-analyses made by the Chalmers group was that meta-analyses
should be updated with each new trial so that when, or if, the combined
P-value becomes statistically significant at a chosen level, the
treatment should be considered proven efficacious, and further trials
may be considered even unethical. This approach was called ‘cumulative
meta-analysis’ (Antman et al. 1992; Lau et al. 1992), a concept
considered to be among the most important contributions to medicine by
Thomas Chalmers (Ian Chalmers 1996; Liberati 1996). Using cumulative
meta-analysis, the Chalmers group showed that the effect of
administering intravenous magnesium in acute myocardial infarction
reached P[2-t] < 0.05 in 1989, and P[2-t] < 0.001 in 1990 with a
cumulative OR of 0.44 (95% CI: 0.27 – 0.71), and they concluded that
the evidence for the benefit of magnesium was persuasive (Antman et al.
1992; Lau et al. 1992). However, a large trial with 58,050 patients
carried out thereafter showed that mortality in the first 5 weeks after
myocardial infarction was, paradoxically, slightly higher in the
magnesium group (+6%; 95% CI: 0% to +12%) (ISIS-4 1995; Egger &
Smith 1995). The ‘cumulative meta-analysis’ thus led to a completely
false conclusion.
Nevertheless, it is of interest that, following the reasoning of
‘cumulative meta-analysis’, if Chalmers (1975; pp
36-8 of Hemilä
2006) had restricted his meta-analysis of vitamin C and the common
cold
to double-blind placebo-controlled trials in which ≥2 g/day of vitamin
C was regularly administered to participants, he might have found
powerful evidence by 1975 from 5 trials that vitamin C alleviates the
symptoms and/or reduces the duration of colds during supplementation (P
= 0.000,002; Hemilä 1996a).
Using the approach of Chalmers’
‘cumulative meta-analysis,’ the addition of 3 trials that were carried
out after 1975 led to P = 0.000,000,02 by 1996 (Hemilä 1996a).
No
trials with this selection criterion have been published since 1996.
References
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992) A
comparison of results of meta-analyses of randomized control trials and
recommendations of clinical experts. JAMA 268:240-8
Egger M, Smith GD (1995) Misleading meta-analysis: lessons from “an
effective, safe, simple” intervention that wasn’t. BMJ
310:752-4
Hemilä H (1996a) Vitamin C supplementation and common cold
symptoms: problems with inaccurate reviews. Nutrition
12:804-9HH
1996a
ISIS-4 [Fourth International Study of Infarct Survival Collaborative
Group] (1995) A randomised factorial trial assessing early oral
capropril, oral mononitrate, intravenous magnesium sulphate in 58 050
patients with suspected acute myocardial infarction. Lancet
345:669-85
Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers
TC (1992) Cumulative meta-analysis of therapeutic trials for myocardial
infarction. N Engl J Med 327:248-54 * comments in: (1992);327:273-4,
1316