The relevance of the biology of the topic under consideration
by Harri Hemilä
This text is
based on p 33 of Hemilä
(2006)
These documents have up to date links to documents that are available
via
the net.
Harri Hemilä
Department of Public Health
University of Helsinki,
Helsinki, Finland
harri.hemila@helsinki.fi
"By means centering, I refer to the
tendency to consider that the essence of science lies in its procedures
rather than in its problems, questions, or
goals … If means-centering philosophies were
extreme, and if they were quite consistent, there would be no way to
distinguish between an important meta-analysis and an unimportant one. There could be only technically
well-prosecuted meta-analyses and technically poorly prosecuted
meta-analyses. Using only methodological criteria,
the most trivial research could demand as much respect as the most
fruitful one."
‘Meta-analysis’ as a tool is quite easy to understand and apparently
powerful, and this has led some people to carry out meta-analyses
without making themselves familiar with the actual biology of the
topic. Yusuf (1997) was worried that "Sometimes individuals with only
limited knowledge of the pharmacologic aspects of treatment, the
biology of disease, or clinical circumstances that relate to the
specific question may perform meta-analyses leading to an analysis with
little clinical relevance." Bailar (1995) commented that "A good
meta-analysis requires at least as much as to do a good original
article. To proceed without this level of understanding is likely to
lead to serious difficulty. The ‘job shop’ that turns its skills to the
formulaic meta-analysis of a sequence of unrelated topics is asking for
trouble – and may often get it. Meta-analysis must not be routinized."
A good example of this procedure as a mechanical endeavor detached from
biology is carrying out meta-analyses on homeopathy, which is not a
reasonable scientific topic for obvious reasons (Vandenbroucke 1997,
1998a; Vandenbroucke & de Craen 2001). A meta-analysis of
homeopathic trials by Kleijnen et al. (1991) did not refer to any
laboratory studies, although a series of papers published in Nature
only a few years before would have provided relevant background in the
form of laboratory experiments (Metzger & Dreskin 1988). In their
meta-analysis, Kleijnen et al. (1991) commented that one homeopathic
"trial of very high quality was … initiated by the French Ministry of
Social Affairs and performed by a group consisting of regular and
homeopathic researchers. After the earlier publication of several
trials in which homeopathy was shown to decrease the time of recovery
of bowel movements after abdominal surgery this hypothesis was retested
in a rigorous trial… No differences at all were found" between the
treatment groups (Mayaux et al. 1988). This is substantially similar to
the ‘positive’ findings in the laboratory study on homeopathy, which
could not be repeated in four other laboratories (Metzger & Dreskin
1988; Seagrave 1988; Bonini et al. 1988; Hirst et al. 1993), and not
even in the original laboratory when the editor of Nature came to visit
(Maddox et al. 1988).
Nevertheless, the number of ‘good quality’ homeopathy trials with
‘positive results’ was so large that Kleijnen et al. (1991) concluded
that "The evidence presented in this review would probably be
sufficient for establishing homeopathy as a regular treatment for
certain indications" without specifying what these were. In this case,
the thoughtful narrative review on this topic by Vandenroucke & de
Craen (2001) is substantially more useful than the meta-analytic list
of homeopathic trials with their ‘quality scores’ by Kleijnen et al.
(1991).
In another meta-analysis, Kleijnen et al. (1989; pp 38-41) focused on
vitamin C and the common cold. This meta-analysis contains no reference
to any papers related to the immune system studies (some 100 listed by
Hemilä 1997a) or to any animal studies (see Table 2 and Appendices
2 and 3 of Hemilä 2006) that would provide important background
for considering whether this is a reasonable biological issue rather
than a topic for mechanical ‘job shop’ pooling. Neither did Chalmers
(1975; pp 36-8 of Hemilä 2006) refer to any immune system studies
or animal studies in his meta-analysis of vitamin C and the common
cold; however, he did mention the Hume and Weyers study (1973)
reporting that vitamin C in leukocytes drops sharply on the first day
of the common cold (Fig. 1), and such a dramatic change in vitamin C
metabolism does provide one way to rationalize the question of whether
large doses of vitamin C might have therapeutic effects on colds.
Pauling (1971a,b; pp 35-6 of Hemilä 2006) did not supply any
direct reference to biological studies in his meta-analyses, merely
referring to a book which discussed the biological background (Pauling
1970a).
Hirst SJ, Hayes NA, Burridge J, et al. (1993) Human basophil
degranulation is not triggered by very dilute antiserum against human
IgE. Nature 366:525-7
Hume R, Weyers E (1973) Changes in leucocyte ascorbic acid during the
common cold. Scott
Med J 18:3-7 * editorial: Pauling
(1973)
Maslow AH (1954) Motivation and Personality, 3rd edn. Reprinted (1987)
by NY: HarperCollins. pp
188-93
Mayaoux MJ, Guihard-Moscato ML, Schwartz D, et al. (1988) Controlled
clinical trial of homeopathy in postoperative ileus [letter]. Lancet
331:528-9
Metzger H, Dreskin SC (1988) Only the smile is left. Nature 334:375-6 *
see also: Maddox et al. (1988) ; Nature (1988);334:285-6,
(1988);335:109, 200, 292, 392, 584, 664 ]
Pauling L (1970a) Vitamin C and the
Common Cold. San Francisco: Freeman
* See
Book
Reviews * Pauling's comments on book reviews:
Pauling (1971c,
1971d)
Vandenbroucke JP (1998a) Medical journals and the shaping of medical
knowledge. Lancet
352:2001-6 * correction: (1999);353:848 ]
Vandenbroucke JP, de Craen AJM (2001) Alternative medicine: a mirror
image for scientific reasoning in conventional medicine. Ann Intern Med
135:507-13
Yusuf S (1997) Meta-analysis of randomized trials: looking back and
looking ahead. Cont Clin
Trials 18:594-601
