Journal of Evaluation in Clinical Practice ISSN 1356-1294

Causality, mathematical models and statistical association:

dismantling evidence-based medicine jep_1383 267..275

R. Paul Thompson BA MA PhD

Professor, Institute for the History and Philosophy of Science, and Department of Ecology and Evolutionary Biology, University of Toronto,
Toronto, Ontario, Canada

Keywords Abstract
causality, evidence-based medicine,
mathematical model, randomized controlled From humble beginnings, largely at the medical school at McMaster University, Canada,
trial, scientific theory the evidence-based medicine (EBM) movement has enjoyed a spectacular rise in interna-
tional acceptance over the last 25 years. Randomized controlled trials (RCTs) and system-
Correspondence atic reviews based on them have pride of place (the gold standard) in EBM’s hierarchy of
Professor R. Paul Thompson evidence; models and theories are relegated to the bottom of the hierarchy. In the last
Institute for the History and Philosophy of decade, RCTs have been extensively criticized. I briefly rehearse those criticisms because
Science and Technology they are an important backdrop to the criticism of EBM developed in this paper. In essence,
Victoria College the argument developed here is that RCTs use mathematics solely as a tool of analysis
University of Toronto rather than as the language of the science and that this fundamentally affects the validity of
91 Charles Street West
causal claims. As EBM gives pride of place to RCTs and devalues theoretical models – a
Toronto, ON
devaluation that would be incomprehensible to a physicist or biologist – the validity of
Canada M5S 1K7
EBM’s causal claims and knowledge claims are weak and far from a ‘gold standard’.
E-mail: p.thompson@utoronto.ca

Accepted for publication: 8 January 2010

doi:10.1111/j.1365-2753.2010.01383.x

upon it, statistics). Section V examines aspects of probability and its

I. Introduction
Sections II through VI of this paper build a foundation for the
central thesis, expounded in section VII, that the construction of
scientific theories is essential to scientific knowledge (i.e. under-
standing the properties of things within a domain of science and
their dynamical behaviour). The construction of scientific theories
requires that mathematics be employed as the language of science
and not merely as a tool of analysis. Evidence-based medicine
(EBM), resting as it does on randomized controlled trials (RCTs)
as the gold standard of evidence, places scant, if any, emphasis on
constructing theories and uses mathematics as a mere tool of data
analysis and experimental design.
In section II, I highlight that EBM is a slogan and, like all slogans,
it needs to be unpacked and analysed. In section III, I do some of that
unpacking and analysis. I do this in reverse order beginning with the
concept of ‘medicine’, then examining the concept ‘based’, and
finally moving to ‘evidence’, the concept on which this paper
focuses. As RCTs are foundational for EBM and an important focus
of my criticism of EBM, section IV provides a brief overview of
RCTs to ensure that a reader and I have the same essential under-
standing. At the heart of RCTs is the probability calculus (and, built
interpretations that are relevant to my thesis. Specifically, it distin-
guishes mathematical calculi (systems) from their interpretations.
This is crucial in the case of probability because there are at least
four much debated interpretations. RCTs rely on a contentious
interpretation, the frequency interpretation. The reliance of RCTs
on the frequency interpretation lays bare that the mathematics is
being used merely as a tool of analysis. In section VI, I illustrate the
difference between using mathematics as the language of a science
and using it as a mere tool of analysis. I do this by looking at
Mendel’s work on hybridization – work that formed the foundation
for modern population genetics. I show that Mendel began his work
by using mathematics as a tool of analysis but made a crucial
transition to using mathematic as the language in which to describe
entities and their behaviours. It was this transition that resulted in a
powerful and robust account of the dynamics of hybridization; a
dynamics that provided a rich and deep understanding of the
phenomena being observed. In section VII, I draw the thread of this
journey together by focusing on the nature and role of theories in
science and the ways in which RCTs, and, by association EBM,
provide a non-theoretical and impoverished account of the phenom-
ena it studies. I conclude, with some suggestion for the kinds of

© 2010 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 16 (2010) 267–275 267
Causality, models and statistical assoc
research directions that might shed light on why, despite compelling
and unanswered criticisms, RCTs are still regarded by regulators
and advocates of EBM as the gold standard of evidence.
II. The power and the poverty
of slogans
A good slogan is a rhetorically powerful tool. In some of the most
successful cases, the slogan derives its rhetorical power from the
apparent unreasonableness of accepting its contradiction. Two
recent examples are ‘smart growth’ related to the increasing size of
cities, or ‘sustainable development’ related to a range of economic
issues – poverty reduction, increases in agricultural yield and the
like. Who is likely to object to housing and feeding people and to
relieving poverty in developing countries? And, who would be
cavalier enough to suggest that growth should be stupid rather than
smart or development should be unsustainable rather than sustain-
able? EBM is yet another example of a rhetorically powerful
slogan. Who in their right mind would suggest that medicine
should not be based on evidence? As with all such slogans,
however, the substantial issues lie in the meanings of the terms
used. What exactly, for example, does ‘smart’ or ‘sustainable’
mean in the specific context.
III. Evidence-based medicine:
the concepts
Each of the three terms in ‘EBM’ is complex and problematic.
Consider first the term ‘medicine’; the domain of medicine is vast,
encompassing, for example, research domains such as molecular
genetics, immunology, physiology and endocrinology and such
clinical domains as family practice, dermatology and paediatrics.
The methodology, experimental techniques, uses of models, etc. in
medical molecular genetics are similar – some might argue iden-
tical – to those found in general molecular biology; in these fields
RCTs play a very small and limited role – but more of that later.
Clinical areas of medicine such as family practice are strikingly
different; prophylaxis, diagnosis, therapy and prognosis rely
heavily on clinical experience, skill in differential diagnosis, and
RCTs. Given these differences, what ‘medicine’ is being referred
to in ‘evidence-based medicine’?
Clinical medicine is clearly identified as the target of EBM by
the movements founders, which leaves a significant and funda-
mental portion of medicine out of the EBM domain; one, of
course, would expect those other areas to involve evidence but,
for the doctrine of EBM, they fall outside its purview – as will
be clear when we turn to what EBM means by ‘evidence’.
Things are still pretty murky, however, because clinical medicine
itself has many faces and fuzzy boundaries. Family practice
seems a reasonable icon of clinical medicine but what about oph-
thalmology that has a clear clinical component but also involves
surgical techniques and contributes to, and draws on, research in
optical theory and cell biology? Areas of clinical medicine such
as ophthalmology and internal medicine are a mixture of
clinical practice, surgical (mechanical) intervention and research
biology.
Within the ideology of EBM, ‘medicine’ appears restricted to
the patient–doctor interaction, specifically the domains of diagno-
sis and therapeutic intervention, and even more specifically
268
R. P. Thompson
therapeutic interventions involving pharmaceuticals, lifestyle
modifications and the like. With these clarifications, the scope
of EBM within the larger domain of medicine has shrunk
considerably.
The second term in EBM, ‘based’, is also not without concep-
tual problems. Many areas of clinical medicine – such as ophthal-
mology, gynaecology and internal medicine – are an inextricable
mixture of clinical practice and research in human biology. In
these domains, talk of one being ‘based’ on the other is conceptu-
ally puzzling. If evidence and clinical practice are inextricably
interconnected, one cannot, in any non-artificial sense, be based on
the other. Hence, either EBM employs a muddled and puzzling
concept of ‘based’, or the scope of EBM has to be narrowed even
further.
The most significant issues with EBM, however, are not with the
B and M but the E; it is that concept that occupies the rest of this
paper. The superficial meaning of ‘evidence’ in EBM is clear.
Below the surface, however, lurk problems of clarity, and problems
with its metaphysical, epistemological, logical and mathematical
assumptions. EBM has an explicit ‘Hierarchy of Evidence’ Pride
of place in this scheme is given to RCTs. In a recent compendium
on EBM [1], evidence is organized into four primary levels (A–D);
within the primary levels A and B are sublevels (e.g. A: level 1b).
RCTs are ranked as A: level 1b, and systematic reviews (SR) –
with homogeneity – are ranked as A: level 1a – the very top. At the
bottom of the levels of evidence (D) is, ‘expert opinion without
explicit critical appraisal, or based on physiology, bench research
or “first principles” ’ [emphasis added] [1]. As systematic reviews
supervene on individual RCTs, individual RCTs are the ‘gold
standard’ of evidence. This stands in stark contrast to sciences such
as physics, chemistry and biology.
The contention of this paper is that the foundation of the EBM
edifice has significant engineering flaws; the entire epistemologi-
cal, logical and mathematical foundations have cracks and fissures
and it has been poured, not on the bedrock claimed, but on sand.
Although what I identify as a crack and fissure is different from,
but additive to, other critics, we all share the view that the engi-
neering flaws are profound and troubling. What is remarkable
about EBM is its continuing lure in the face of relentless and
compelling criticism.
A large and growing body of literature has identified flaws and
weaknesses with EBM and particularly with its enshrining of
RCTs as the gold standard of evidence in medicine (medicine, of
course, as narrowly circumscribed above). For example, Salsburg
has argued that the current application of probability is mathemati-
cally and philosophically flawed [2]. Kravitz et al. have argued
that the heterogeneity of treatment effects is profound and not
incorporated into the interpretation of data or in patient application
[3]. Ashcroft has argued that the concept and nature of ‘clinical
effectiveness’ is fundamentally unclear [4]. Schaffner has argued
that the concept of ‘causality’ in RCTs is unclear [5]. Howson and
Urbach have exposed numerous problems with the frequentist
interpretation of probability, the lack of clarity about randomness
and the lack of justification for the requirement of randomization
in RCTs – as also has Bennett [6,7]. All these critics offer rem-
edies; surprisingly neither the criticisms nor the remedies seem to
have had any impact on the doctrines of EBM. In the first decade
of the 21st century, the criticisms have continued to mount (again
without discernable impact on EBM); Upshur, Worrall, Bluhm,
© 2010 Blackwell Publishing Ltd
R. P. Thompson
Borgerson and Cartwright, to mention a few key people, have
provided critical appraisals [8–13].
IV. Randomized controlled trials
The mantra of RCTs owes a great deal to Ronald A. Fisher who
held that randomization, control and repeatability was a jointly
sufficient condition for the assertion of a causal connection. If a
target population was randomized into two groups, one of which
received an intervention while the other did not (the control group)
and the experimental circumstances were carefully managed and
if, when the experimental circumstances were repeated, the same
outcome occurred, then one could assert absolutely that the inter-
vention caused the specific outcome. According to this view, ran-
domization ensures that any potentially confounding factors
(factors, other than the intervention itself, that might affect the
outcome) are equally distributed in the two groups. Hence, the
assumption is that the only difference between the two groups is
the intervention. The control group serves as the required com-
parator – the outcome in the absence of the intervention (the
putative cause). Repetition of the trial increases confidence that
there are no distorting artefacts of the design, conduct, data col-
lection and analysis of the trial. It also increases confidence in the
assumption of a structural identity of the intervention and control
groups. Contemporary RCTs add ‘blinding’ to the mix. Most com-
monly RCTs are ‘double blind’, which means neither those con-
ducting the experiment nor those in the two groups (commonly
referred to as ‘arms’ of the study) know which group is receiving
the intervention. In some cases, again without the knowledge of
those conducting the experiment or those in the two groups, the
groups are switched partway through the trial: control becomes
intervention, intervention becomes control. There are other vari-
ants as well but this will suffice as capturing the essential and most
common features of RCTs. The assumed goal of all this is to
uncover causes; however, it does this in name only, as I expose
later. But, an equally important feature of the structure of RCTs is
to allow the application of mathematical analysis (especially prob-
ability and statistics). As a result, this has all the appearance of
scientific rigour. A closer examination, however, will uncover
some reasons for scepticism.
V. Probability: axiomatizations
and interpretations
The now most widely accepted mathematical axiomatization of
probability theory is Kolmogorov’s 1933 set-theoretic axiomati-
zation [14]. To the extent that mathematicians almost universally
accept his axiomatization, the mathematical theory of probability
is a settled matter. Things are considerably less settled when one
moves to the interpretation of probability. There are two broad
interpretations: an objective interpretation and a subjective inter-
pretation. Each broad interpretation admits of refined interpreta-
tions. Before exploring these, however, it is worth pausing to
consider what it means for these to be ‘interpretations’ and why
this matters to RCTs and therefore to EBM.
The essential distinction between an axiomatization and its
interpretation rests on specific understandings of the meaning, and
hence application, of the system. For example, game theory can be
set out in purely mathematical symbolic form. Abstract axioms can
© 2010 Blackwell Publishing Ltd
Causality, models and statistical assoc
x
x + y < 180°
y
Figure 1 Euclid’s formulation of the parallel postulate.
be provided and theorems deduced from them. When interpreted in
an economic context, some of the symbolic elements will be inter-
preted as interacting economic agents in a market structure. In an
ecological context, some of the symbolic elements will be inter-
preted as predators or prey interacting in an ecosystem. In a social
psychology context, some of the symbolic elements will be inter-
preted as interacting members of a social group exhibiting
co-operation or deception and so on. The abstract axiomatic
system is the same; the interpretation depends on the specific
domain of application.
An instructive example of the nature and importance of the
distinction between a mathematical theory and its interpretation
can be drawn from geometry. Euclid wrote his Elements around
300 bce. In it (13 books in all), he set out the postulates of
geometry [15]. Although a landmark work during his time and for
centuries after, by modern standards, his explicit postulates are
deemed inadequate. As Boyer points out, Euclid often introduced
other postulates into his proofs [16]. Later formulations of geom-
etry provided more rigorous postulates (axioms). One of Euclid’s
postulates is: if a straight line falling on two straight lines makes
the interior angles on the same side less than two right angles, the
two straight lines, if produced indefinitely, meet on that side on
which the angles are less than two right angles (see Fig. 1). This
has become known as the parallel postulate.
A more familiar formulation (a variant on John Playfair’s 1795
formulation) is: given a line and point not lying on that line, one
and only one new line can be drawn through the given point such
that the two lines never cross when extended infinitely in both
directions (i.e. they are parallel) [17]. Proof of this postulate fas-
cinated and eluded generations of mathematicians from Euclid to
the 19th century. During the 1820s, Nikolai Lobachevski and
Janos Bolyai independently developed a new ‘non-Euclidean’
geometry by assuming one of the contradictions of the parallel
postulate [18,19]. They assumed that more than one line passing
through the given point can be parallel to the given line. Subse-
quently, Eugenio Beltrami proved that this non-Euclidean geom-
etry was consistent if Euclid’s was consistent [20]. Bernhard
Riemann, in a lecture in 1854 (later published), indicated that
various non-Euclidean geometries were possible [21]. The one that
today bears his name assumes another contradiction of the postu-
late, namely that no lines can be drawn through the given point that
is parallel to the given line.
Hence, by the latter part of the 19th century, there were a number
of geometries all shown to be internally consistent if Euclid’s is
consistent. The importance of this for the focus of this paper lies in
its laying bare the need to separate a mathematical calculus from its
physical interpretation or application. The standard interpretations
of these geometries are given using geodesic models. For example,
269
Causality, models and statistical assoc
one model for Euclidean geometry is the geometry of a plane (a flat
sheet of paper for example); one model for Riemannian geometry
(no lines can be drawn parallel to a given line through a point outside
that line) is the geometry of the surface of a sphere; hyperbolic
geometry (there are many lines that can be drawn parallel to a given
line through a point outside that line) can be modelled as the
geometry of the surface of a hyperbole.
This array of geometries and their interpretations has yet
another dimension. Even after it was shown that consistent non-
Euclidean geometries were mathematically valid, and geodesic
models for each were articulated, the actual nature of physical
space was still deemed to conform to Euclidean geometry. But in
1912–1914, all that changed; space in Einsteinian general relativ-
ity is deemed to be Riemannian (non-Euclidean). This is captured
well by Hilbert whose development of geometry:
. . . emphasized that the undefined terms in geometry should
not be assumed to have any properties beyond those indicated
in the axioms. The intuitive-empirical level of the older geo-
metric views must be disregarded, and points, lines, and
planes are to be understood merely as elements of certain
given sets. . . . Similarly, the undefined relations are to be
treated as abstractions indicating nothing more than a corre-
spondence or mapping. ([16], p. 609)
This brief discourse on geometry is intended to explicate the
distinction between (A) a mathematical calculus and its interpre-
tation; and (B) its application, or not, to empirical phenomena. In
order to apply a domain of mathematics to the understanding of
empirical phenomena – such as the nature of space and the move-
ment of things in it – one must provide an interpretation. Some
interpretations are more suited to the task than others. Also, each
interpretation brings with it commitments and assumptions (as
seen, for example, in (1)–(4) in the next paragraph). This is no
small issue in the case of RCTs. The mathematical probability
calculus employed in the analysis of RCT data assumes, mostly
without justification or even user awareness, an interpretation of
the calculus. And, allusion to the fact that probability arose in the
context of games of chance (i.e. in an empirical context) no more
justifies the collapsing of the distinction, than the observation that
geometry arose in the context of understanding or resolving
empirical matters. There is much hard work to be done to justify
any empirical/scientific use of the probability calculus.
Generally, four interpretations of the probability calculus are
recognized; these are set out crisply by Donald Gillies [22]:
1 The Logical theory identifies probability with degree of
rational belief. It is assumed that given the same evidence, all
rational human beings will entertain the same degree of belief
in a hypothesis or prediction (John Maynard Keynes held this
view). [23]
2 The subjective theory identifies probability with the degree
of belief of a particular individual. Here it is no longer
assumed that all rational human beings with the same evi-
dence will have the same degree of belief in a hypothesis or
prediction. Differences of opinion are allowed.
3 The frequency theory defines the probability of an outcome
as the limiting frequency with which that outcome appears in
a long series of similar events.
4 The propensity theory, or at least one of its versions, takes
probability to be a propensity inherent in a set of repeatable
conditions. To say that probability of a particular outcome is p
270
R. P. Thompson
is to claim that the repeatable conditions have a propensity
such that, if they were to be repeated a large number of times,
they would produce a frequency of the outcome close to p.
The RCTs assume, for the most part, the frequency interpreta-
tion. The significance of this for my thesis, as will become clear in
section VII, lies in the fact that the frequency interpretation
assumes that probability is a relationship of events to each other
and not a relationship between theory and evidence. The logical
theory or subjective theory (usually in a Bayesian guise) is more
suited to exploring and justifying the relationship between theory
and evidence. A quick look back to the ways Gilles describes the
different interpretations will reveal that for (1) and (2), the empha-
sis is on ‘evidence’ and the rational ‘degree of belief’ that evidence
warrants. In the case of (3) and (4), the emphasis is on ‘events’ and
‘outcomes’ and their connections. RCTs relate events to each other
so in that respect the adoption of frequency interpretation in the
context of RCTs is defensible. Because RCTs explore the connec-
tions among events (such as treatments and outcomes, lifestyle
choices and health status, and such), considerable reconceptual-
ization of the nature, purposes and methods of RCTs would be
required to make a subjectivist theory such a Bayesian approaches
even remotely appropriate. And, moving RCTs into a framework
where the essential connection was evidence to belief and not
events to events would refocus the purpose and methods to exactly
where I will argue they should be: the relationship between theory
and evidence. That would not be a pleasing outcome for the EBM
hierarchy of evidence because a robust, and therefore explanatory,
theory would become the gold standard, not RCTs.
As indicated, the frequency interpretation is suited to RCTs
because RCTs explore the relationship between events. As a result,
RCTs have been subjected to an extensive body of criticism, a brief
indication of which I note here in passing and without amplifica-
tion of the details of the arguments – those are found in the source
documents cited. Howson and Urbach have argued that the fre-
quency interpretation is deeply flawed and that the Neyman–
Pearson theory of significance tests commonly employed within it
fails to place statistical inductive reasoning on an entirely objective
and rational basis [6]. The statistician David Salsberg is more
forceful:
And so, the Neyman-Pearson formulation lays in rubble at our
feet. It is an arbitrary construction with no apparent relation-
ship to the needs of clinical research. It rests on the rotten
beam of frequentist probability. ([2], pp. 23–24)
Salsburg is no lightweight or ‘ivory tower’ theoretician. He was
honoured with the Distinguished Statistician Lifetime Achieve-
ment Award from the Pharmaceutical Research and Manufacturers
Association for his theoretical and applied contributions to
statistics.
Howson and Urbach also argue that the emphasis on the neces-
sity and value of randomization is unwarranted:
We have argued that randomization does not solve the
problem for which it was designed, and, moreover, there are
good reasons for not regarding it as an absolute precondition
on trials. ([6], p. 154)
Many mathematicians and philosophers have provided argu-
ments along the same lines; RCTs rest on an irreparably flawed
frequency interpretation of probability and that the highly touted
‘randomization’ fails to deliver the promised goods. These are
indeed damaging criticisms but I think there is a more fundamental
© 2010 Blackwell Publishing Ltd
R. P. Thompson Causality, models and statistical assoc

epistemological problem with EBM and RCTs; a problem to which
I now turn.
VI. Mathematics in science: lessons
from Mendel
Mathematics plays at least two different roles in science. One role
is as the language of science. In the same way that English for me
is the language in which I express things, for example, ‘I am
hungry’, so mathematics is the language in which scientific claims
are expressed, for example, V(dX/dt) = I - F1(X) + F2(Y) expresses
the rate of insulin change in the Bolie description of insulin-
glucose regulation [24]. The other is as a tool of analysis. Failure
to appreciate the fundamental epistemological and logical differ-
ences between these two roles underlies the deep flaws in EBM
and its untenable claims regarding RCTs and causality and their
evidential role in science. A straightforward and instructive
example of the distinction in the roles of mathematics can be found
in the work of Gregor Mendel [25]. Hence, I spend some time in
this section, elucidating Mendel’s reasoning and methodology; the
goal is to expose clearly the distinction I am drawing and which I
will be employing in the next section.
Mendel was interested in hybridization in plants (inter-
fertilizing two varieties of a plant) and set out to discover what
happens in subsequent generations of intra-bred hybrids. His
explicit goal was to discover generally applicable laws (see, pp.
8–11). Part of Mendel’s success lay in his clarity on the experi-
mental requirements. One requirement was accurate bookkeeping.
He identified three important bookkeeping aspects: determination
of the number of distinct forms, careful organization of progeny by
generations, determination of statistical (numerical) relations. A
second requirement was to construct an experimental design that
would reveal answers (see, pp. 9–10) and then to carry it out with
precision and care.
His experimental design had six steps. First, plants must be
selected that will yield answers. Hence: they must have differen-
tiating characteristics, the offspring in each generation must be
able to be protected from foreign pollen, they should suffer no
marked reduction of fertility through the generations, they must
remain ‘constant without any exception’ (this means checking that
seeds obtained from ‘seedsmen’ breed true through many genera-
tions). Mendel chose peas of the genus Pisum because they had
these required characteristics.
Second, observation of successive generations must be carefully
undertaken to ensure generational delineation and correct assign-
ment of results. Third, a methodical process must be carefully
followed. Mendel obtained seeds of 34 ‘more or less’ distinct
varieties which he checked to ensure that they bred true. He then
selected 22 for the experiment and focused on seven characters
which differed in form in plants that bred true:
Round vs. wrinkled peas
Yellow vs. orange peas (seen through the transparent seed
coats)
Seed coats white vs. grey, grey-brown, leather brown
Smooth or wrinkled ripe seed pods
Green vs. yellow unripe seed pods
Axial or terminal flowers
Long vs. short stem (he chose 6–7 feet and 3/4–11/2 feet).
Fourth, numerous plants with each character difference need to
be crossed (his term was ‘trial’) to obtain hybrids (F1 generation).
Having done this, he observed that only one characteristic of each
pair of characteristics, a, b (e.g. wrinkled seeds or orange peas)
was present in all the F1 progeny. He then intra-fertilized the
hybrids to obtain the F2 generation. He observed, that for each pair
of characteristics in a, b, they emerged in F2 in the ratio 3:1
(see Table 1).
Fifth, intra-fertilize b, and intra-fertilize a. He observed that b,
when intra-fertilized, bred true to form in all subsequent genera-
tions. Whereas, a, when intra-fertilized, yielded a ratio of 3a : 1b.
Sixth, continue intra-fertilization of the offspring plants. This he
observed led to same results in all subsequent generations (see
Fig. 2).
The crucial point to note is that to this point Mendel has been
using mathematics to tabulate and calculate ratios. Hence, he has
F0 a b cross purebreds
F1 a intra-fertilize a and intra-fertilize b
F2 a b
F3 a b
Continues to breed true
Figure 2 Mendel’s pattern of crosses and results.

Table 1 Mendel’s experiments (trials) in

Outcome Ratio
tabular form ([25], pp. 16–17)
Trial Description 1 2 1 divided by 2

1 Form of seed Smooth = 5474 Wrinkled = 1850 2.958918919

2 Colour of seed Yellow = 6022 Green = 2001 3.009495252
3 Colour of seed coats Grey brown = 705 White = 224 3.147321429
4 Form of pods Smooth = 882 Wrinkled = 299 2.949832776
5 Colour unripe pods Green = 428 Yellow = 152 2.815789474
6 Flower position Axial = 651 Terminal = 207 3.144927536
7 Stem length Long = 785 Short = 277 3.45814978
14 947 5010 2.983433134

© 2010 Blackwell Publishing Ltd 271

Causality, models and statistical assoc
Table 2 Mendel’s model of generational sequence based on the
1:2:1 ratio
Seed outcome Ratios
Generation A Aa A A Aa a plants.
1 1 2 1 1 2 1 Mendel summed up his findings and his theoretical model
2 6 4 6 3 2 3 (dynamical system) as follows:
3 28 8 28 7 2 7 So far as experience goes, we find it in every case confirmed
4 120 16 120 15 2 15
5 496 32 496 31 2 31
n 2n - 1 2n - 1
Each plant is assumed to produce four seeds per generation ([25], p. 22).
Example: generation 2: Seed outcome A.
4 from A.
2 from Aa : 2 plants, 4 seeds from each = 8 seeds in 1:2:1 ratio = 2A :
4 Aa : 2aa.
been using mathematics as an instrument (a tool of analysis);
mathematics – especially statistics – used in this way is, without
question, a powerful instrument. However, Mendel’s genius was in
moving beyond this use of mathematics to using it as the language
in which to describe the dynamics of a system, a step which EBM
– as I argue in the next section – does not take.
Describing the dynamics of a system requires three key ele-
ments. First, the entities of the system must be specified (e.g. for
Mendel, ‘factors’; today ‘alleles’) and their characteristics/
properties (e.g. dominance and recessiveness). For Mendel, these
were postulated and not observed entities. This is the case with
many entities in robust contemporary scientific theories. Second, a
mathematical account of the relationships among the entities must
be provided. This specification of the entities and their properties
and relationships is the specification of an ontology for the system.
Third, a mathematical specification of the behaviour of the system
over time must be provided (e.g. segregation and recombination).
These are regularities or, to be more mathematically precise, trans-
formation functions in the system, and they specify the dynamics
of the system. That is, they specify the ways in which the system
will change (transform) over time. They are often differential
equations (ordinary or partial) but could be any of a number of
other equations such as recursion equations.
Mendel provided all the required elements of a dynamical
system. He postulated, the a-plants in F2 and following are really
composed of two factors (A) and (a) in the ratio : 2Aa : 1a – factors
are the entities in his system. As b-plants from the F1 generation
onward breed true, the ratio of the factors for both intra- breedings
is (using Mendel’s own notation): A + 2Aa + a. Today, this would
be expressed: AA + 2Aa + aa.
Mendel’s groundbreaking contribution was to set out math-
ematically, using the factors as the entities of the system, the
dynamics of his system of heredity (see Table 2).
This, for Mendel, is a purely mathematical model of what one
would expect based on the existence of the entities (factors) with
their specified properties and the dynamics of their behaviour.
Mendel continued his experimentation by examining multiple
characters through the generations. He begins with two characters
(round (A) vs. wrinkled (a) seed and yellow (B) vs. green (b) seed
colour) and then moves to three and then four. The outcomes of
these multiple – character experiments, with some mathematical
manipulations, revealed that each character conforms to the 1:2:1
272
R. P. Thompson
ratio predicted mathematically by the model [although differences
can be drawn (see Thompson [26]), for the purposes of this
paper ‘model’, ‘theory’ and ‘mathematical model’ are used inter-
changeably]. Mendel continued his experimentation using other
that constant progeny can only be formed when the egg cells
and the fertilising pollen are of like character, so that both are
provided with the material for creating quite similar individu-
als, as is the case with normal fertilisation of pure species. We
must therefore regard it as certain that exactly similar factors
must be at work also in the production of the constant forms
in the hybrid. Since the various constant forms are produced
in one plant, or even in one flower of a plant, the conclusion
appears logical that in the ovaries of the hybrids there are
formed as many sorts of eggs cells, and in the anthers as
many sorts of pollen cells, as there are possible combination
forms, and that the egg and pollen cells agree in their internal
composition with those separate forms. (p. 29)
Mendel’s theory was amplified and deepened in the 1920s
beginning with the contribution of the mathematician G. H. Hardy
who mathematically demonstrated that in every generation
after the first the same proportion of alleles is obtained:
p2AA : 2pqAa : q2aa [27]. This is the famous Hardy-Weinberg
equilibrium (Wilhelm Weinberg: 1862–1937: doctor, Stuttgart,
Germany also published it 1908). It is one of the cornerstones of
modern population genetics; it serves as an equilibrium principle
within the dynamical system similar to that played by Newton’s
first law in his dynamical system. In both cases, they state that if
nothing happens nothing will happen. Other principles specify
what kinds of things will cause something to happen.
VII. Theories in science: ontology,
dynamics and evidence
The lesson embedded in the Mendel example is that the strength
and credibility of knowledge claims in a domain of science are
directly proportional to:
1 the robustness of the formulation of a mathematical system (an
ontology and dynamics formulated using the language of an
appropriate mathematical calculus), and
2 the strength of the demonstration that empirical phenomena
within that domain are found to be in accordance with the systems
ontology and dynamics (the system and the phenomena are iso-
morphic).
Let me draw out this lesson by juxtaposing two quotations from
Mendel with one from Sackett.
Mendel wrote:
In point of fact it is possible to demonstrate theoretically that
this hypothesis would fully suffice to account for the develop-
ment of the hybrids in the separate generations, if we might at
the same time assume that the various kinds of egg and pollen
cells were formed in the hybrid on the average in equal
numbers. (p. 29, emphasis added)
Experimentally therefore the theory is confirmed that the pea
hybrids form egg and pollen cells which, in their constitution,
© 2010 Blackwell Publishing Ltd
R. P. Thompson
represent in equal numbers all constant forms which result
from the combination of the characters united in fertilisation.
(p. 34, emphasis added)
Points (1) and (2) were clearly fundamental to Mendel’s method-
ology and reasoning; and, it is worth adding, to the subsequent
edifice of contemporary population genetics. By contrast, Sackett
and company claim that the lowest level of evidence (D) (10th
place out of 10 levels of evidence) – is:
Expert opinion without explicit critical appraisal, or based on
physiology, bench research or ‘first principles’. (p. 175,
emphasis added)
In an attempt to make sense of the starkness of this contrast, one
might speculate that perhaps Sackett and others are not referring to
theoretical knowledge – knowledge derived from well-confirmed
and accepted theories. Perhaps the authors mean ‘physiology’ in
some special sense that does not include, for example, our knowl-
edge of the feedback (self-regulating) mechanisms of various hor-
monal systems; physiology would then be more akin to descriptive
anatomical knowledge. Perhaps they have in mind as ‘bench
research’ something along the lines of discovering, refining and
using PCR. Perhaps ‘first principles’ is an oblique reference to
metaphysical systems or to the appeal to ‘common sense’ alla
Thomas Reid rather than a reference to well-confirmed axioms or
postulates of a theory, such as Newton’s laws of motion and the
law of gravitational attraction [28]. Given statements made else-
where in the book, this does not strike me as a fruitful line of
interpretation; furthermore, if something like this is intended, then
an even more worrying feature of EBM emerges: theoretical
knowledge fails to even make the list of levels of evidence.
Perhaps Sackett and other EBM advocates think that clinical
medicine is unique. That is, when it comes to decisions about
medical treatments, theories (models) play no role or, at best, a
very minor role; in brief, the best evidential bases for clinical
medicine are RCTs. But, the case for clinical medicine being
unique has yet to be made. In what ways does evidence and
decision making in clinical medicine differ from evidence and
decision making in computer and electrical engineering? In both
cases the systems are extremely complex. In both cases, there are
self-regulating sub-systems. In both cases, interventions (modifi-
cations) have a cascade of effects. In both cases, different physical
(physiological) architectures produce dramatically different
responses to the same intervention. Yet, an engineer would never
relegate theories and models to the back burner of evidence with
respect to the outcome of an intervention; and she would not rely
principally, instead, on a RCT trial where different architectures
are randomized. Theories and models are the stuff and substance
of the causal framework for an engineer and a causal framework is
essential in determining ‘why’ a particular outcome occurred as a
result of an intervention. As I set out more fully in what follows,
RCTs, at best, determine ‘that’ a particular outcome occurred.
Theories are a fundamental element of the epistemological
underpinnings of modern science. Among many other things, theo-
ries integrate a large body of otherwise disparate knowledge; they
make possible explanations (answering ‘why?’ questions) and pre-
dictions; they ground counterfactual claims; they allow new
knowledge to be generated by exploring the implications of a
theory’s ontology (the entities it claims to exist and their proper-
ties) and dynamics (the ways a system can changes over time).
Although there are different conceptions of how theories can best
© 2010 Blackwell Publishing Ltd
Causality, models and statistical assoc
be formulated, all agree on the utility and centrality of theories to
a robust science. Furthermore, on all such conceptions, the dis-
tinction is made between mathematics as the language of a domain
of science and mathematics as a tool of analysis.
The account of scientific theories that most sharply delineates
this distinction has been called the semantic view of theories; the
name, however, matters little in this context [26,29]. On this
account of theories, theories are mathematical models (such as
Mendel’s): some are very sophisticated and complex, others quite
simple. The model is formulated using some field of mathematics;
the formulation at a minimum specifies the ontology (the entities it
claims to exist and their properties: variables and parameters) and
dynamics (the ways a system can change over time: mathematical
functions such as recursion equations and differential equations)
of a system. The explicit claim is that empirical phenomena within
the domain for which the model was developed have exactly the
same ontology and dynamics as the model. Justifying this claim is
an extra-theoretical endeavour. Mature sciences such as physics,
chemistry and biology encompass both endeavours: the endeavour
to formulate a mathematical model and the endeavour to demon-
strate the sameness of the model’s ontology and dynamics and that
of the phenomena under study. Formulating a model requires using
mathematics as the language of the system. Demonstrating same-
ness of model and phenomena involves many things including
other theories (models). For example, using a light microscope to
observe chromosomal segregation as a partial demonstration of the
isomorphism of Mendel’s model with phenomena requires the
theory of optics.
Evidence-based medicine seems both to eschew theories and
models and, by giving pride of place to RCTs, employs mathemat-
ics principally as a tool of analysis. The branch of mathematics
employed in RCTs is probability and statistics – largely in the
statistical analysis of data from a trial but also in designing trials
and in meta-analyses (systematic reviews) of trials. This use of
probability and statistics as a tool of analysis contrasts sharply
with their use in statistical mechanics and in population genetics.
In both of these cases, the domain of mathematics employed is
probability and statistics but in both cases the mathematical cal-
culus is used as the language of the system. And, in both cases,
mathematics is used to formulate a system (a state space, an
ontology and a dynamics). In these cases, probability and statistics
play an essential role in the formulation of a causal dynamical
system – in contrast to their role in RCTs. This difference is not
innocuous.
First, it should be noted that many areas of medical research and
medical knowledge involve models in which mathematics is used
as the language in which to describe dynamical systems. These
include physiology (which Sackett et al. explicitly relegate to
lowest level of evidence), immunology, medical genetics, neuro-
sciences and similar fields. Epidemiology and biostatistics
(broadly understood to include RCTs to determine the efficacy of
pharmaceutical interventions as well as lifestyle modifications,
etc.) is the field where RCTs are prominent and models of dynami-
cal systems are rare. RCTs dominate epidemiology and biostatis-
tics but play a vastly smaller, if any, role in other fields of scientific
enquiry and knowledge. Modern physics, astronomy, chemistry
and biology make exceedingly little use of RCTs. But why should
this matter? Perhaps RCTs are well suited to epidemiological
research. Nancy Cartwright cuts to the core on this question.
273
Causality, models and statistical assoc
In an RCT, if we are lucky, we find the average difference in
effect produced by the treatment in the population sampled.
That does not tell us what the overall outcome on this effect
in question would be from introducing the treatment in some
particular way in some uncontrolled situation, even if we con-
sider introducing it only in the very population sampled. For
that we need a causal model. Even less does it tell us about
‘side-effects’ of introducing the treatment, either from the
treatment itself or from our way of implementing it. These too
are crucial in calculating the costs and benefits of a proposed
policy. Or, as Heckman argues, suppose one wants to predict
what portion of the population will experience a given degree
of improvement. RCTs do not deliver that kind of result.
Again we need a causal model. ([13], p. 238)
Causal models are those that use mathematics as a language
with which to formulate a dynamical system. As Cartwright cor-
rectly observes, RCTs fail to give answers to crucially important
questions and EBM’s almost total reliance on RCTs and eschew-
ing of dynamical models means that it is never in a position to
provide answers to these questions. Probability and statistics have
important uses as tools of analysis in many areas of science but it
is always in the context of a dynamical model; the mathematics
can be used, for example, to normalize data for comparison with
the model, to determine goodness of fit between the model’s
prediction and the empirical data. And, in fields like statistical
mechanics and population genetics probability and statistics are
used as the language in which their dynamical systems are speci-
fied. By contrast, probability and statistics in RCTs provide an
analysis of data but without reference to any dynamical causal
model. As such ‘why?’ questions cannot be answered. To answer
the question, ‘why does this pharmaceutical produce this effect in
this individual?’ one needs to employ dynamical models from a
particular combination of fields such as physiology, biochemistry,
endocrinology and immunology.
The distinction I have drawn regarding the uses of mathematics
is important in at least two respects. First, it distinguishes clearly
explanations (scientific answers to ‘why’ questions) that employ
dynamical models and explanations that employ analytical tools.
The former provide an account of the entities involved and the
dynamics of their behaviour. The latter provide, at best in ideal
circumstances, a link between an input and an output. Some might
be willing to assert, again in ideal circumstances, that the input is
a cause and the output is an effect; suffice here to note that there
are a host of dangers in doing so. More importantly, using those
labels should not delude one into thinking that a scientific expla-
nation has been given; just how and why the input and output are
connected is not answered by using mathematics to analyse data.
RCTs do not provide scientific explanations (the how and the
why); dynamical models do.
To illustrate this point, consider iodine and thyroxin produc-
tion. We, of course, have considerable knowledge of the connec-
tion which is why this will serve well as an example; but to make
the point, one has to assume a time when the connection is being
explored. In this early period of investigation, a researcher might
have noticed that an enlargement of the thyroid gland (goitre)
occurs more frequently in areas of the world where access to
dietary iodine is restricted (some regions of central Africa and
central South America, for example). The researcher constructs
an RCT using a population of central South American children in
274
R. P. Thompson
an area with a high incidence of goitres. In one arm, the ran-
domly selected and assigned subjects are given an iodine pill in
a dose approximating that found by chemical analysis to be part
of the diet in the Mediterranean region. The other arm receives a
pill with no active ingredients. The trial is robust and has 10 000
people in each arm and runs for 10 years; it is double blind. The
results indicate a statistically significant decrease in goitres in the
iodine arm (at the 0.01 significance level). What do we now
know? With some confidence we can declare that there is a con-
nection between iodine deficiency and goitres and that iodine
supplements are efficacious in preventing goitres. This is obvi-
ously a therapeutically useful piece of knowledge but it does not
provide any answer to the question, ‘why does an iodine defi-
ciency produce goitres?’
The answer to that question requires the construction of a
dynamical model of a homeostatic endocrine mechanism.
Thyroid-stimulating hormone (TSH) is secreted by the thyro-
trope cells of the anterior pituitary. Hypothalmic thyrotropin-
releasing hormone (TRH) stimulates the pituitary production of
TSH. This stimulates thyroid hormone synthesis and its secre-
tion. Thyroid-secreted hormones inhibit the production of TRH.
This is the feedback (homeostatic) endocrine dynamics. Ingested
iodine is bound to serum proteins, especially albumin. The
thyroid gland extracts the albumin-bound iodine (unbound iodine
is excreted in the urine) from the blood stream. This is part of a
dynamical system involving NIS (Na+/I- symporter). When
access to dietary iodine is restricted the extraction process is
depressed. As iodine is an essential component in thyroxin (T4),
the production of thyroxin (T4) by the thyroid is disrupted. As a
result, TRH inhibition is decreased leading to higher levels of
TRH and, as a consequence, TSH production is increased. TSH
stimulates the thyroid to increase thyroxin production. In the
absence of iodine, however, it cannot do so; its increase in size
is an ineffective attempt to respond to the continual stimulation.
Although the complete dynamics are much more complex, this
suffices to show that one now has an answer to why restricted
access to dietary iodine results in goitres and why providing an
iodine supplement reduces or eliminates the problem. And, the
entire system can be described by a set of differential equations.
This dynamical model explains scientifically the causal structure
of the relationship. That is, it explains the relationship as
opposed to simply demonstrating that the relationship exists.
Hence, contrary to EBM, this is the gold standard of evidence; a
powerful understanding of the dynamical mechanisms, the
availability of which make an RCT otiose. By contrast with this
deep understanding, the RCT – EBM’s gold standard – provides,
as Bluhm and Borgerson note, a crude empiricist description
[11].
The second importance of the distinction I have drawn between
the different uses of mathematics is that the distinction makes clear
that although mathematics is central to scientific research and
knowledge, not all uses of mathematics are equal. Using math-
ematics as a language with which to formulate dynamical systems
allows deep explanations of phenomena. Using mathematics as a
tool of analysis does not. Failure to make this distinction may
make an area of research appear far more rigorous and substantial
in its knowledge claims than is warranted, just because it employs
mathematics in a dazzling and impressive way; a way that none-
theless remains a tool of data analysis.
© 2010 Blackwell Publishing Ltd
R. P. Thompson
VIII. The lure of EBM and RCTs
Just as using mathematics as a tool was an important first step in
Mendel’s development of his model, so RCTs can play, and do
play, an important investigative role in medicine. The central claim
in this paper is that to answer ‘why’ questions, to provide a causal
account of phenomena and to integrate disparate bits of knowledge
into a cohesive whole requires the formulation of a dynamical
system. A formulation of such a dynamical system is, and since at
least Galileo has always been, the real ‘gold standard’ in science.
Given the large body of exceptionally robust and sophisticated
theorizing and model building in medicine (e.g. physiology,
immunology, medical genetics) and the barrage of criticisms of
RCTs – most unanswered, perhaps even unnoticed by advocates
and practitioners – the obvious question is: why are RCTs con-
sidered by regulators and researchers in certain fields to be the
‘gold standard’ of evidence? There is much important work to be
done in answering this question. Some obvious arenas of inves-
tigation can be identified: things such as the political dynamics of
medicine (funding, authority, prestige and the like), the sociology
of decision making and defending of decisions, and the psychol-
ogy and politics of litigation aversion. Also, in an environment of
open intellectual investigation, an investigation of the role of
dogma and ideology ought not to be summarily disallowed.
Vladimir Lenin is purported to have said, ‘A lie told often enough
becomes the truth’; a chilling but seemingly correct observation.
Slogans become mantras and, in turn, become doctrines and ide-
ologies, which ultimately become entrenched in research and
regulatory communities. By the time the flaws and foundation-
of-sand are detected and exposed, the doctrines and ideologies
have been propagated often enough to become unassailable truth;
truth on which rests the vested interests of many. Even if one
believes that such an investigation will not yield fruit, there are
far too many examples in history – even the history of science –
to make such an investigation off limits or to assume in advance
that the thesis is dubious.
References
1. Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W. &
Haynes, R. B. (2000) Evidence-Based Medicine: How to Practice and
Teach EBM, 2nd edn. Edinburgh: Harcourt Publishers Limited (see
especially pp. 173–177).
2. Salsburg, D. (1993) The use of statistical methods in the analysis of
clinical studies. Journal of Clinical Epidemiology, 46, 17–27.
3. Kravitz, R. L., Duan, N. & Braslow, J. (2004) Evidence-based medi-
cine, heterogeneity of treatment effects, and the trouble with averages.
The Milbank Quarterly, 82, 661–687.
4. Ashcroft, R. (2002) What is clinical effectiveness? Studies in History
and Philosophy of Biological and Biomedical Sciences, 33, 219–
233.
5. Schaffner, K. F. (1993) Clinical trials and causation: Bayesian per-
spectives. Statistics in Medicine, 12, 1477–1494.
6. Howson, C. & Urbach, P. (1989) Scientific Reasoning: The Bayesian
Approach. La Salle, IL: Open Court.
© 2010 Blackwell Publishing Ltd
Causality, models and statistical assoc
7. Bennett, D. J. (1998) Randomness. Cambridge, MA: Harvard Univer-
sity Press.
8. Upshur, R. E. G. (2005) Looking for rules in a world of exceptions:
reflections on evidence-based practice. Perspectives, 48, 477–489.
9. Worrall, J. (2002) What evidence in evidence-based medicine? Phi-
losophy of Science, 69, S316–S330.
10. Bluhm, R. (2005) From hierarchy to network: a richer view of evi-
dence for evidence-based medicine. Perspectives, 48, 535–547.
11. Bluhm, R. & Borgerson, K. (forthcoming) Evidence-based medicine.
In Philosophy of Medicine (ed. F. Grifford). Amsterdam: Elsevier.
12. Borgerson, K. (2005) Evidence-based alternative medicine. Perspec-
tives, 48, 502–515.
13. Cartwright, N. (2007) Hunting Causes and Using Them. Cambridge:
Cambridge University Press.
14. Kolmogorov, A. N. (1933) Foundations of the Theory of Probability,
1st English edition, 1950: 2nd English edition, 1956. New York:
Chelsea Pub Co.
15. Euclid of Alexandria (circa 300BCE) Elements (first printed edition
(1482) – a Latin translation by Johannes Campanus).
16. Boyer, C. B. (1991) A History of Mathematics, 2nd edn. New York:
John Wiley & Sons.
17. Playfair, J. (1795) Elements of Geometry; Containing the First Six
Books of Euclid, with Two Books on the Geometry of Solids. To
Which are Added, Elements of Plane and Spherical Trigonometry.
Edinburgh: printed for Bell & Bradfute, and G. G. & J. Robinson,
London.
18. Lobachevski, N. I. (1840) Geometrical Researches on the Theory of
Parallels. English translation appears in Roberto Bonola Non-
Euclidean Geometry, Dover, 1955. Non-Euclidean Geometry: A Criti-
cal and Historical Study of Its Development. New York: Dover, 1955.
19. Bolyai, J. (1832) The science of absolute space. English translation in
Roberto Bonola Non-Euclidean Geometry: A Critical and Historical
Study of Its Development. New York: Dover, 1955.
20. Beltrami, E. (1902) Opere Matematiche, Vol. 1. Milano: Ulrico
Hoepli.
21. Riemann, B. (1868) Über die Hypothesen Welche der Geometrie zu
Grunde liegen (On the hypotheses which underlie geometry) [given
first as a lecture June 10, 1854 from his Habilitationsschrift on the
foundations of geometry].
22. Gilles, D. (2000) Philosophical Theories of Probability. London: Rou-
tledge.
23. Keynes, J. M. (1921) A Treatise on Probability. London: Macmillan
and Co (reprinted 2004 by Dover Publications, Inc., Mineola,
NewYork).
24. Bolie, V. W. (1960) Coefficients of normal blood glucose regulation.
Journal of Applied Physiology, 16, 783–788.
25. Mendel, G. (1865) Versuche über Pfanzenhybriden. Verhandlungen
des Naturforschenden Vereins in Brünn, 4, 3–47 (all references are to
Fisher’s reprinting and modification of Bateson’s English translation,
Bateson, W. (1909) Mendel’s Principles of Heredity. Cambridge:
Cambridge University Press).
26. Thompson, P. (2007) Formalisations of evolutionary biology. In Hand-
book of the Philosophy of Science: Philosophy of Biology (eds M.
Matthen & C. Stephens), pp. 485–523. Amsterdam: Elsevier.
27. Hardy G. H. (1908) Mendelian proportions in a mixed population.
Science, 28, 49–50.
28. Reid, T. (1764) An Inquiry in the Human Mind on the Principles of
Common Sense. Dublin: Alexander Ewing.
29. Thompson, P. (1989) The Structure of Biological Theories. Albany,
NY: State University of New York Press.
275