THERAPY STUDY:

Are the results of the trial valid? (Internal Validity)

What question did the study ask?

1a. R- Was the assignment of patients to treatments r​ andomised​?

What is best? Where do I find the information?

Centralised computer randomisation​ is The M​ ethods​ s​ hould tell you how

ideal and often used in multi-centred
trials. Smaller trials may use an
independent​ person (e.g, the hospital
pharmacy) to “police” the
randomization.
patients were allocated to groups and
whether or not randomisation was
concealed.

This paper: Yes ​•​ No ​•​ Unclear •​

Comment:

1b. R- Were the groups ​similar​ at the start of the trial?

What is best? Where do I find the information?

If the randomisation process worked The R​ esults​ should have a table of

(that is, achieved comparable groups)
the groups should be similar. The more
similar the groups the better it is.
There should be some indication of
whether differences between groups
are statistically significant (ie. p
values).
"Baseline Characteristics" comparing
the randomized groups on a number of
variables that could affect the outcome
(ie. age, risk factors etc). If not, there
may be a description of group similarity
in the first paragraphs of the ​Results
section.

This paper: Yes ​•​ No ​•​ Unclear •​

Comment:

2a. A – Aside from the allocated treatment, were groups treated

equally?

What is best? Where do I find the information?

Apart from the intervention the Look in the ​Methods​ section for the
patients in the different groups should follow-up schedule, and permitted
be treated the same, eg., additional additional treatments, etc and in
treatments or tests. Results​ for actual use.
This paper: Yes ​•​ No ​•​ Unclear •​

Comment:

2b. A – Were all patients who entered the trial accounted for? – and
were they analysed in the groups to which they were randomised?

What is best? Where do I find the information?

Losses to follow-up should be minimal The R

– preferably less than 20%. However,
if few patients have the outcome of
interest, then even small losses to
follow-up can bias the results. Patients
should also be analysed in the groups
to which they were randomised –
‘​intention-to-treat analysis’​.
​ esults​ section should say how
many patients were andomised (eg.,
Baseline Characteristics table) and how
many patients were actually included in
the analysis. You will need to read the
results section to clarify the number
and reason for losses to follow-up.

This paper: Yes ​•​ No ​•​ Unclear •​

Comment:

3. M - Were measures ​objective​ or were the patients and clinicians kept

“​blind​” to which treatment was being received?

What is best? Where do I find the information?

It is ideal if the study is First, look in the M

‘double-blinded’ – that is, both patients
and investigators are unaware of
treatment allocation. If the outcome is
objective​ (eg., death) then blinding is
less critical. If the outcome is
subjective​ (eg., symptoms or function)
then blinding of the outcome assessor
is critical.
​ ethods​ section to
see if there is some mention of masking
of treatments, eg., placebos with the
same appearance or sham therapy.
Second, the ​Methods​ section should
describe how the outcome was
assessed and whether the assessor/s
were aware of the patients' treatment.

This paper: Yes ​•​ No ​•​ Unclear •​

Comment:
What were the results?

1. How large was the treatment effect?

Most often results are presented as dichotomous outcomes (yes or not outcomes
that happen or don't happen) and can include such outcomes as cancer
recurrence, myocardial infarction and death. Consider a study in which 15%
(0.15) of the control group died and 10% (0.10) of the treatment group died
after 2 years of treatment. The results can be expressed in many ways as shown
below.

What is the measure? What does it mean?

Relative Risk (RR)​ = risk of the
outcome in the treatment group /
risk of the outcome in the control
group.
The relative risk tells us ​how many times
more likely​ it is that an event will occur in
the treatment group relative to the control
group. An ​RR o ​ f1
​ ​ means that there is no
difference between the two groups thus,
the treatment had n ​ o effect​. An RR < 1
means that the treatment decreases the risk
of the outcome. An RR > 1 means that the
treatment increased the risk of the
outcome.

Since the RR < 1, the treatment decreases

Absolute Risk Reduction (ARR)
= risk of the outcome in the
control group - risk of the outcome
in the treatment group. This is also
known as the ​absolute risk
difference.
the risk of death.
The absolute risk reduction tells us the
absolute difference in the rates of events
between the two groups and gives an
indication of the baseline risk and treatment
effect. An A
​ RR​ of 0
​ ​ means that there is no
difference between the two groups thus,
the treatment had n ​ o effect​.

The absolute benefit of treatment is a 5%

reduction in the death rate.

Relative Risk Reduction (RRR) The relative risk reduction is the

= absolute risk reduction / risk of
the outcome in the control group.
An alternative way to calculate the
RRR is to subtract the RR from 1
(eg. RRR = 1 - RR)
complement of the RR and is probably the
most commonly reported measure of
treatment effects. It tells us the reduction in
the rate of the outcome in the treatment
group relative to that in the control group.
 The treatment reduced the risk of death by
33% relative to that occurring in the control
group.

Number Needed to Treat The number needed to treat represents the

(NNT)​ = inverse of the ARR and is
calculated as 1 / ARR.
number of patients we need to treat with
the experimental therapy in order to
prevent 1 bad outcome and incorporates
the duration of treatment. Clinical
significance can be determined to some
extent by looking at the NNTs, but also by
weighing the NNTs against any harms or
adverse effects (NNHs) of therapy.

We would need to treat 20 people for 2

years in order to prevent 1 death.

2. How precise was the estimate of the treatment effect?

The true risk of the outcome in the population is not known and the best we can
do is estimate the true risk based on the sample of patients in the trial. This
estimate is called the p ​ oint estimate​. We can gauge how close this estimate is
to the true value by looking at the confidence intervals (CI) for each estimate. If
the confidence interval is fairly narrow then we can be confident that our point
estimate is a precise reflection of the population value. The confidence interval
also provides us with information about the statistical significance of the result. If
the value corresponding to ​no effect​ falls outside the 95% confidence interval
then the result is statistically significant at the 0.05 level. If the confidence
interval includes the value corresponding to n ​ o effect​ then the results are not
statistically significant.

Will the results help me in caring for my patient?

(ExternalValidity/Applicability)
The questions that you should ask before you decide to apply the results of
the study to your patient are:

● Is my patient so different to those in the study that the results cannot

apply?
● Is the treatment feasible in my setting?
● Will the potential benefits of treatment outweigh the potential harms of
treatment for my patient?