Lec 2 Slides

Randomization and Simple Comparative Experiments
Dr. Zou
Department of Statistics and Biostatistics

CSUEB
Jan 23, 2020
1/30
One more concept
Placebo is a null treatment that is used when the act of applying a

treatment-any treatment-has an effect. E.g., sugar pills to patients in a
double-blind manner.
Usually in medical studies patients that receive Placebo serve as a control

group.
2/30
Dr. Zou (csueb) Randomization and Simple Comparative Experiments Jan 23, 2020 2 / 30
Example: A fascinating landmark study of placebo surgery
Moseley et al. (2002) showed that in this controlled trial involving patients
with osteoarthritis of the knee, the outcomes after arthroscopic lavage or
arthroscopic debridement were no better than those after a placebo
procedure.
Remark: For more details see ”A Controlled Trial of Arthroscopic Surgery

for Osteoarthritis of the Knee” at The New England Journal of Medicine
(uploaded in Blackboard).
3/30
Confounding occurs when the effect of one factor or treatment can not
be distinguished from that of another factor or treatment. The two factors
or treatments are said to be confounded. E.g., the factor word processing
packages (A and B) and the factor ”the order of the document entered”
are confounded in our previous example.
4/30
More on responses
Oftentimes more than one response will be collected from a subject in an

experiment:
1 By addressing several questions, experiments often need a different
response for each question. Responses such as these are often called
primary responses.
2 To address a single question, in some cases we need to collect
multiple responses.
5/30
An example of multiple responses
In order to capture Parkinson’s disease disability, we need responses such

as motor (Yi1 ) and non-motor functions (Yi2 ), cognition (Yi3 ) and drug
complications (Yi4 ). Thus, the response of the ith patient is
 
Yi1
Yi2 
Yi = 
Yi3 

Yi4
Remark: In this class, we only focus on the case of a primary response.
6/30
Surrogate responses are responses that are supposed to be related to
and predictive for the primary response. They are oftentimes shorter to
follow up, easier and cheaper. Example, increase in life expectancy vs. the
fraction of patients still alive after five years.
7/30
Randomization
Randomization is a method for assigning treatments to experimental

units using a known, well-understood probabilistic scheme. We say an
experiment is randomized if a randomization is applied to the experiment.
8/30
Why do we need randomization?
1 Randomization protects against systematic errors (or confounding).
2 Randomization itself can be used to conduct statistical inferences.
9/30
Two treatments comparison example
Suppose that we wish to compare a new drug treatment with a surgery

procedure for a certain disease. The surgery is more invasive. We have 20
patients as volunteers to participate in this experiment.
What happens if we let patients decide which treatment to be used by

themselves?
10/30
Since surgery is a more invasive procedure, patients with better health
conditions will be more willing to take the surgery. Thus the drug therapy
would likely to have a lower effect score due to getting the weaker
patients, even if those two treatments are as effective as each other.
(confounding appears here.)
11/30
Randomization Schemes
Lets see two randomization schemes for this experiment:
1 Toss a coin for every patient; heads-the drug, tails-the surgery;
2 Randomly draw 10 patients to receive the drug therapy; The rest 10

patients receive the surgery.
What is the difference between these two randomization schemes?
Which one is better assuming all other factors are equal?
How many different assignments does each of the two randomizations
have?
We now see how to conduct these randomizations in R.
12/30
How does this work? Randomization against confounding.
There are many potential features of the population of experiment units

are associated with our response, randomizations put approximately half
the patients with these features in each treatment group.
Approximately half men get the drug; whereas the other half get the
surgery.
Approximately half of patients with better health conditions get the
drug.
Approximately half the older patients get the drug.
As a result, randomization will generate more homogeneous groups for
comparisons. Of course, if you want to have a better control over some
features, you can resort to more complicated randomization procedures
such as stratified randomization (Show some details: male-female, ages).
13/30
A quick test
Consider the paired design we saw last time, a company is evaluating two
different word processing packages (A and B) for use by its clerical staff.
The goal is to see how quickly a test document can be entered correctly
using two programs. Suppose that 20 test secretaries need to enter the
same document twice using each program. How will you apply
randomization to this case in order to avoid the previous confounding
factor e.g., order?
14/30
A possible method: We randomly select 10 secretaries to enter the
document twice using each in the order A first and B second; The rest 10
secretaries will enter the document twice using each program in the order
B first and A second. Later, when we perform paired t-test, the order
effect will be averaged out.
15/30
Randomization techiniques are used throughout the
exepriments
Some examples:
If experimental units are not used simultaneously, you can randomize
the order in which they are used.
If you use more than one measuring instrument for determining

response, you can randomize which units are measured on which
instruments.
When we anticipate that one of these might cause a change in the
response, we can often design the corresponding problems into the
experiment (e.g., Ch 13 blocking), and randomize everything else. That
says if you expect some potential factors that may influence the response
in an systematic manner, you should consider them in the experiment
designs.
16/30
Simple Comparative Experiments
We consider experiments to compare two treatments (sometimes called

conditions). These are often called simple comparative experiments.
We also refer the two different treatments as two levels of a factor of
interests.
17/30
An example
An engineer is studying the formulation of a Portland cement mortar. He
has added a polymer latex emulsion during mixing to determine if this
impacts the curing time and tension bond strength of the mortar. The
experimenter prepared 20 experimental samples and randomly assign 10
samples to receive the original formulation and 10 samples to receive the
modified formulation. When the cure process was completed, the
experimenter did find a very large reduction in the cure time for the
modified mortar formulation. Then he began to address the tension bond
strength of the mortar. If the new mortar formulation has an adverse
effect on bond strength, this could impact its usefulness.
Remark: see Chapter 2 of Montgomery’s book 6th edition for more details.
18/30
19/30
The crude average tension bond of the modified mortar is
ȳ1 = 16.76 kgf /cm2 compares with the average tension bond
ȳ2 = 17.04 kgf /cm2 of the unmodified mortar. The average tension bond
strengths in these two samples differ by what seems to be a modest
amount. However, it is not obvious that this difference is large enough to
imply that the two formulations really are different. Perhaps this observed
difference in average strengths is the result of sampling fluctuation and the
two formulations are really identical.
20/30
1. Assumptions
Let y11 , y12 , ..., y1n1 represent the n1 observations from the first treatment
(or the first factor level).
Let y21 , y22 , ..., y2n2 represent the n2 observations from the second
treatment (or the second factor level).
Assumptions:
1 We will assume that these observations are independent with each
other.
2 We will also assume that the observations are normally distributed.
In a word, the samples are drawn at random from two independent normal
populations.
21/30
2. A Model for the Data
yij = µi + ij , i = 1, 2, j = 1, 2, ..., ni .
”Response = Treatment effect + Random error ”
yij is the jth obs from factor level i (or ith treatment).
µi is the mean of the response at the ith factor level.
ij are independent as N(0, σi2 ).
22/30
3. Statistical Hypotheses are derived from research
questions
A statistical hypothesis is a statement either about the parameters of a
probability distribution or the parameters of a model. The hypothesis
reflects some conjecture about the problem situation. For example, in the
Portland cement experiment, we may think that the mean tension bond
strengths of the two mortar formulations are equal. This may be stated
formally as
H0 : µ 1 = µ 2
vs.
H1 : µ1 6= µ2
where µ1 is the mean tension bond strength of the modified mortar and µ2
is the mean tension bond strength of the unmodified mortar.
Remark: In general, we usually set our research hypotheses as alternative

hypotheses.
23/30
3. Hypotheses testing
To test a hypothesis, we devise a procedure for taking a random sample,

computing an appropriate test statistic and its sampling distribution, and
then rejecting or failing to reject the null hypothesis H0 based on the
computed value of the test statistic. Part of this procedure is specifying
the set of values for the test statistic that leads to rejection of H0. This
set of values is called the critical region or rejection region for the test.
Informally, when null hypothesis is correct, we do not expect to see a

surprise e.g., usually an unusually large or small value of Test Statistic.
And that how large or small the value is considered as unusual is decided
by its sampling distribution and the significance level α.
24/30
Two kinds of errors
If the null hypothesis is rejected when it is true, a type I error has

occurred. If the null hypothesis is not rejected when it is false, a type II
error has been made.
α = P(Type I error ) = P(reject H0 |H0 is true)

β = P(Type II error ) = P(fail to reject H0 |H0 is false).
Sometimes it is more convenient to work with the power of the test, where
Power = 1 − β = P(reject H0 |H0 is false).
25/30
The general procedure in hypothesis testing is to specify a value of the
probability of type I error α, often called the significance level of the
test, and then design the test procedure so that the probability of type II
error β has a suitably small value.
26/30
4.1 The pooled Two-Sample t-Test
Suppose that we could assume that the variances of tension bond

strengths were identical for both mortar formulations. Then the
appropriate test statistic to use for comparing two treatment means in the
completely randomized design is
ȳ − ȳ2
t0 = q1 ,
Sp n11 + n12
where Sp2 is an estimate of the common variance σ12 = σ22 = σ 2 computed

from
(n1 − 1)S12 + (n2 − 1)S22
Sp2 = .
n1 + n2 − 2
27/30
To test the null hypothesis that H0 : µ1 = µ2 vs. H0 : µ1 6= µ2 in a
two-sided fashion, we would compare the value of t0 to the t distribution
with n1 + n2 − 2 degrees of freedom.
If |t0 | ≥ tα/2,n1 +n2 −2 where tα/2,n1 +n2 −2 is the upper α/2 percentage
point of the t distribution, then we would reject the null hypothesis
H0 and conclude that the mean strengths of the two formulations of
Portland cement mortar differ.
Show some details related to justification (rationale) of this approach
1: Pivotal quantity; 2: Likelihood ratio test in Stat 640.
(Draw a plot and see this in R)
28/30
4.2 Unpooled Two-Sample t-test
Without assuming equal variances. Then the appropriate test statistic to

use for comparing two treatment means is
ȳ1 − ȳ2 − (µ1 − µ2 )

t0 = q 2 ,
S1 S12
n1 + n2
where the degree of freedom v of t0 is obtained by Welch’ approximation.
The unpooled t-test is a default in t.test in R.
29/30
4.3 Which test should we choose in practice?
Pooled t−test vs. Unpooled t − test?

In experiment designs, when randomization takes place, the pooled
t-test oftentimes work.
For observational studies, you should always use the unpooled t-test
since there are no information about equality of variances.
You should also realize that when variances are different, the
unpooled t-test actually tests whether two distributions are the same
or not. And the inferences following up the test will be quite
complicated. (Draw a plot)
Remark: I hope to convince you of the complexity of even simple tests.
Once you have a clear idea of your research problems, you will be fine.
30/30

Lec 2 Slides

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Lec 2 Slides

Caricato da

Copyright:

Formati disponibili

Randomization and Simple Comparative Experiments

Department of Statistics and Biostatistics

Jan 23, 2020

Placebo is a null treatment that is used when the act of applying a

Usually in medical studies patients that receive Placebo serve as a control

Remark: For more details see ”A Controlled Trial of Arthroscopic Surgery

Oftentimes more than one response will be collected from a subject in an

In order to capture Parkinson’s disease disability, we need responses such

Remark: In this class, we only focus on the case of a primary response.

Randomization is a method for assigning treatments to experimental

1 Randomization protects against systematic errors (or confounding).

2 Randomization itself can be used to conduct statistical inferences.

Suppose that we wish to compare a new drug treatment with a surgery

What happens if we let patients decide which treatment to be used by

Lets see two randomization schemes for this experiment:

1 Toss a coin for every patient; heads-the drug, tails-the surgery;

2 Randomly draw 10 patients to receive the drug therapy; The rest 10

There are many potential features of the population of experiment units

If you use more than one measuring instrument for determining

We consider experiments to compare two treatments (sometimes called

2 We will also assume that the observations are normally distributed.

yij = µi + ij , i = 1, 2, j = 1, 2, ..., ni .

”Response = Treatment effect + Random error ”

Remark: In general, we usually set our research hypotheses as alternative

To test a hypothesis, we devise a procedure for taking a random sample,

Informally, when null hypothesis is correct, we do not expect to see a

If the null hypothesis is rejected when it is true, a type I error has

α = P(Type I error ) = P(reject H0 |H0 is true)

Power = 1 − β = P(reject H0 |H0 is false).

Suppose that we could assume that the variances of tension bond

where Sp2 is an estimate of the common variance σ12 = σ22 = σ 2 computed

Without assuming equal variances. Then the appropriate test statistic to

ȳ1 − ȳ2 − (µ1 − µ2 )

where the degree of freedom v of t0 is obtained by Welch’ approximation.

The unpooled t-test is a default in t.test in R.

Pooled t−test vs. Unpooled t − test?

Potrebbero piacerti anche

yij = µi + ij , i = 1, 2, j = 1, 2, ..., ni .