Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
to
Bayesian
Adap)ve
Methods
Lecture
notes
modied
from
Dr.
Melanie
Quintana,
Fall,
2013
Arm
Dropping
If
a
route
took
WAY
too
long
then
you
may
want
to
drop
it
all
together
from
your
op)ons
and
never
take
it
again
Early
Stopping
How
long
un)l
you
had
enough
data
to
convince
you
of
the
fastest
route?
1.00
0.85
0.90
Control Group
(n=16)
0.80
Eventfree Probability
0.95
Device Group
(n=9)
4
Years to Event
Adap)ve randomiza)on
Analyze
Available Data
Continue Data
Collection
Revise Allocation
and Sampling Rules
per Adaptive Algorithm
Stopping
Rule Met?
Stop Trial or
Begin Next
Phase in
Seamless
Design
Adap)ve
Advantages
Companies
save
resources
and
successful
drugs/devices
get
to
market
faster
Stop
a
trial
early
if
it
is
most
likely
fu)le
and
move
on
to
the
next
promising
drug/device
Stop
a
trial
early
for
success
What
is
Bayesian?
X ~ N ( ,1
Frequen/st
is
an
unknown
constant
If
the
true
mean,
,
were
0,
how
likely
is
2?
0.1
0.2
0.3
0.4
0.0
Pr ( X | = 0 )
What
is
Bayesian?
X ~ N ( ,1
Bayesian
is
a
random
variable!
What
is
the
distribu)on
of
aper
observing
X?
Pr ( | X )
What
is
Bayesian?
do
we
get
there?
How
2
From
X
~
N
,1
to
Pr
(
|
X
)
?
Bayes
Theorem
(the
theorem
of
Inverse
Probability)
is
the
crank
that
gets
from
Pr(A|B)
to
Pr(B|A)
Pr ( A | B ) Pr ( B )
Pr ( B | A ) =
Pr ( A | B ) Pr ( B ) + Pr ( A | BC ) Pr ( BC )
What
is
Bayesian?
Beliefs
+
Data
=
Posterior
Probability
Prior
Posterior
Probability
Prior Probability
Pr ( A | B ) Pr ( B )
Pr ( B | A ) =
Pr ( A | B ) Pr ( B ) + Pr ( A | BC ) Pr ( BC )
Likelihood
(data)
Coin,
Pr(HEADS)
=
=
0.25
or
=0.75,
equally
likely.
DATA:
Flip
coin
twice,
both
heads.
???
2
0.75 ) ( 0.5 )
(
=
( 0.75 )2 ( 0.5 ) + ( 0.25 )2 ( 0.5 )
= 0.90
L ( X | ) ( )
L ( X | ) ( ) d
0.4
0.4
0 1 2 3 4
0.8
0.0
0.4
0.8
After SSF
After SSFS
After SSFSF
0.8
Density
Density
0.4
0.0
0.4
0 1 2 3 4
Pi
0 1 2 3 4
Pi
0.8
0.0
0.4
0.8
Pi
After SSFSFS
After SSFSFSS
After SSFSFSSSFSSSSSSFS
0.4
Pi
0.8
Density
0.0
0.4
Pi
0.8
0 1 2 3 4
Pi
0 1 2 3 4
Pi
Density
0.0
Density
0.0
Pi
0 1 2 3 4
0.0
After SS
0 1 2 3 4
0.8
0 1 2 3 4
Density
0.0
Density
After S
Density
0 1 2 3 4
Density
Prior Beta(1,1)
0.0
0.4
Pi
0.8
Posterior
Probability
Data|p ~ Bin(6,20)
p ~ Beta(1,1)
p|Data ~ ?
!
Beta(a,b) =
!
(a + b) (a1)
x
(1 x)(b1)
(a)(b)
(a) = (a 1)!
pr(p|Data) pr(Data|p)pr(p)
20 6
14
p
(1
p)
6
Beta(7,15)
20! 6
14
!
p (1 p)
6!14!
(21)
p(71)(1 p)(151)
(7)(15)
Posterior Probability
3
0
Density
2
1
0
Density
Posterior Distribution
p|data~Beta(7,15)
Prior Distribution
p~Beta(1,1)
0.0
0.2
0.4
0.6
p
0.8
1.0
0.0
0.2
0.4
0.6
p
0.8
1.0
Posterior
Probability
4
2
1
0
Density
0.0
0.2
1.0
p
Posterior Probability
2
1
0
Density
Posterior 95% CI
0.0
0.15
0.52
p
1.0
Predic)ve
Probability
If
we
have
74
or
more
successes
out
of
300
we
will
win
Phase
3
(alpha
=
.025)
How
likely
is
it
that
we
will
see
74
or
more
successes
given
data
we
have
seen
so
far?
What
is
the
the
distribu)on
of
future
data
given
data
we
have
seen
so
far?
Predic)ve
Probability
Pr(x | data) = pr(x | p) pr( p | data) dp
Beta-Binomial Dist
300 x
(22)
300x
151
71
=
p (1 p )
p (1 p ) dp
( 7 ) (15)
x
Simulate
a
(p
|
data)
from
beta(7,15)
Simulate
an
(x
|
p)
from
bin(300,p)
Distribu)on
of
xs
is
beta-binomial
the
predic)ve
distribu)on
Can
count
the
propor)on
of
these
xs
that
are
greater
than
or
equal
to
74
to
get
the
predic)ve
probability
of
success
in
Phase
3
0.05
Binomial Distribution
p=6/20
0.04
0.03
0.02
Density
0.03
0.75
0.00
0.01
0.02
0.01
0.00
Density
0.04
0.98
50
100
150
200
250
300
50
100
150
200
250
300
0.05
Binomial Distribution
p=12/40
0.04
0.03
0.02
Density
0.03
0.80
0.00
0.01
0.02
0.01
0.00
Density
0.04
0.98
50
100
150
200
250
300
50
100
150
200
250
300
Same
condi)onal
power
(since
we
are
condi)oning
on
same
value
of
p=0.3)
but
predica)ve
power
changes
even
though
the
es)mate
of
the
response
rate
does
not
change!
Predic)ve
power
takes
into
considera)on
how
much
data
we
have
seen
and
how
certain
we
are
of
the
response
rate.
0.05
Binomial Distribution
p=30/100
0.04
0.03
0.02
Density
0.03
0.87
0.00
0.01
0.02
0.01
0.00
Density
0.04
0.98
50
100
150
200
250
300
50
100
150
200
250
300
Synthesis
of
Evidence
Trial
Simula)on
For
complex
adap)ve
designs
you
can
not
calculate
opera)ng
characteris)cs
analy)cally
(Type
I
Error
and
Power)
(integra)on
can
not
be
done
in
closed
form)
Simula)on
the
only
way
to
do
this
0.8
0.6
0.6
0.4
Probability
0.4
Probability
0.2
0.2
0.2
50
100
Number of Subjects
0.4
Pr(Response)
0.6
150
0.8
1.0
*Low
probability
Dose
1
is
max
*Start
with
20
pa)ents
on
each
dose
0.8
Subject Allocation
200
Pr(Response)
Dose
2
Dose
0.0
0.0
0.0
2
Dose
2
Dose
0.8
0.6
Probability
0.6
100
Probability
0.4
0.4
0.4
Number of Subjects
Pr(Response)
0.6
150
0.8
1.0
1.0
0.8
Subject Allocation
200
Pr(Response)
0.2
0.2
2
Dose
2
Dose
0.0
0.0
0.0
50
0.2
2
Dose
2
Dose
0.8
1.0
0.6
0.4
Probability
0.6
Probability
100
0.4
Number of Subjects
0.4
Pr(Response)
0.6
150
0.8
0.8
Subject Allocation
200
Pr(Response)
0.2
0.2
0.2
50
2
Dose
2
Dose
0.0
0.0
0.0
2
Dose
2
Dose
Subject Allocation
200
Pr(Response)
0.8
0.6
0.4
Probability
0.6
Probability
100
0.4
0.4
Number of Subjects
Pr(Response)
0.6
150
0.8
0.8
1.0
0.2
0.2
0.2
50
2
Dose
2
Dose
0.0
0.0
0.0
2
Dose
2
Dose
Dose 1
Dose 2
Dose 3
All Null
.2
.2
.2
All Good
Max
Max
Max
One Good
.2
.2
Max
Increase
.2
1/2Max + .1
Max
All Good
(Max-Max-Max)
One
Good
(.2-.2-Max)
Increase
(.2-.5Max+.1-Max)
Max
Prob.
of
Prob.
of
Mean
Mean
Response
Success
Fu/lity
Subjects
Dura/on
Dose 2
Dose 3
.2
0.04
0.13
280.56
4.68
0.34
0.31
0.35
.3
0.82
0.00
162.12
2.70
0.37
0.32
0.31
.4
1.00
0.00
75.60
1.26
0.32
0.34
0.34
.5
1.00
0.00
61.08
1.02
0.34
0.33
0.33
.3
0.49
0.01
231.30
3.86
0.04
0.06
0.91
.4
0.99
0.00
112.86
1.88
0.01
0.00
0.98
.5
1.00
0.00
76.20
1.27
0.01
0.00
0.99
.3
0.52
0.02
223.44
3.72
0.04
0.00
0.96
.4
0.99
0.00
108.00
1.80
0.01
0.00
0.99
.5
1.00
0.00
76.80
1.28
0.00
0.00
1.00
All Good
(Max-Max-Max)
One
Good
(.2-.2-Max)
Increase
(.2-.5Max+.1-Max)
Max
Prob.
of
Prob.
of
Mean
Mean
Response
Success
Fu/lity
Subjects
Dura/on
Dose 2
Dose 3
.2
0.04
0.13
280.56
4.68
0.34
0.31
0.35
.3
0.82
0.00
162.12
2.70
0.37
0.32
0.31
.4
1.00
0.00
75.60
1.26
0.32
0.34
0.34
.5
1.00
0.00
61.08
1.02
0.34
0.33
0.33
.3
0.49
0.01
231.30
3.86
0.04
0.06
0.91
.4
0.99
0.00
112.86
1.88
0.01
0.00
0.98
.5
1.00
0.00
76.20
1.27
0.01
0.00
0.99
.3
0.52
0.02
223.44
3.72
0.04
0.00
0.96
.4
0.99
0.00
108.00
1.80
0.01
0.00
0.99
.5
1.00
0.00
76.80
1.28
0.00
0.00
1.00
All Good
(Max-Max-Max)
One
Good
(.2-.2-Max)
Increase
(.2-.5Max+.1-Max)
Max
Prob.
of
Prob.
of
Mean
Mean
Response
Success
Fu/lity
Subjects
Dura/on
Dose 2
Dose 3
.2
0.04
0.13
280.56
4.68
0.34
0.31
0.35
.3
0.82
0.00
162.12
2.70
0.37
0.32
0.31
.4
1.00
0.00
75.60
1.26
0.32
0.34
0.34
.5
1.00
0.00
61.08
1.02
0.34
0.33
0.33
.3
0.49
0.01
231.30
3.86
0.04
0.06
0.91
.4
0.99
0.00
112.86
1.88
0.01
0.00
0.98
.5
1.00
0.00
76.20
1.27
0.01
0.00
0.99
.3
0.52
0.02
223.44
3.72
0.04
0.00
0.96
.4
0.99
0.00
108.00
1.80
0.01
0.00
0.99
.5
1.00
0.00
76.80
1.28
0.00
0.00
1.00
All Good
(Max-Max-Max)
One
Good
(.2-.2-Max)
Increase
(.2-.5Max+.1-Max
Max
Prob.
of
Prob.
of
Mean
Response
Success
Fu/lity
Subjects
Dose 1
Dose 2
Dose 3
.2
0.12
0.25
253.20
4.22
0.34
0.32
0.34
.3
0.93
0.00
116.46
1.94
0.33
0.33
0.35
.4
1.00
0.00
63.96
1.07
0.32
0.36
0.32
.5
1.00
0.00
60.12
1.00
0.35
0.31
0.34
.3
0.70
0.02
181.02
3.02
0.08
0.07
0.86
.4
1.00
0.00
93.72
1.56
0.02
0.02
0.96
.5
1.00
0.00
67.92
1.13
0.01
0.01
0.98
.3
0.69
0.03
183.18
3.05
0.09
0.00
0.91
.4
1.00
0.00
93.84
1.56
0.02
0.00
0.98
.5
1.00
0.00
67.56
1.13
0.01
0.00
0.99
Stress
test
the
machine
under
dierent
assumed
truths
(drug
doesnt
work,
drug
is
really
good,
etc)
Sample Trials
Watch
progress
of
a
virtual
trial.
The
real
trial
should
not
be
the
rst
)me
your
design
is
run
Dont
want
surprises
during
the
trial
Great
for
debugging
(even
when
you
think
its
right)
Great
way
to
illustrate
the
adap)ve
process
to
collaborators,
management,
IRBs,
DMCs,
investors,
etc.
Opera)ng Characteris)cs