Why do people use LOCF? Or why not?

Naitee Ting, Allison Brailey Pfizer Global R&D CT Chapter Mini Conference
1

Outline

Last Observation Carried Forward (LOCF) Data set description Modeling approaches Concerns in clinical Trials SAP concerns Why or why not use LOCF
2

Observed data from each patient over time

Patients
0

10

Observed Weeks in Study

Complete Data
13 12 11

VAS Pain

10 9 8 7 6 0 1 2 3 4 5 6
PID 1 PID 2 PID 3 PID 4

Visits
4

Last-Observation-CarriedForward
13 12 11 10 9 8 7 6 0 1 2 3 4 5 6
PID 1 PID 2 PID 3 PID 4

VAS Pain

Visits
5

LOCF

Conservative? Or anti-conservative? Biased point estimate May underestimate variance

75

Mean Change from Baseline

70

65

60

55

50 0 1 2 3 4 5 Time (in weeks) Placebo Treatment 6 7 8 9 10

75

Mean Change from Baseline

70

65

60

55

50 0 1 2 3 4 5 Time (in weeks) Treatment Placebo 6 7 8 9 10

Data set

Simulated - standing diastolic BP Eight week study of test drug vs placebo Clinic visit every 2 weeks Primary endpoint change in standing BP from baseline to week 8 Patients completed the study or dropped out at various time points Missing completely at random
9

Simulated data

ctr 501 501 501 501 501 501 501 501 502 502 502 502 502 502 502 502

pid 1 2 5 6 11 15 16 18 1 4 5 8 9 12 13 16

trt 1 0 0 1 0 0 1 0 0 1 1 0 1 0 0 1

wk0 103.9 105.9 93.8 102.8 109.4 93.9 92.4 99.3 105.8 102.0 110.3 125.6 92.9 123.7 107.7 112.1

wk2 102.0 111.8 98.4 87.4 105.3 81.6 83.6 99.0 102.7 100.3 116.8 121.7 91.4 121.7 121.4 109.6

wk4 103.6 112.5 103.4 72.8 99.2 66.1 71.7 101.9 87.5 101.1 120.6 116.1 82.1 118.3 141.5 103.6

wk6 102.2 115.0 104.5 60.9 96.9 50.5 66.2 102.5 84.9 95.7 132.7 110.0 . 122.0 154.7 103.3

wk8 100.4 117.0 116.7 48.5 89.7 40.3 56.5 103.2 78.8 . 136.8 108.5 . 120.3 168.9 104.2

10

Modeling approaches

Many proposals to deal with dropouts Mixed model approach

Repeated measures Random intercept, random slope

Single imputation Multiple imputation

Imputation model Analysis model

11

ANCOVA on LOCF data

Source TREATMENT CENTER BASELINE ERROR Statistic Raw Mean Adj Mean Std Error N

| df | 1 | 8 | 1 |119

MS 2441.0 765.8 318.4 591.1

F 4.13 1.30 0.54

p-Value 0.0444 0.2523 0.4644

Test Drug Placebo -9.40 -0.54 -8.93 -0.26 3.08 3.01 65 65

12

Analysis of completed cases

Source TREATMENT CENTER BASELINE ERROR Statistic Raw Mean Adj Mean Std Error N

| df | 1 | 8 | 1 |109

MS 1963.6 1007.2 73.2 592.0

F 3.32 1.70 0.12

p-Value 0.0713 0.1060 0.7258

Test Drug Placebo -10.40 -1.72 -10.23 -2.11 3.27 3.14 60 60

13

Naive interpretation

If LOCF provides statistical significance If completer analysis supports LOCF True story may lie between the two Clinical conclusion can be made

14

Mixed model analysis

For demonstration purposes, only repeated measure results are presented

proc mixed method=reml ; where week>0 ; class pid trt week ctr ; model y=wk0 trt ctr week trt*week/solution ; repeated week / type=cs subject=pid r rcorr ; estimate 'trt dif at week 8' trt -1 1 trt*week 0 0 0 -1 0 0 0 1 / cl alpha=0.05 ;

15

Results from PROC MIXED

Effect

Num DF 1 1 8 3 3

Den DF 456 16 85 108 46

F Value

Pr > F
0.0826 0.0313 <.0001 0.0662 0.3132 t Value 2.45 Pr > |t| 0.0183

Baseline Treatment Center Week Trt*week

Label week 8 dif

3.03 5.57 5.43 2.46 1.22 Standard Estimate Error DF 7.3739 3.0127 46

16

Single or multiple imputation

Mixed model can be considered as single imputation For imputation, we can use the same model for imputation and analysis However, one model can be used for imputation, but a different one is for analysis
17

Should LOCF be used?

After the modeling approaches became available, use of LOCF have been discouraged Models are developed with assumptions More complicated models require more assumptions Are these assumptions justified?
18

Should LOCF be used?

LOCF is a model and there are simple assumptions behind it In New Drug Applications (NDA), LOCF is still widely used Why?

19

Different phases in clinical trials

Phase Phase Phase Phase Phase

I, II, III, IV I How often? II How much? III Confirm IV Post-Market

20

DOES THE DRUG WORK?

Double-blind, placebo controlled, randomized clinical trial Test hypothesis - does the drug work? Null hypothesis (H0) - no difference between test drug and placebo Alternative hypothesis (Ha) - there is a difference
21

TYPES OF ERRORS

Regulatory agencies focus on the control of Type I error Probability of making a Type I error is not greater than a In general, a = 0.05; i.e., 1 in 20 Avoid inflation of this error Changing the method of analysis to fit data will inflate a
22

MULTIPLE COMPARISONS

For 20 independent variables (clinical endpoints), one significant at random For 20 independent treatment comparisons, one significant at random Subgroup analyses can also potentially inflate a Multiple comparison adjustment
23

Report all data

Scientific experiments generate data Outliers may be observed Delete outlier? Clinical trials generate data A wonder drug cures 9,999 patients of 10,000 One died outlier delete?
24

Statistical Analysis Plan (SAP)

Pre-specification of analysis Prior to breaking blind Internal agreement within project team Binding document to communicate with regulatory authorities Use of LOCF or modeling approach need to be pre-specified in SAP
25

Modeling approaches

Assumptions Can be complicated Difficult to explain to end users George Box All models are wrong, some are useful

26

Why LOCF? Or why not?

Easy to understand Easy to communicate between statisticians and clinicians, and between sponsor and regulators Lots of prior examples Biased point estimate, biased variance

27

Recommendations

Understand the disease Understand data to be collected Understand the dropout issues Make use of Phase II results Encourage use of statistical models LOCF may still be considered as supportive
28

29