1

Confidence Intervals
Q1. Give a reasonably detailed explanation of what Maximum Likelihood is.
A1.Maximum likelihood is a technique used to estimate statistical models like logistic regression
(and any other general linear model (GLM)). The idea behind maximum likelihood is to choose
parameter estimates such that they maximize the probability of observing the data that was in fact
observed (maximizing the likelihood). This is achieved by common methods to maximizing any
mathematical function: 1) writing down a probability expression for the likelihood, 2) taking the
derivative of the likelihood function (or log likelihood as its easier to work with), setting equal to 0,
and solving to ascertain a maximum like in Calculus. Usually this maximum is estimated using
iterative methods however. Maximum likelihood estimates have good properties: they are
asymptotically unbiased, have the smallest variance and are Normally distributed. Because of
these desirable properties, ML has become a popular technique for fitting these regression
models.
Q2. Fit a simple logistic regression model to these data using treatment group as a
predictor for myocardial infarction at 5 years (MI).
a. What is the form of the estimated logistic regression equation?
A. We can use SAS Enterprise to fit a simple logistic regression model for MI with treatment
group by going to Analyze->Regression->Logistic.
Doing so gives the following relevant output:
Model Information
Data Set WORK.SORTTEMPTABLESORTED
Response Variable mi
Number of Response Levels 2
Model binary logit
Optimization Technique Fisher's scoring
Probability modeled is mi=1.
Analysis of Maximum Likelihood Estimates
Parameter DF Estimate Standard
Error
Wald
Chi-Square
Pr > ChiSq
Intercept 1 -0.4382 0.2868 2.3348 0.1265
itrt 1 -0.5268 0.4106 1.6464 0.1995
Thus, the form of the estimated logistic regression model is:
2
, )
, )
Pr 1
ln 0.4382 0.5268*
1 Pr 1
MI
itrt
MI
| =
=
|
|
=
' .
b. What is the estimated odds ratio and 95% CI for the OR for the treatment group effect?
Interpret the odds ratio.
A. From the same output this OR is given by:
Odds Ratio Estimates
Effect Point Estimate 95% Wald
Confidence Limits
Itrt 0.590 0.264 1.320
Thus, the OR for treatment vs. control = 0.590, with a 95% Confidence Interval = (0.264, 1.320).
Therefore, the odds of a MI were 41% lower for patients in the treatment group compared to the
control group.
c. Add gender to the logistic regression model. Is there possible evidence of gender
confounding on the treatment effects for MI? How did adding gender affect the predictive
ability of the model?
A. If were using ORs to quantify effects in a logistic regression model, then we need to compare
the OR of itrt in models with and without ifemale in order to determine if there is confounding due
to gender.
After adding ifemale to the model, the odds ratios are given by:
Odds Ratio Estimates
Effect Point Estimate 95% Wald
Confidence Limits
Itrt 0.664 0.290 1.523
Ifemale 1.825 0.741 4.495
Thus, the OR for itrt went from 0.590 to 0.664, which is a 12.5% change. Thus, we conclude
there is confounding on treatment effects due to gender (and thus we would include gender in the
model).
The Area under the ROC curve (c statistic) without ifemale in the model was 0.565 and with
ifemale was 0.620. So although the predictive ability of the model improved somewhat by adding
gender this was not a dramatic improvement.
Q3. Fit a multiple logistic regression model for response with treatment arm, systolic
blood pressure (SBP) at baseline and gender. Use the centered version of SBP (which is
centered at its mean value and include this centered variable in your model.
a. What is the form of the estimated logistic regression equation?
A. Using SAS Enterprise to fit this model gives the following relevant output:
3
Analysis of Maximum Likelihood Estimates
Parameter DF Estimate Standard
Error
Wald
Chi-Square
Pr > ChiSq
Intercept 1 -0.5959 0.3481 2.9308 0.0869
itrt 1 -0.5354 0.4388 1.4889 0.2224
ifemale 1 0.5220 0.4650 1.2600 0.2617
Csbp 1 -0.0249 0.0203 1.4997 0.2207
Thus, the form of the estimated logistic regression model is:
, )
, )
, )
Pr 1
ln 0.5959 0.5354* 0.5220* 0.0249* 138.0734
1 Pr 1
MI
itrt ifemale SBP
MI
| =
= +
|
|
=
' .
b. What is the estimated odds ratio and 95% CI for the OR for the gender effect (3 points)?
Interpret the odds ratio.
The ORs are given in the following output:
Odds Ratio Estimates
Effect Point Estimate 95% Wald
Confidence Limits
Itrt 0.585 0.248 1.383
Ifemale 1.685 0.677 4.193
Csbp 0.975 0.937 1.015
The adjusted OR for ifemale is 1.685, with a 95% CI = (0.677, 4.193). Thus, the odds of a MI are
68.5% higher for females compared to males, adjusting for treatment group and systolic blood
pressure.
c. Give and interpret the odds ratio for baseline SBP.
A. The odds ratio for SBP centered at its mean is 0.975. Thus, for every unit increase in SBP (1
mmHg) the odds of MI decrease by 2.5%, controlling for treatment group and gender.
However the 95% confidence interval for this adjusted OR includes 1 (and thus stretched below
and above 1). So although the point estimate suggests an inverse association, this association is
not statistically significant (more about this later).
d. Conduct a Likelihood ratio test and a Wald test to see if there are significant treatment
arm effects.
A.The Wald test for treatment arm effects is given directly off the SAS output:
Analysis of Maximum Likelihood Estimates
4
Parameter DF Estimate Standard
Error
Wald
Chi-Square
Pr > ChiSq
Intercept 1 -0.5959 0.3481 2.9308 0.0869
itrt 1 -0.5354 0.4388 1.4889 0.2224
ifemale 1 0.5220 0.4650 1.2600 0.2617
Csbp 1 -0.0249 0.0203 1.4997 0.2207
Here, the Wald Chi-square = 1.4889 with df = 1 and p-value = 0.2224. Thus using the Wald test
we would fail to reject the null hypothesis of no treatment group effect at the 5% significance
level. Here, we conclude there is no treatment effect on the odds of having an MI.
To conduct a Likelihood ratio test we must compare the -2lnL value from the current (full) model
with treatment group:
Model Fit Statistics
Criterion Intercept
Only
Intercept
and
Covariates
AIC 140.293 141.390
SC 142.985 152.155
-2 Log L 138.293 133.390
To a reduced model without treatment group (but with ifemale and Csbp):
Model Fit Statistics
Criterion Intercept
Only
Intercept
and
Covariates
AIC 140.293 140.889
SC 142.985 148.963
-2 Log L 138.293 134.889
Thus the Likelihood ratio test statistic = 134.889 133.390 = 0.125. Since this does not exceed
the critical value of 3.84, we fail to reject the null hypothesis that full model has similar fit as the
reduced model. Thus there is not evidence of a treatment group effect on the odds of having a
MI.
e. How did adding SBP affect the predictive ability of the model? Give a plot of the
corresponding ROC curve which helps summarize this graphically.
A. The Area under the ROC curve without SBP was 0.620 and with SBP was 0.637. So again
the models predictive ability was just slightly improved.
5
SAS Enterprise gives the following ROC curve
plot:
f. Does the treatment effect depend on gender (in a model with itrt, ifemale, Csbp)?
Conduct an investigation using the data in order to address this question and justify your
answer.
A. In order to answer this question, we need to fit a model with itrt, ifemale, Csbp and an
interaction effect between itrt and ifemale (i.e., include INTitrtifemale, which is INTitrtifemale = itrt
* ifemale). If the treatment effect depends upon gender this means there is an interaction
between itrt and ifemale.
Doing this gives the following relevant SAS Enterprise output:
Analysis of Maximum Likelihood Estimates
Parameter DF Estimate Standard
Error
Wald
Chi-Square
Pr > ChiSq
Intercept 1 -0.3333 0.3647 0.8353 0.3607
itrt 1 -1.0270 0.5148 3.9799 0.0460
ifemale 1 -0.1819 0.5996 0.0920 0.7616
Csbp 1 -0.0203 0.0206 0.9677 0.3252
INTitrtifemale 1 1.7870 0.9572 3.4852 0.0619
6
Here, the Wald test for interaction gives a p-value = 0.0619. Thus, there is not evidence of
interaction at the 5% significance level, although the p-value was close. Thus, we conclude there
is not significant evidence that the treatment effect depends upon gender.
Q4. Answer the following questions by hand (not using SAS). The fitted multiple logistic
regression models for baseline SBP and gender for a subset of the study participants
looks like:
, )
, )
, )
Pr
ln 0.87 0.68* 0.03 137.2
1 Pr
mi
ifemale sbp
mi
|
= +
|
|

' .
and
, )
0.46
ifemale
SE

=
.
a. Estimate the odds ratio and 95% CI for gender. Is there evidence of a gender effect?
A. The odds ratio can be estimated by exp(0.68)=1.97 . Since SE=0.46 then a 95% CI is given
by:
, ) , ) , ) , ) , )
, ) , ) , )
, )
95% CI = exp 0.68- 1.96*0.46 , exp 0.68+ 1.96*0.46
= exp -0.2216 , exp 1.5816
= 0.80, 4.86
The 95% CI is pretty wide, indicating there was lower precision in estimating gender effects.
Since the 95% CI for the OR includes 1, we can deduce that there is not a significant gender
effect at the 5% level.
However we could formally carry out a Wald test by calculating Z = 0.68/0.46 = 1.478. Since this
does not exceed the critical value of 1.96 at the 5% significance level, we fail to reject the null
hypothesis of no gender effect.
b. Estimate the probability that a Female with baseline SBP=150 has a MI by 5 years.
A. We can estimate this probability by simply plugging in the covariate information into the
probability form of the estimated logistic regression model:
, )
, ) , )
, ) , )
, )
, )
exp 0.87 0.68*1 0.03 150 137.2
Pr 1
1 exp 0.87 0.68*1 0.03 150 137.2
exp 0.574
1 exp 0.574
0.3603
MI
+
= =
+ +

=
+
=
Thus a female with SBP=150 at baseline has about a 36% chance of having a MI.
7
c. Estimate the probability that a Male at the average SBP of the sample (137.2) will have a
MI by 5 years
A. Using the same formula we can calculate this probability:
, )
, ) , )
, ) , )
, )
, )
exp 0.87 0.68*0 0.03 137.2 137.2
Pr 1
1 exp 0.87 0.68*0 0.03 137.2 137.2
exp 0.87
1 exp 0.87
0.2953
MI
+
= =
+ +

=
+
=
Thus a male with average SBP has about a 29.5% chance of having a MI.
d. Explain what quasi-complete separation is and how you would remediate it if you had it
in a multiple logistic regression model with 3 categorical predictor variables. Include the
detailed steps you would take.
A. Quasi-complete separation occurs when a predictor variable is so good at predicting the
dichotomous outcome that for some linear combination of the predictors the outcome is
perfectly predicted. When this happens statistical estimation breaks down and SEs and
CIs become extremely inflated. SAS prints out a warning in the SAS Log file when this
occurs and also can be diagnosed after observing extreme, nonsensical SEs and very
wide CIs. To remediate for a categorical predictor one could collapse the problematic
categories by reducing the number of groups. If convergence problems remain other
options include dropping the variable or excluding the cases where perfect outcome
prediction is observed.
Confidence Intervals
1. Margin of error and the confidence interval.
A study based on a sample of size 25 reported a mean of 93 with a margin of error
of 11 for 95% confidence.
(a) Give the 95% confidence interval.
93 11 produces the interval (93 11, 93 + 11); (82, 104)
(b) If you wanted 99% confidence for the same study, would your margin of error
be greater than equal to, or less than 11? Explain your answer.
For a 99% confidence interval the margin of error would be larger since Z
*
for a 99%
confidence interval is larger than for a 95% confidence interval.
2. A study with 36 observations had a mean of 70. Assume that the standard
deviation is 12. Make a diagram similar to Figure 6.6 (page 364) that
8
illustrates the effect of the confidence level on the width of the interval. Use
80%, 90%, 95%, and 99%. Summarize what the diagram shows.
Z
*
for 80% is 1.282
Z
*
for 90% is 1.645
Z
*
for 95% is 1.960
Z
*
for 99% is 2.576
x
=
12
36
= 2
36 1.282(2) (33.436, 38.564)
36 1.645(2) (32.71, 39.29)
36 1.96(2) (32.08, 39.92)
36 2.576(2) (30.848, 41.152)
3.
3. The mean number of calories consumed by women in the United States who are
19 to 30 years of age is x = 1791 calories per day. The standard deviation is 31
calories.
5
You will study a sample of 200 women in this age range, and one of the
variables you will collect is calories consumed per day.
(a) What is the standard deviation of the sample mean x ?
31
200
= 2.19 The purpose of this question is to make sure you understand what
x
=
n
o
o really means. Study how this question was phrased.
(b) The 68-95-99.7 rule says that the probability is about 0.95 that x is
within
__2(2.19) = 4.38__ calories of the population mean . Fill in the blank.
The purpose of this question is to tie in the visual that we had in section 1.3, when the
normal distribution was first introduced along with the 68-95-99.7 rule. We know now
that the numbers we associated with 68-95-99.7 are just approximations so it would be
easier to remember.
31.00 32.00 33.00 34.00 35.00 36.00 37.00 38.00 39.00 40.00
9
(c) About 95% of all samples will capture the true mean of calories consumed per
day in the interval x plus or minus 2(2.19) = 4.38 calories/day. If you use the
actual multiplier of 1.96 then the answer is 1.96(2.19) 4.29 cal/day.
The difference between (b) and (c) concerns what was discussed in class with respect
how a confidence interval works:
(b) There is approximately a 95% chance that x is in the interval ( - 4.38, + 4.38).
(c) There is approximately a 95% chance that is in the interval ( x - 4.38, x + 4.38).
4. Computers in some vehicles calculate various quantities related to
performance. One of these is the fuel efficiency, or gas mileage, usually
expressed as miles per gallon (MPG). For one vehicle equipped in this way,
the MPG were recorded each time the gas tank was filled, and the computer
was then reset.
6
Here are the mpg values for a random sample of 20 of these records.
41.
0
50.
7
36.
6
37.
3
34.
3
45.
0
48.
0
43.
2
47.
7
42.
2
43.
2
44.
6
48.
4
46.
4
46.
8
39.
2
37.
3
43.
5
44.
3
43.
3
Suppose that the standard deviation is known to be o = 3.5 mpg.
(a) What is
x
, the standard deviation of x ?
x
=
3.5
20
|
|
' .
= 0.7826
(b) Give a 95% confidence interval for , the mean mpg for this vehicle x = 43.15
mpg
43.15 1.96 (0.7826)
(41.62, 44.68)
5. To assess the accuracy of a laboratory scale, a standard weight known to
weigh 10 grams is weighed repeatedly. The scale readings are Normally
distributed with unknown mean (this mean is 10 grams if the scale has no
bias). The standard deviation of the scale readings is known to be 0.0002
gram.
(a)The weight is measured five times- The mean result is 10.0023 grams. Give a 98%
confidence interval for the mean of repeated measurements of
the weight.
10
10.00232.326
0.0002
5
(10.0021, 10.0025)
(b) How many measurements must be averaged to get a margin of error of 0.0001
with 98% confidence?
|
|
' .
2
2.326(0.0002)
0.0001
= 21.64. We need a sample of 22 values.