Fu Ch11 Linear Regression

EPI 809/Spring 2008 1
Chapter 11
Regression and Correlation
methods
Learning Objectives
1. Describe the Linear Regression Model
2. State the Regression Modeling Steps
3. Explain Ordinary Least Squares
4. Compute Regression Coefficients
5. Understand and check model assumptions
6. Predict Response Variable
7. Comments of SAS Output
Learning Objectives
8. Correlation Models
9. Link between a correlation model and a
regression model
10. Test of coefficient of Correlation

Models
What is a Model?

1. Representation of
Some Phenomenon

Non-Math/Stats Model

1. Representation of
Some Phenomenon

Non-Math/Stats Model

What is a Math/Stats Model?
1. Often Describe Relationship between
Variables

2. Types
- Deterministic Models (no randomness)

- Probabilistic Models (with randomness)

Deterministic Models
1. Hypothesize Exact Relationships
2. Suitable When Prediction Error is Negligible
3. Example: Body mass index (BMI) is measure of
body fat based

Metric Formula: BMI = Weight in Kilograms
(Height in Meters)
2

Non-metric Formula: BMI = Weight (pounds)x703
(Height in inches)
2

Probabilistic Models
1. Hypothesize 2 Components
Deterministic
Random Error
2. Example: Systolic blood pressure of newborns
Is 6 Times the Age in days + Random Error
SBP = 6xage(d) + c
Random Error May Be Due to Factors
Other Than age in days (e.g. Birthweight)
Types of
Probabilistic
Models
Regression
Models
Correlation
Models
Other
Models
Regression Models
Types of
Probabilistic
Models
Regression
Models
Correlation
Models
Other
Models
Regression Models
Relationship between one dependent
variable and explanatory variable(s)
Use equation to set up relationship
Numerical Dependent (Response) Variable
1 or More Numerical or Categorical Independent
(Explanatory) Variables
Used Mainly for Prediction & Estimation
Regression Modeling Steps
1. Hypothesize Deterministic Component
Estimate Unknown Parameters
2. Specify Probability Distribution of
Random Error Term
Estimate Standard Deviation of Error
3. Evaluate the fitted Model
4. Use Model for Prediction & Estimation
Model Specification
Specifying the deterministic
component
1. Define the dependent variable and
independent variable

2. Hypothesize Nature of Relationship
Expected Effects (i.e., Coefficients Signs)
Functional Form (Linear or Non-Linear)
Interactions
Model Specification
Is Based on Theory
1. Theory of Field (e.g., Epidemiology)
2. Mathematical Theory
3. Previous Research
4. Common Sense
Thinking Challenge:
Which Is More Logical?

Years since seroconversion
CD+ counts
CD+ counts
CD+ counts
CD+ counts
OB/GYN Study
Types of
Regression Models
Types of
Regression Models
Regression
Models
Types of
Regression Models
Regression
Models
Simple
1 Explanatory
Variable
Types of
Regression Models
Regression
Models
2+ Explanatory
Variables
Simple Multiple
1 Explanatory
Variable
Types of
Regression Models
Regression
Models
Linear
2+ Explanatory
Variables
Simple Multiple
1 Explanatory
Variable
Types of
Regression Models
Regression
Models
Linear
Non-
Linear
2+ Explanatory
Variables
Simple Multiple
1 Explanatory
Variable
Types of
Regression Models
Regression
Models
Linear
Non-
Linear
2+ Explanatory
Variables
Simple Multiple
Linear
1 Explanatory
Variable
Types of
Regression Models
Regression
Models
Linear
Non-
Linear
2+ Explanatory
Variables
Simple Multiple
Linear
1 Explanatory
Variable
Non-
Linear
Linear Regression
Model
Types of
Regression Models
Regression
Models
Linear
Non-
Linear
2+ Explanatory
Variables
Simple
Non-
Linear
Multiple
Linear
1 Explanatory
Variable
Y
Y = mX + b
b = Y-intercept
X
Change
in Y
Change in X
m = Slope
Linear Equations
1984-1994 T/Maker Co.
Y X
i i i
= + + | | c
0 1
Linear Regression Model
1. Relationship Between Variables Is a
Linear Function
Dependent
(Response)
Variable
(e.g., CD+ c.)
Independent
(Explanatory) Variable
(e.g., Years s. serocon.)
Population
Slope
Population
Y-Intercept
Random
Error
Population & Sample
Regression Models
Population & Sample
Regression Models
Population

Population & Sample
Regression Models
Unknown
Relationship
Population
Y X
i i i
= + + | | c
0 1

Population & Sample
Regression Models
Unknown
Relationship
Population Random Sample
Y X
i i i
= + + | | c
0 1

Population & Sample
Regression Models
Unknown
Relationship
Population Random Sample
Y X
i i i
= + + | | c
0 1
Y X
i i i
= + +

| | c
0 1

Y
X
Population Linear Regression
Model
Y X
i i i
= + + | | c
0 1
( )
i
X Y E
1 0
| | + =
Observed
value
Observed value
c
i
= Random error
Y
X
Y X
i i i
= + +

| | c
0 1
Sample Linear Regression
Model

Y X
i i
= + | |
0 1
Unsampled
observation
c
i
= Random
error
Observed value
^
Estimating Parameters:
Least Squares Method
0
20
40
60
0 20 40 60
X
Y
Scatter plot
1. Plot of All (X
i
, Y
i
) Pairs
2. Suggests How Well Model Will Fit
Thinking Challenge
How would you draw a line through the
points? How do you determine which line
fits best?
0
20
40
60
0 20 40 60
X
Y
Thinking Challenge
fits best?
0
20
40
60
0 20 40 60
X
Y
Slope changed
Intercept unchanged
Thinking Challenge
fits best?
0
20
40
60
0 20 40 60
X
Y
Slope unchanged
Intercept changed
Thinking Challenge
fits best?
0
20
40
60
0 20 40 60
X
Y
Slope changed
Intercept changed
Least Squares
1. Best Fit Means Difference Between
Actual Y Values & Predicted Y Values Are
a Minimum. But Positive Differences Off-
Set Negative ones
Least Squares
Actual Y Values & Predicted Y Values is a
Minimum. But Positive Differences Off-Set
Negative ones. So square errors!

( )

= =
=
n
i
i
n
i
i i
Y Y
1
2
1
2
c
Least Squares
Actual Y Values & Predicted Y Values Are
a Minimum. But Positive Differences Off-
Set Negative. So square errors!

2. LS Minimizes the Sum of the Squared
Differences (errors) (SSE)
( )

= =
=
n
i
i
n
i
i i
Y Y
1
2
1
2
c
Least Squares Graphically
c
2
Y
X
c
1
c
3
c
4
^
^
^
^
Y X
2 0 1 2 2
= + +

| | c

Y X
i i
= + | |
0 1
LS minimizes

c c c c c
i
i
n
2
1
1
2
2
2
3
2
4
2
= + + +
=
Coefficient Equations
Prediction equation

Sample slope

Sample Y - intercept

i i
x y
1 0

| | + =
( )( )
( )

= =
2
1
x x
y y x x
SS
SS
i
i i
xx
xy
|
x y
1 0

| | =
Derivation of Parameters (1)
Least Squares (L-S):
Minimize squared error

x y
1 0

| | =
( )
( )
2
2
0 1
0 0
0 1
0
2
i i i
y x
ny n n x
c | |
| |
| |
c c
= =
c c
=

( )
2
2
0 1
1 1
n n
i i i
i i
y x c | |
= =
=

Derivation of Parameters (1)
Least Squares (L-S):
Minimize squared error

( )
( )
( )
2
2
0 1
1 1
0 1
1 1
0
2
2
i i i
i i i
i i i
y x
x y x
x y y x x
c | |
| |
| |
| |
c c
= =
c c
=
= +

( ) ( )
( )( ) ( )( )
1
1
1
i i i i
i i i i
xy
xx
x x x x y y
x x x x x x y y
SS
SS
|
|
|
=
=
=

Computation Table
X
i
Y
i
X
i
2
Y
i
2
X
i
Y
i
X
1
Y
1
X
1
2
Y
1
2
X
1
Y
1
X
2
Y
2
X
2
2
Y
2
2
X
2
Y
2
: : : : :
X
n
Y
n
X
n
2
Y
n
2
X
n
Y
n
EX
i
EY
i
EX
i
2
EY
i
2
EX
i
Y
i
Interpretation of Coefficients
1. Slope (|
1
)
Estimated Y Changes by |
1
for Each 1 Unit
Increase in X
If |
1
= 2, then Y Is Expected to Increase by 2 for
Each 1 Unit Increase in X
^
^
^
1. Slope (|
1
)
Estimated Y Changes by |
1
for Each 1 Unit
Increase in X
If |
1
= 2, then Y Is Expected to Increase by 2 for
Each 1 Unit Increase in X
2. Y-Intercept (|
0
)
Average Value of Y When X = 0
If |
0
= 4, then Average Y Is Expected to Be
4 When X Is 0
^
^
^
^
^
Parameter Estimation Example
Obstetrics: What is the relationship between
Mothers Estriol level & Birthweight using the
following data?
Estriol Birthweight
(mg/24h) (g/1000)
1 1
2 1
3 2
4 2
5 4
0
1
2
3
4
0 1 2 3 4 5 6
Scatterplot
Birthweight vs. Estriol level
Birthweight
Estriol level
Parameter Estimation Solution
Table
X
i
Y
i
X
i
2
Y
i
2
X
i
Y
i
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
( )( )
( )
( )( ) 10 . 0 3 70 . 0 2

70 . 0
5
15
55
5
10 15
37
1 0
2
1
2
1
2
1
1
1
1
= = =
=
=
|
.
|
\
|
|
.
|
\
|
|
.
|
\
|
=
=
=
=
=
X Y
n
X
X
n
Y
X
Y X
n
i
n
i
i
i
n
i
i
n
i
i
n
i
i i
| |
|
Coefficient Interpretation
Solution
Solution
1. Slope (|
1
)
Birthweight (Y) Is Expected to Increase by .7
Units for Each 1 unit Increase in Estriol (X)
^
Solution
1. Slope (|
1
)
Birthweight (Y) Is Expected to Increase by .7
Units for Each 1 unit Increase in Estriol (X)
2. Intercept (|
0
)
Average Birthweight (Y) Is -.10 Units When
Estriol level (X) Is 0
Difficult to explain
The birthweight should always be positive
^
^
SAS codes for fitting a simple linear
regression
Data BW; /*Reading data in SAS*/
input estriol birthw@@;
cards;
1 1 2 1 3 2
4 2 5 4
;
run;

PROC REG data=BW; /*Fitting linear regression
models*/
model birthw=estriol;
run;
Parameter Estimates

Parameter Standard
Variable DF Estimate Error t Value Pr > |t|

Intercept 1 -0.10000 0.63509 -0.16 0.8849
Estriol 1 0.70000 0.19149 3.66 0.0354
Parameter Estimation
SAS Computer Output
|
0
^
|
1
^
Parameter Estimation Thinking
Challenge
Youre a Vet epidemiologist for the county
cooperative. You gather the following data:
Food (lb.) Milk yield (lb.)
4 3.0
6 5.5
10 6.5
12 9.0
What is the relationship
between cows food intake and milk yield?
1984-1994 T/Maker Co.
0
2
4
6
8
10
0 5 10 15
Scattergram
Milk Yield vs. Food intake*
M. Yield (lb.)
Food intake (lb.)
Table*
X
i
Y
i
X
i
2
Y
i
2
X
i
Y
i
4 3.0 16 9.00 12
6 5.5 36 30.25 33
10 6.5 100 42.25 65
12 9.0 144 81.00 108
32 24.0 296 162.50 218
Parameter Estimation Solution*
( )( )
( )
( )( ) 80 . 0 8 65 . 0 6

65 . 0
4
32
296
4
24 32
218
1 0
2
1
2
1
2
1
1
1
1
= = =
=
=
|
.
|
\
|
|
.
|
\
|
|
.
|
\
|
=
=
=
=
=
X Y
n
X
X
n
Y
X
Y X
n
i
n
i
i
i
n
i
i
n
i
i
n
i
i i
| |
|
Solution*
Solution*
1. Slope (|
1
)
Milk Yield (Y) Is Expected to Increase by
.65 lb. for Each 1 lb. Increase in Food intake
(X)
^
Solution*
1. Slope (|
1
)
Milk Yield (Y) Is Expected to Increase by
.65 lb. for Each 1 lb. Increase in Food intake
(X)

2. Y-Intercept (|
0
)
Average Milk yield (Y) Is Expected to Be 0.8
lb. When Food intake (X) Is 0
^
^

Fu Ch11 Linear Regression

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Fu Ch11 Linear Regression

Caricato da

Copyright:

Formati disponibili

EPI 809/Spring 2008 1

Potrebbero piacerti anche