Mathematical Modelling

Mathematical Modeling
My Students
February, 2010
It is mostly based on the textbook, Frank R. Giordano, Maurice D. Weir, and William P. Fox, A First Course in
Mathematical Modeling, 3rd Ed and it has been reorganized and retyped by Jae Lee.
Mathematical Modeling Spring, 2010
Page 2 of 57
CONTENTS
1 Modeling Change 5
1.1 Modeling Change with Difference Equations . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Approximating Change with Difference Equations . . . . . . . . . . . . . . . . . . . . . . . 7
Topic I. Discrete Versus Continuous Change . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Topic II. Model Renement: Modeling Births, Deaths, and Resources . . . . . . . . . . . . 8
1.3 Solutions to Dynamical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Topic I. Method of Conjecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Topic II. Homogeneous Linear Dynamical System a
n+1
= ra
n
, r constant . . . . . . . . . . 10
Topic III. LongTerm Behavior of a
n+1
= ra
n
, r constant . . . . . . . . . . . . . . . . . . . 11
Topic IV. Nonhomogeneous Linear Dynamical System a
n+1
= ra
n
+b, r and b constant . . . 11
Topic V. Finding and Classifying Equilibrium Values . . . . . . . . . . . . . . . . . . . . . 13
Topic VI. Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4 Systems of Difference Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 The Modeling Process 17
2.1 Mathematical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Modeling Using Proportionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Topic I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Topic II. Geometric Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Topic III. Modeling Vehicular Stopping Distance . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Modeling Using Geometric Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Topic I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Topic II. Testing Geometric Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4 Automobile Gasoline Mileage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5 Body Weight and Height, Strength and Agility . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Modeling Fitting 27
Topic I. Relationship Between Model Fitting and Interpolation . . . . . . . . . . . . . . . . 27
Topic II. Sources of Error in the Modeling Process . . . . . . . . . . . . . . . . . . . . . . 28
3.1 Fitting Models to Data Graphically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Topic I. Visual Model Fitting with the Original Data . . . . . . . . . . . . . . . . . . . . . . 28
Topic II. Transforming the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 Analytic Methods of Model Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Topic I. Chebyshev Approximation Criterion . . . . . . . . . . . . . . . . . . . . . . . . . 30
Topic II. Minimizing the Sum of the Absolute Deviations . . . . . . . . . . . . . . . . . . . 31
Topic III. LeastSquares Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Topic IV. Relating the Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 Applying the LeastSquares Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Topic I. Fitting a Straight Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3
Topic II. Fitting a Power Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Topic III. Transformed LeastSquares Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.4 Choosing a Best Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4 Chapter 7 Discrete Optimization Modeling 45
Section 7.4 Linear Programming III: Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . 45
5 Chapter 8 Dimensional Analysis 53
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Dimensions as Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6 Chapter 10 Modeling with a Differential Equation 55
10.5 Numerical Approximation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7 Chapter 11 Modeling with Systems of Differential Equations 57
11.1 Graphical Solutions of Autonomous Systems of FirstOrder Differential Equations . . . . . . 57
Page 4 of 57
Chapter 1
Modeling Change
A mathematical model is an idealization of the realworld phenomenon and never a completely accurate
representation. In modeling our world, we are often interested in predicting the value of a variable at some
time in the future such as a population, a real estate value, and the number of people with a communicative
disease.
.
.RealWorld Data .Model
.Mathematical Conclusions
.Predictions/Explanations
.Simplication
.Analysis
.Interpretation
.Verication
Figure 1.1: A ow of the modeling process beginning with an examination of realworld data
One very powerful simplifying relationship is proportionality.
Denition 1.0.1. Two variables, x and y, are proportional (to each other) if there is a nonzero constant k such
that y = kx. We write y x.
When x and y are proportional, the graph of y versus x is a straight line passing through the origin. When x
and y are proportional, one of our concerns is to nd the constant of proportionality k. Moreover, we observe
x and y are proportional if and only if y/x or x/y or (y/x)
p
or (x/y)
p
is constant, where p is any real number.
Example 1.0.2. Consider a springmass system. An experiment gives the following table.
Elongation (e) 1.000 1.875 2.750 3.250 4.375 4.875 5.675 6.500 7.250 8.000 8.750
Mass (m) 50 100 150 200 250 300 350 400 450 500 550
A simple computation shows that the ratio of e over m is roughly a constant:
e
m
= 0.0171,
where the number is the average of each ratio e
i
/m
i
, i = 1, 2, . . . , 11. Since the ratio e/m is roughly a constant
0.017, it is allowed to say that m and e are proportional with the relation e = 0.0171m.
When we plot the points of (m, e), we can observe that a graph close to all points looks like a straight line
passing through the origin. See the gure 1.2.
A paradigm to use in modeling change is
future value = present value + change, i.e., change = future value present value.
If the pattern of the model is taking place over discrete time periods, the preceding construction leads to
a difference equation. In the case that it is taking place continuously with respect to time, it leads to a
differential equation.
5
Figure 1.2: Data from springmass system with proportionality line
1.1 Modeling Change with Difference Equations.
Denition 1.1.1. For a sequence of numbers, A = {a
0
, a
1
, a
2
, . . .}, the difference a
n+1
a
n
is called the n
th
rst difference and denoted by a
n
, i.e.,
a
n
= a
n+1
a
n
, n = 0, 1, 2, . . . .
Geometrically, the rst difference represents the vertical change in the graph of the sequence during one time
period.
Example 1.1.2 (Savings Certicate). Consider the value of a savings certicate initially worth $1000 that
accumulates interest paid each month at 1% per month.
Let a
n
be the value of the certicate after n months. Then a
0
= 1000 and because of the interest,
a
0
= a
1
a
0
= 0.01a
0
a
1
= a
2
a
1
= 0.01a
1
.
.
.
a
n
= a
n+1
a
n
= 0.01a
n
.
The last equation can be rewritten by
a
n+1
= a
n
+0.01a
n
= 1.01a
n
, i.e., a
n+1
= 1.01a
n
with a
0
= 1000,
which is called the dynamical system model and the equation is called the dynamical system.
Example 1.1.3. Consider the value of a savings certicate initially worth $1000 that accumulates interest
paid each month at 1% per month. We withdraw $50 from the account each month.
Let a
n
be the value of the certicate after n months. Then a
0
= 1000 and because of the interest and the
withdrawal, we have
a
n
= a
n+1
a
n
= 0.01a
n
50, i.e., a
n+1
= 1.01a
n
50 with a
0
= 1000.
How to describe a change mathematically? Often it is necessary to plot the change and observe a pattern and
describe the change in mathematical terms. Simply we will try to nd
change = a
n
= some function f .
Page 6 of 57
Example 1.1.4 (Mortgaging
a Home). Six years ago your parents purchased a home by nancing $80, 000
for 20 years paying monthly payments of $880.87 with a monthly interest of 1%. Currently they have made
72 payments. Currently how much do they owe on the mortgage?
ANSWER. Let b
n
be the amount of money owed to the bank after n months. Then we have
b
n
= b
n+1
b
n
= 0.01b
n
880.87, i.e., b
n+1
= 1.01b
n
880.87 with b
0
= 80, 000.
The answer to the raised question is b
27
, which can be easily computable.
Month (n) 0 1 2 3 . . . 72
Owed Money (b
n
) 80000 79919.1 79837.5 79755 71532.1
Month (n) 237 238 239 240
Owed Money (b
n
) 2589.58 1734.6 871.078 1.0814
Based on the table of n and b
n
, we have the gure 1.3 below from 0 month to 240 months.
Figure 1.3: Mortgaging a Home
Denition 1.1.5. A sequence is a function whose domain is the set of all nonnegative integers and whose
range is a subset of the real numbers. A dynamical system is a relationship among terms in a sequence. A
numerical solution is a table of values satisfying the dynamical system.
1.2 Approximating Change with Difference Equations.
In this section, we approximate some observed change to complete the expression
change = a
n
= some function f .
Topic I. Discrete Versus Continuous Change.
Some changes takes place in discrete time intervals such as the depositing of interest in an account. In this
case, we consider a difference equation. But some changes happen continuously such as the change in the
temperature of a cold can of soda on a worm day. In this case, a differential equation can be dealt with.
Example 1.2.1 (Growth of a Yeast
Culture). By an experiment measuring the growth of a yeast culture, we

have the following table.
Mortgage: a legal agreement by which a bank, building society, etc. lends money at interest in exchange for taking title of
the debtors property, with the condition that the conveyance of title becomes void upon the payment of the debt (Concise Oxford
English Dictionary)
Yeast: a microscopic singlecelled fungus capable of converting sugar into alcohol and carbon dioxide
Page 7 of 57
n p
n
p
n
0 9.6 8.7
1 18.3 10.7
2 29.0 18.2
3 47.2 23.9
4 71.1 48.0
5 119.1 55.5
6 174.6 82.7
7 257.3
Here n represents the time in hour, p
n
observed yeast biomass, and p
n
= p
n+1
p
n
the change in biomass.
We observe the ratio (p
n
)/p
n
is roughly a constant,
p
n
p
n
= 0.6057,
which is the average of the ratios (p
i
)/p
i
, i =0, 1, 2, . . . , 6. It implies that p
n
and p
n
are proportional with
the constant of proportionality 0.6057 so that
p
n
= 0.6057p
n
, i.e., p
n+1
= 1.6057p
n
.
Since the ratio p
n+1
/p
n
= 1.6057 > 1, so the model predicts the population will increase forever. See the
gure 1.4.
Figure 1.4: Change in Biomass versus Biomass
Topic II. Model Renement: Modeling Births, Deaths, and Resources.
Certain resources (e.g., food) can support only a maximum population level rather than one that increases
indenitely.
Example 1.2.2 (Growth of a Yeast Culture Revisited). Under the restriction, suppose we have the table 1.8
on the page 11 of the textbook. The table gives the plot (the one on the lefthand side).
From the graph of the population above, the population appears to be approaching a limiting value, which
seems to be 665. So we may propose
p
n
(665 p
n
)p
n
.
By computing the ratio (p
n
)/((665 p
n
)p
n
), we can estimate a constant
p
n
(665 p
n
)p
n
= 0.000802886,
Page 8 of 57
which is the average of (p
i
)/((665 p
i
)p
i
), i = 1, 2, . . . , 18. It implies
p
n
= 0.000802886(665 p
n
)p
n
, p
n+1
= p
n
+0.000802886(665 p
n
)p
n
with p
0
= 9.6.
(The textbook uses the linear proportionality with the proportionality k = 0.00082.)
Let us solve our model p
n+1
= p
n
+0.000802886(665p
n
)p
n
with p
0
=9.6 numerically. That is, for each n,
we compute p
n
and plot the data (n, p
n
) and compare the one obtained by the experiment. See the gure 1.5.
Figure 1.5: Red #: Experimental Result, Blue : Model Result

1.3 Solutions to Dynamical Systems.
Topic I. Method of Conjecture.
The method of conjecture is a powerful mathematical technique to hypothesize the form of a solution to a
dynamical system and then to accept or reject the hypothesis. The method has four steps: Look for Pattern,
Conjecture, Test Conjecture and Conclusion.
Example 1.3.1 (Savings Certicate (Revisited)). A savings certicate is initially worth $1000 accumulated
interest paid each month at 1% of the balance. No deposits or withdrawals occurred in the account. Letting
a
n
be the amount in the account after n months, we deduce the dynamical system
a
n+1
= 1.01a
n
with a
0
= 1000. (1.3.1)
Page 9 of 57
Step 1 Look for Pattern:
a
1
= 1.01a
0
a
2
= 1.01a
1
= 1.01(1.01a
0
) = 1.01
2
a
0
a
3
= 1.01a
2
= 1.01(1.01
2
a
0
) = 1.01
3
a
0
. . . . . .
a
n
= 1.01
n
a
0
.
Step 2 Conjecture: From the step 1, we conjecture a
n
= 1.01
n
a
0
.
Step 3 Test Conjecture: The conjecture implies
a
n+1
= 1.01
n+1
a
0
= 1.01(1.01
n
a
0
) = 1.01a
n
,
which is the dynamical system (1.3.1). Thus, our conjecture is right.
Step 4 Conclusion: The solution of the dynamical system (1.3.1) is a
n
= 1.01
n
a
0
= 1.01
n
1000.
Topic II. Homogeneous Linear Dynamical System a
n+1
= ra
n
, r constant.
Theorem 1.3.2. The solution of the linear dynamical system a
n+1
= ra
n
with constant r = 0 is a
n
= r
n
a
0
,
where a
0
is the given initial value.
Example 1.3.3 (Sewage

Treatment). A sewage treatment plant processes raw sewage to produce usable
fertilizer and clean water by removing all other contaminants. The process is such that each other 12% of
remaining contaminants in a processing tank are removed.
Questions:
1. What percentage of the sewage would remain after 1 day?
2. How long would it take to lower the amount of sewage by half?
3. How long until the level of sewage is down to 10% of the original level?
ANSWER. Let a
n
be the amount of sewage contaminants after n hours and a
0
the initial amount. Then we
build the model
a
n+1
= a
n
0.12a
n
= 0.88a
n
, i.e., a
n+1
= 0.88a
n
.
Using the Theorem above, we have the solution of the dynamical system,
a
n
= 0.88
n
a
0
.
Answer to Question 1: Since 1 day is equivalent to 24 hours, so the answer is
a
24
= 0.88
24
a
0
= 0.0465a
0
.
It means the level of contaminants in the sewage can be reduced by more than 95% at the end of the rst day.
Answer to Question 2: The question is about the time n satisfying a
n
= 0.5a
0
. So we solve
0.5a
0
= 0.88
n
a
0
= 0.5 = 0.88
n
= n =
ln0.5
ln0.88
= 5.42
Hence, it takes about 5.42 hours to lower the contaminants to half their original level.
Answer to Question 3: The question is about the time n satisfying a
n
= 0.1a
0
. So we solve
0.1a
0
= 0.88
n
a
0
= 0.1 = 0.88
n
= n =
ln0.1
ln0.88
= 18.01
Hence, it takes about 18 hours before the contaminants are reduced to 10% of their original level.
Sewage: refuse liquids or waste matter usually carried off by sewers

Page 10 of 57
Topic III. LongTerm Behavior of a
n+1
= ra
n
, r constant.
By the Theorem, we recall that the solution of a
n+1
= ra
n
is a
n
= r
n
a
0
. We observe the longterm behavior
of a
n
= r
n
a
0
when n is sufciently large.
r Behavior of a
n
r = 0 The sequence a
n
converges to a
0
and so the dynamical system has a constant solution and equilibrium value at 0.
r = 1 The sequence a
n
converges to a
0
and so all initial values are constant solutions.
r < 0 The sequence a
n
oscillates.
|r| < 1 The sequence a
n
converges to 0 and so it decays to the limiting value of 0.
|r| > 1 The sequence a
n
diverges and so it grows without bound.
Topic IV. Nonhomogeneous Linear Dynamical System a
n+1
= ra
n
+b, r and b constant.
Denition 1.3.4. If a dynamical system a
n+1
= f (a
n
) has a constant solution a
n
= constant, say a
n
= c, then
the constant c is called an equilibrium value or xed point of the system.
Example 1.3.5. Consider a
n+1
= 0.5a
n
+0.1.
(1) When a
0
= 0.1, the given dynamical system implies
a
1
= (0.5)(0.1) +0.1 = 0.15, a
2
= (0.5)(0.15) +0.1 = 0.175
. . . . . . a
15
= 0.19999695
So we may expect that as n , a
n
0.2.
(2) When a
0
a
1
= (0.5)(0.2) +0.1 = 0.2, a
2
= (0.5)(0.2) +0.1 = 0.2
. . . . . . a
15
= 0.2
So we can deduce a
n
= 0.2 = a
0
for any integer n. That is, 0.2 is the equilibrium value.
(2) When a
0
a
1
= (0.5)(0.3) +0.1 = 0.25, a
2
= (0.5)(0.25) +0.1 = 0.225
. . . . . . a
15
= 0.20000305
n
0.2.
When we see the graphs of the sequences (1), (2) and (3), we observe that whatever the initial value is, the
sequence converges to the equilibrium value 0.2. We say this equilibrium value is stable.
Example 1.3.6. Consider b
n+1
= 1.01b
n
1000.
(1) When b
0
= 90000, the given dynamical system implies
b
1
= (1.01)(90000) 1000 = 89900
b
2
= (1.01)(89900) 1000 = 89799
. . . . . .
b
15
= 88390
n
100000.
(2) When b
0
b
1
= (1.01)(100000) 1000 = 100000
b
2
= (1.01)(100000) 1000 = 100000
Page 11 of 57
Figure 1.6: Red (Lower @): a
0
= 0.1, Blue (Middle #): a
0
= 0.2, Black (Upper ): a
0
= 0.3
. . . . . .
b
15
= 100000
So we can deduce b
n
= 100000 = b
0
for any integer n. That is, 100000 is the equilibrium value.
(3) When b
0
b
1
= (1.01)(110000) 1000 = 110100
b
2
= (1.01)(110100) 1000 = 110201
. . . . . .
b
15
= 111610
n
100000.
When we see the graphs of the sequences (1), (2) and (3), we observe that whatever the initial value is, the
sequence does not converge to the equilibrium value 100000. We say this equilibrium value is unstable.
Figure 1.7: Red (Lower): b
0
= 90000, Blue (Middle): b
0
= 100000, Black (Upper): b
0
= 110000
Page 12 of 57
Topic V. Finding and Classifying Equilibrium Values.
Suppose a
n+1
= ra
n
+b has an equilibrium value a = 0. Then by denition, we get
a
n+1
= a = a
n
.
Putting them into the dynamical system, we have
a = ra+b = a =
b
1r
(r = 1).
It implies the following Theorem.
Theorem 1.3.7. If a
n+1
= ra
n
+b has a nonzero equilibrium value a, then the equilibrium value a is given
by
a =
b
1r
(r = 1).
(1) If r = 1 and b = 0, then the dynamical system becomes a
n+1
= a
n
, i.e., any initial value is an equilibrium
value, i.e., every number is an equilibrium value.
(2) If r = 1 but b = 0, then there is no equilibrium value.
Example 1.3.8. Let us use the Theorem 1.3.7 to nd the equilibrium values of the dynamical systems a
n+1
=
0.5a
n
+0.1 in the Example 1.3.5 and b
n+1
= 1.01b
n
1000 in the Example 1.3.6 above.
The Theorem 1.3.7 implies the equilibrium values a and b for each system
a =
0.1
10.5
= 0.2, b =
1000
11.01
= 100000.
Through examples, we can observe the following longterm behavior for a
n+1
= ra
n
+b, b = 0.
r LongTerm Behavior
|r| < 1 Stable equilibrium value
|r| > 1 Unstable equilibrium value
r = 1 Straight line with no equilibrium value
Theorem 1.3.9. The dynamical system a
n+1
= ra
n
+b has the solution
a
n
= r
n
c +
b
1r
, (1.3.2)
where c is a constant depending on the initial condition, explicitly,
a
0
= r
0
c +
b
1r
= c +
b
1r
= c = a
0
b
1r
.
So the solution can be rewritten by
a
n
= r
n
(
a
0
b
1r
)
+
b
1r
= r
n
a
0
+
b(1r
n
)
1r
.
PROOF. Substituting the result (1.3.2) into the given system, we have
a
n+1
= r
n+1
c +
b
1r
, and ra
n
+b = r
(
r
n
c +
b
1r
)
+b = r
n+1
c +
b
1r
.
So a
n
given in (1.3.2) satises the given system a
n+1
= ra
n
+b. Therefore, (1.3.2) is the solution.
Page 13 of 57
We observe that the second term in the solution (1.3.2) is the equilibrium value of the given system a
n+1
=
ra
n
+b.
Example 1.3.10. Solve a
n+1
= 1.01a
n
1000.
ANSWER. By the Example 1.3.6 or Example 1.3.8, we recall that the given system has the equilibrium value
100000. The Theorem 1.3.9 above implies the solution
a
n
= 1.01
n
c +100000 = a
0
= c +100000 = c = a
0
100000
= a
n
= 1.01
n
(a
0
100000) +100000,
where a
0
is the initial value.
Topic VI. Nonlinear Systems.
We recall in the Example of Growth of a Yeast Culture (Revisited):
p
n+1
= p
n
+0.00082(665 p
n
)p
n
= 1.5453p
n
0.00082p
2
n
= 1.5453(10.0005306p
n
)p
n
0.0005306p
n+1
= 1.5453(10.0005306p
n
)0.0005306p
n
which is a nonlinear dynamical system. Letting a
n
= 0.0005306p
n
and r = 1.5453, the equation can be
rewritten by
a
n+1
= r (1a
n
)a
n
.
In this system, when we play with various rs: r = 1.5453, r = 2.750, r = 3.250, r = 3.525 and r = 3.555,
we have plots showing various longterm phenomena. (See the gure 1.21 on page 32 in the textbook.)
1.4 Systems of Difference Equation.
In the previous section, we have studied the equilibrium value of one linear/nonlinear dynamical system. In
this section, we study the equilibrium value of a pair of dynamical systems involved with each other.
Example 1.4.1 (Car Rental Company). A car rental company has distributorships in Orlando and Tampa.
In analyzing the historical records, it is determined that 60% of the cars rented in Orlando are returned to
Orlando, whereas 40% end up in Tampa. Of the cars rented from the Tampa ofce, 70% are returned to
Tampa, whereas 30% end up in Orlando.
Questions:
1. Will a sufcient number of cars end up in each city to satisfy the demand for cars in that city?
2. If not, how many cars must the company transport from Orlando to Tampa or from Tampa to Orlando?
ANSWER. Let O
n
and T
n
be the number of cars in Orlando and Tampa, respectively, at the end of day n.
Then, we can build the following dynamical system model:
O
n+1
= 0.6O
n
+0.3T
n
, and T
n+1
= 0.7T
n
+0.4O
n
,
which is a system of difference equations.
Suppose the system has the equilibrium values O
n
= O and T
n
= T. Then putting them into the equations, we
have
O = 0.6O+0.3T and T = 0.7T +0.4O = O =
3
4
T.
So if the Orlando and Tampa ofces initially have O
0
= 3000 and T
0
= 4000 cars, respectively, then we
observe
O
1
= 0.6(3000) +0.3(4000) = 3000, T
1
= 0.7(4000) +0.4(3000) = 4000
O
2
= 0.6(3000) +0.3(4000) = 3000, T
2
= 0.7(4000) +0.4(3000) = 4000
Page 14 of 57
. . . . . . . . . . . .
O
n
= 3000, T
n
= 4000.
That is, the system remains at the initial value (O
n
, T
n
) = (3000, 4000) = (O
0
, T
0
).
In fact, even if we change the initial values, we observe eventually O
n
3000 and T
n
4000 (assuming the
total number of cars is 7000).
Case 1 Case 2 Case 3 Case 4
O
0
7000 5000 2000 0
T
0
0 2000 5000 7000
For the various starting values in the table above, the gures 1.8 shows that O
n
and T
n
approach to the
equilibrium values, i.e., O
n
3000 and T
n
4000.
Answers to Question 1 and 2: Even though an ofce starts with insufcient number of cars, it can satisfy the
demand on that day. Even 2 days later, it can have the ideal number of cars (i.e., equilibrium values). So we
dont have to transport any car from one city to the other.
Figure 1.8: Red: O
n
, Blue: T
n
Example 1.4.2 (Competitive Hunter Model Spotted Owls and Hawks). Suppose a species of spotted owls
competes for survival in a habitat that also supports hawks. Suppose also that in the absence of the other
species, each individual species exhibits unconstrained growth in which the change in the population during
an interval of time (e.g., 1 day) is proportional to the population size at the beginning of the interval.
Page 15 of 57
Let O
n
and H
n
denote the size of the spotted owls and hawks population, respectively, at the end of day n.
Then by the assumption on the proportionality, we have
O
n
O
n
and H
n
H
n
= O
n
= k
1
O
n
and H
n
= k
2
H
n
,
where k
1
and k
2
are constants.
Assuming that the decrease of the population is proportional to the product of O
n
and H
n
, the system becomes
O
n
= k
1
O
n
k
3
O
n
H
n
and H
n
= k
2
H
n
k
4
O
n
H
n
O
n+1
= (1+k
1
)O
n
k
3
O
n
H
n
and H
n+1
= (1+k
2
)H
n
k
4
O
n
H
n
,
where k
1
, k
2
, k
3
and k
4
are constants. We x the constants k
i
s and consider the system:
O
n+1
= 1.2O
n
0.001O
n
H
n
and H
n+1
= 1.3H
n
0.002O
n
H
n
.
Letting O and H be the equilibrium values, we have
O = 1.2O0.001OH, H = 1.3H 0.002OH = 0 = O(0.20.001H) 0 = H(0.30.002O).
It gives the equilibrium values (O, H) = (150, 200).
Plotting O
n
and H
n
with various initial values as given in the table,
Case 1 Case 2 Case 3
O
0
151 149 10
H
0
199 201 10
we observe O
n
150 and H
n
200 in any case. In Case 1, the population of owls grows indenitely while
the population of hawks goes extinct. In Case 2, the opposite phenomenon occurs. That is, in either case, one
of the two pieces drives the other to extinct. However, it is interesting to see in Case 3 that the population of
hawks grows while owls goes to be vanished. Confer the gure 1.28 on pages 4345 in the textbook.
Let us compare the results on equilibrium values in Examples 1.4.1 (Car Rental Company) and 1.4.2 (Owls
and Hawks). The equilibrium values in Example 1.4.1 (Car Rental Company) are stable and insensitive to
the initial conditions, while those in Example 1.4.2 (Owls and Hawks) are unstable and very sensitive to the
initial conditions.
Page 16 of 57
Chapter 2
The Modeling Process
2.1 Mathematical Models.
Read the textbook.
2.2 Modeling Using Proportionality.
Topic I. Introduction.
From Chapter 1, we recall
y x (i.e., x and y are proportional to each other) if and only if y = kx for some constant k > 0.
It is easy to see
y x x y.
Example 2.2.1.
1. y x
2
if and only if x y
1/2
.
2. y x
n
for a xed constant n if and only if x y
1/n
.
3. y lnx if and only if x e
y
.
4. y e
x
if and only if x lny.
PROOF. Skip. But, one should be able to prove all of them.
Property 2.2.2 (TRANSITIVITY). If z y and y x, then z x.
Topic II. Geometric Interpretation.
Suppose x and y are proportional to each other. Then there is a constant k such that y = kx. It is clear to see
that the graph of y = kx is a line with a slope of k passing through the origin. That is, for x and y which are
proportional, its graph should be a line and should pass the origin. So u and v satisfying v = mu +b with
constant m and b = 0 cannot be proportional.
Example 2.2.3. Suppose there is an open box oating in the water tank. As we add heavy objects into the
box, the water in the tank will ow out. We recall the fact that the volume of the water displaced by the loaded
box is equal to the weight of the loaded box. For an example, if we add a ball into the box and the volume
of the water own out of the tank is 5, then we can say the weight of the ball with the box is 5 (without
considering all the units).
Let y be the volume of the water displaced by the loaded box and x be the weight of the loaded box.
Then we have y = x. However, letting z be the weight of the loaded ball alone, we should have y =
z +(weight of the box), i.e., y and z cannot be proportional, because of the constant term. See the gure 2.1.
If we have a case as in the Example 2.2.3, then is it prohibited from assuming the proportionality? No it is
not. In fact, the answer depends on the problem, specically, the slope. Let us consider two lines having
same slope, L : y = mx and M : y = mx+b, where b = 0. Let (x
0
, y
L
) and (x
0
, y
M
) be points on lines L and M,
respectively. That is,
y
L
= mx
0
, and y
M
= mx
0
+b.
17
.
.Added weight x
.Displaced volume y
Figure 2.1: It is not a proportionality because the line fails to pass through the origin.
Then, we can compute
y
M
y
L
= mx
0
+bmx
0
= b, i.e., y
M
y
L
= b.
We divide both sides by y
M
:
y
M
y
L
y
M
=
b
y
M
. (2.2.1)
Now we observe:
1. If the slope m is relatively large (e.g., compared to 1), then y
M
should be relatively large too, which
implies the ratio (2.2.1) is close to zero, i.e.,
y
M
y
L
y
M
0, i.e., y
M
y
L
0, i.e., y
M
y
L
.
So in this case, we may assume the proportionality for the data tting the model y = mx +b.
2. If the slope m is relatively small (e.g., compared to 1), then y
M
should be relatively small too, which
implies the ratio (2.2.1) is not close to zero, i.e.,
y
M
y
L
y
M
0, i.e., y
M
y
L
0, i.e., y
M
y
L
.
So in this case, we cannot assume the proportionality for the data tting the model y = mx +b.
In a nutshell, if the data t to the model y = mx +b and m is relatively large, then we can assume the
proportionality even though b = 0. See the gure 2.2.
Example 2.2.4 (KEPLERS THIRD LAW). What is the relationship between the orbital period and the mean
distance between the sun and the planet in the solar system?
ANSWER. Method 1. Keplers Third Law:
Keplers Third Law: The square of the orbital period of a planet is directly proportional to the cube of the
semi-major axis of its orbit.
Symbolically, it can be written by
T
2
R
3
,
where T is the orbital period of planet and R is the semimajor axis of the orbit, i.e., mean distance between
the sun and the planet. The proportionality constant is same for any planet around the Sun,
T
2
Earth
R
3
Earth
=
T
2
Mars
R
3
Mars
=
T
2
Planet
R
3
Planet
.
Page 18 of 57
.
.x
.y
.y = mx +b
.y = mx
.
.x
.y
.y = mx +b
.y = mx
Figure 2.2: Assumable and Unassumable Proportionality
Computing the ratio for the Earth, T
Earth
=365.25 days and R
Earth
=92.9 millions of miles =149.508058 millions of kilometers,
we have the constant of proportionality,
365.25
2
92.9
3
= 0.166392 (days
2
/millions of miles
3
).
Thus, we deduce that for any planet, T
2
= 0.166392R
3
, i.e., T = 0.166392
1/2
R
3/2
= 0.407912R
3/2
.
Method 2. Modeling Method based on Data: The following table is from 1993 World Almanac.
Period (T) Mean Distance (R)
Planet (days) (millions of miles)
Mercury 88.0 36.
Venus 224.7 67.25
Earth 365.3 93.
Mars 687.0 141.75
Jupiter 4331.8 483.80
Saturn 10760.0 887.97
Uranus 30684.0 1764.50
Neptune 60188.3 2791.05
Pluto 90466.8 3653.90
When we plot the point (R
3/2
, T), we can approximate a straight line passing through the origin. The slope
(constant of proportionality) can be obtained by choosing any points. However, let us use the leastsquares
criterion in SECTION 3.3 APPLYING THE LEASTSQUARES CRITERION. Then we deduce T = 0.40948R
3/2
.
See the gure 2.3.
Remark 2.2.5 (ASIDE). Keplers First Law: Each planet moves along an ellipse with the sun at one focus.
Keplers Second Law: For each planet, the line form the sun to the planet sweeps out equal areas in equal
times.
We have some famous formulas involved with the proportionalities as follows:
1. (Hookes Law) F = kS, where F is the restoring force in a spring stretched or compressed a distance S.
2. (Newtons Law) F = ma or a = F/m, where a is the acceleration of a mass m subjected to a net external
force F.
3. (Ohms Law) V = iR, where i is the current induced by a voltage V across a resistance R.
Page 19 of 57
Figure 2.3: Keplers Third Law as proportionality
4. (Boyles Law) V = k/p, where under a constant temperature k the volume V is inversely proportional to
the pressure p.
5. (Einsteins Theory of Relativity) E =c
2
M, where under the constant speed of light squared c
2
the energy
E is proportional to the mass M of the object.
6. (Keplers Third Law) T = cR
3/2
, where T is the period (days) and R is the mean distance to the sun.
Topic III. Modeling Vehicular Stopping Distance.
We start with recalling the onecarlength rule (OCL) that allows one car length for every 10 mph of speed.
That rule was also stated as the 2seconds rule (2S) which allows for 2 seconds between cars. In fact, these
two rules are not same and they cannot be compatible. If one is true, the other one should be wrong. First we
will show this incompatibility and then develop a better model on the stopping distance.
If we stick to the rule 2S, then a simple computation shows
1 car length = distance =
(
speed in ft
sec
)
(2 sec)
=
(
10 miles
hr
)(
5280 ft
mile
)(
1 hr
3600 sec
)
(2 sec) = 29.33 ft.
It says that if we follow the rule 2S, then a car should be about 29.33 ft long. However, by statistics, since the
average car length is 15 ft, so the rule OCL should be wrong.
When we use the correct information on the average car length, we have
15 ft = 1 car length = distance =
(
speed in ft
sec
)
(x sec)
=
(
10 miles
hr
)(
5280 ft
mile
)(
1 hr
3600 sec
)
(x sec) =
44
3
x
= x =
45
44
= 1.02273.
It says that the time should be 1.02273 seconds rather than 2 seconds. So if we follow the rule OCL, then the
rule 2S becomes wrong. Although two rules are incompatible, it may be preferable to keep using both rules
for the road safety.
Now, we recall from Chapter 1,
total stopping distance = reaction distance + braking distance.
Page 20 of 57
Using the collected data on the reaction and braking distances, we observe the proportionalities,
d
r
v, d
b
v
2
,
where v is the car speed and d
r
and d
b
are reaction and braking distances, respectively. Explicitly, we deduce
d
r
= 1.1v, d
b
= 0.054v
2
, d = 1.1v +0.054v
2
,
where d is the total stopping distance. See the gure 2.4.
Figure 2.4: Red : Given Data, Blue +: Prediction by Model, Black: OCL rule
In the left one of the gure 2.4, the black line is the prediction by the OCL rule of which equation is given by
d = 1.5v, because the rule says d/v = 15/10 (ft/mph). However, as we can see from the gure, it is denitely
useless especially after the car speed 20. So when we take the guideline given in the table,
Speed (mph) 0 10 10 40 40 60 60 75
Guideline (sec) 1 2 3 4
the modied OCL rule will be better than the original one, as shown in the right one of the gure 2.4.
2.3 Modeling Using Geometric Similarity.
Geometric similarity is a concept related to proportionality and can be useful to simplify the mathematical
modeling process.
Topic I. Introduction.
Denition 2.3.1. Two objects are said to be geometrically similar if there is a onetoone correspondence
between points of the objects such that the ratio of distances between corresponding points is constant for all
possible pairs of points.
Example 2.3.2. Consider two boxes X and X
, where each one has length l, l
, width w, w
, and height h, h
,
respectively. See the gure 2.5. Suppose X and X
are geometrically similar so that there is a onetoone

correspondence between points A, B, C, and A
, B
and C
, and other points and the ratio of the distances

between corresponding points is constant. Then it must be true that
l
l
=
w
w
=
h
h
= k,
Page 21 of 57
.
.B .D
.A
. C
.l
.w
.h
.
.B
.D
.A
. C
.l
.w
.h
Figure 2.5: Two geometrically similar objects X and X
for some constant k > 0.

1. For two triangles ABC and A
in the boxes X and X
, respectively, we observe that the angles are

same, i.e., BCA = B
and BAC = B
. It is easy to understand that the boxes X and X
are
geometrically similar and so those triangles also should be geometrically similar.
The shape is the same for two geometrically similar objects and one object is simply an enlarged copy of the
other. We can think of geometrically similar objects as scaled replicas of one another, as in an architectural
drawing in which all the dimensions are simply scaled by some constant factor.
2. One of the advantages with the geometric similarity lies on simplifying the computations. For the boxes
X and X
above, the volumes of X and X
are, respectively, V
X
= lwh and V
X
= l
. It is easy to see
V
X
V
X
=
lwh
l
= k
3
, V
X
= k
3
V
X
, V
X
V
X
.
Similarly, for the total surface areas S
X
and S
X
of the boxes X and X
, we have
S
X
S
X
=
2(lw+wh+hl)
2(l
+w
+h
)
= k
2
, S
X
= k
2
S
X
, S
X
S
X
.
Moreover, we can nd a relationship between the ratio of volumes and the ratio of surface areas:
V
X
/V
X
S
X
/S
X
=
k
3
k
2
= k,
V
X
V
X
= k
S
X
S
X
,
V
X
V
X
S
X
S
X
.
Now let us choose l and l
between the dimensions of the boxes. Since l/l
= k and S
X
/S
X
= k
2
, so we have
S
X
S
X
= k
2
=
l
2
l
2
,
S
X
l
2
=
S
X
l
2
Aside
= constant.
It implies
S
X
= (constant) l
2
, S
X
l
2
, similarly S
X
l
2
.
By the same argument on V
X
and V
X
, it follows
V
X
l
3
, V
X
l
3
.
Remark 2.3.3 (ASIDE). From S

X
/S
X
= l
2
/l
2
= constant, how can we deduce S
X
/l
2
= S
X
/l
2
= constant?
First it is easy to see
S
X
S
X
=
l
2
l
2
=
S
X
l
2
=
S
X
l
2
,
Page 22 of 57
by multiplying both sides by S
X
/l
2
. Let us consider two functions f (x) and g(y) where we have only two
independent variables x and y. Suppose
f (x)
g(y)
=
x
y
= constant. Then, we have
f (x)
x
=
g(y)
y
.
Letting F(x) = f (x)/x and G(y) =g(y)/y, the equation says F(x) =G(y). If the function of x and the function
of y are same, then both of them should be a constant, i.e., F(x) = G(y) = constant. (For instance, one may
recall that a polynomial of any independent variable of degree 0 is a constant.) This kind of technique is
typically used under the topic, separation of variables, in PARTIAL DIFFERENTIAL EQUATIONS.
In the Example 2.3.2 above, we have argued on S and V with the length l. We can develop the same argument
with the width w and the height h, i.e.,
S w
2
, S h
2
, V w
3
, V h
3
.
Once we choose a dimension (in the Example above, it was the length l), it is called the characteristic
dimension.
Suppose a function f depends on the length l and surface area S and volume V of a box. Then since we
can express S and V in terms of the length l, eventually the function f can be expressed by l, l
2
and l
3
.
For instance, if y = f (l, S,V) = 3l +S V, then there are some constants k
1
and k
2
such that S = k
1
l
2
and
V = k
2
l
3
. So, we have
y = 3l +SV = 3l +k
1
l
2
k
2
l
3
,
which is a function of l, l
2
and l
3
.
Example 2.3.4 (RAINDROP FROM A MOTIONLESS CLOUD). Suppose we are interested in the terminal
velocity (i.e., maximum velocity) of a raindrop from a motionless cloud. We assume only two forces exert
on the raindrop, F
d
due to the air resistance and F
g
due to the gravity. Then the net force F (i.e., sum of all
forces) becomes F = F
g
F
d
so that it falls down. By Newtons Second Law, the net force F should be equal
to ma, i.e.,
F
g
F
d
= ma,
where a is the acceleration and m is the mass of the raindrop. Since the maximum velocity occurs when the
acceleration vanishes (i.e., a = 0), the equation for the terminal velocity becomes
F
g
F
d
= 0, F
g
= F
d
.
Question: What is the relationship between the terminal velocity and the mass of the raindrop?
Assumptions:
(A1) F
d
is proportional to the surface area S times the square of its speed v, i.e., F
d
Sv
2
.
(A2) F
g
is proportional to weight w, i.e., F
g
w.
(A3) Mass m is proportional to the weight w, i.e., m w.
(A4) All the raindrops are geometrically similar.
ANSWER. Thanks to (A4), we can use the proportionality. We recall in general that the surface area S and
the volume V of an object are proportional to l
2
and l
3
for any characteristic dimension l, i.e.,
S l
2
, V l
3
= S V
2/3
.
Because weight w and mass m are proportional to volume, the transitive rule for proportionality gives
S V
2/3
, and V m = S m
2/3
.
Page 23 of 57
With this result and (A1), we have
F
d
Sv
2
m
2/3
v
2
, i.e., F
d
m
2/3
v
2
.
The assumptions (A2) and (A3) yield
F
g
w, and w m = F
g
m.
Since F
d
m
2/3
v
2
and F
g
m, there are some positive constants k
1
and k
2
such that
F
d
= k
1
m
2/3
v
2
, F
g
= k
2
m.
The equation F
g
= F
d
given in the problem implies
k
2
m = k
1
m
2/3
v
2
, m
1/3
=
k
1
k
2
v
2
, m
1/3
v
2
, m
1/6
v.
Thus, the terminal velocity of the raindrop is proportional to its mass raised to the onesixth power.
Remark 2.3.5 (ASIDE: STOKES LAW
). Droplets falling in a motionless air can be modeled by the differ-

ential equation,
d
2
y
dt
2
= 32.2
c
D
2
dy
dt
,
where 32.2 is the gravitational acceleration and c is a xed constant and D is the diameter of the spherical
raindrop and dy/dt is the velocity of the raindrop. So the terminal velocity can be obtained by solving the
differential equation,
0 =
d
2
y
dt
2
= 32.2
c
D
2
dy
dt
,
dy
dt
=
32.2
c
D
2
.
The diameter D is proportional to the radius r of the spherical raindrop and the volume V of the raindrop is
proportional to r
3
, so we can deduce
V D
3
, D
2
V
2/3
.
Since the mass m is proportional to the volume V, the result above becomes
D
2
V
2/3
, V m = D
2
m
2/3
.
Hence, the differential equation implies
dy
dt
D
2
m
2/3
,
dy
dt
m
2/3
,
i.e., the terminal velocity is proportional to the mass raised to the (2/3) power. This is a quite different result
than the one we deduced in the Example 2.3.4 above. Why? What happened? Its because in formulating
the differential equation, we assume the constant gravitational acceleration 32.2. But in the Example 2.3.4
above, we assume that F
g
is not a constant and so F
g
is involved with the mass m.
A droplet falling according to the differential equation above never quite reaches its terminal velocity, but gets
closer and closer to it. Unless its fall is interrupted by hitting the ground, the velocity eventually becomes so
close to the solution of the differential equation that for practical purpose, we consider it equal to the terminal
velocity.
From the book, Concepts of Mathematical Modeling, written by Walter J. Myer

Page 24 of 57
Topic II. Testing Geometric Similarity.
Are a triangle and a rectangle geometrically similar? Clearly, they are not. Because of the differences between
the vertices of two polygons, there is no onetoone correspondence. When we use the geometric similarity
assumption, those involved objects should be of same shape.
Now let us x a polygon, for example, a circle. Then we can think of the radius, area, circumference, arc,
angle of the arc, length of the arc, and so on. We consider two circles of diameters d
1
and d
2
, respectively.
Letting C
1
and C
2
be the circumferences of the circles, it is straightforward to see
C
1
d
1
=
C
2
d
2
= , and
C
1
C
2
=
d
1
d
2
=
d
1
d
2
. (2.3.1)
We recall that the length l of an arc (part of a circle) having the angle in a circle with radius r is given by
l = r. Letting l
1
and l
2
be the lengths of arcs in circles above having the angles
1
and
2
, respectively, we
have
l
1
= r
1
1
and l
2
= r
2
2
.
If
1
=
2
, then there is no geometric similarity between those two arcs. So assuming
1
=
2
, we deduce
l
1
l
2
=
r
1
r
2
=
1
=
2
, and
l
1
l
2
=
2r
1
2r
2
=
d
1
d
2
. (2.3.2)
Combining two results above yields,
C
1
C
2
=
d
1
d
2
=
l
1
l
2
, i.e.,
C
1
C
2
=
l
1
l
2
.
From those two equations, it is deduced that the ratio of distances between corresponding points around any
two circles is always the ratio of their diameters.
Example 2.3.6 (MODELING A BASS FISHING DERBY). A sport shing club wishes to encourage its mem-
bership to release their sh immediately after catching them. The club also wishes to grant awards based on
the total weight of sh caught. It is suggested that each individual carry a small portable scale. Question:
How does someone shing determine the weight of a sh he/she has caught?
ANSWER. Skip. Read the textbook.
2.4 Automobile Gasoline Mileage.
Read the textbook.
2.5 Body Weight and Height, Strength and Agility.
Read the textbook.
Page 25 of 57
Page 26 of 57
Chapter 3
Modeling Fitting
When analyzing a collection of data points, it is suggested to consider the following three tasks.
1. Fitting a selected model type or types to the data.
2. Choosing the most appropriate model from competing types that have been tted. For example, we
may need to determine whether the besttting exponential model is a better model than the besttting
polynomial model.
3. Making predictions from the collected data.
In the rst two tasks, we do have a model or competing models explaining the observed behavior of the data.
It will be discussed in this chapter under the model tting. For the third case, since no model can explain the
observed behavior, so we will try to construct an empirical model based on the collected data, which will be
studied in the following chapter.
Topic I. Relationship Between Model Fitting and Interpolation.
Consider the gure 3.1 of the collected data.
Figure 3.1: Observations relating the variables y and x
There are mainly two ways to approximate the given data.
1. Based on the shape of the data, we make the assumption on the model and nd the better one. For
example, for the data in the gure 3.1, we assume a quadratic model and nd a best tting parabola
y = ax
2
+bx +c such as in the gure 3.2. In this way, we may explain the situation on which the data
lie. Usually this approach is theory driven.
2. We can nd a curve passing through all those points. Finding such a curve is called the spline interpo-
lation and it will be studied in the following chapter. In this way, we can capture the trend of the data
to predict in between the data points. Usually this approach is data driven. For the collected data, the
gure 3.3 shows the curve obtained by the spline interpolation.
27
Figure 3.2: Fitting a parabola y = ax
2
+bx +c to the
data points
Figure 3.3: Interpolating the data using a smooth poly-
nomial
Topic II. Sources of Error in the Modeling Process.
For purposes of easy reference, we classify errors under the following category scheme:
1. Formulation error: for instance, in the Example of Stopping Distance, we ignored the road friction for
the braking distance. Because of this ignorance, the model may be less effective.
2. Truncation error: for instance, when we compute the value of sinx, we may use only x x
3
/3! +x
5
/5
and because of the truncation of the other terms, the computation cannot be accurate.
3. Roundoff error: for instance, rigorously speaking, 0.3333333331/3 = 0. So if we use 0.333333333,
then we may confront an error.
4. Measurement error: one can understand this error as human error. When we measure an object in a
naked eye, it may not be accurate compared to the one measured by a machine.
3.1 Fitting Models to Data Graphically.
Topic I. Visual Model Fitting with the Original Data.
Figure 3.4: Minimizing the sum of the absolute deviation from the tted line
Suppose we want to t the model y = ax +b to the data shown in gure 3.4. All of them cannot be expected
to lie exactly along a single straight line. So there will be some vertical discrepancy between a few of the data
points and any particular line under consideration. These vertical discrepancies are called absolute deviation.
Page 28 of 57
Based on the deviation, we may think of two cases.
1. Minimizing the sum of the absolute deviation from the tted line and
2. Minimizing the largest absolute deviation form the tted line.
For the besttting line, we might try to achieve the rst one, minimizing the sum of deviation. For the
second one, minimizing the largest deviation, see the gure 3.5.
Figure 3.5: Minimizing the largest absolute deviation from the tted line
Topic II. Transforming the Data.
Suppose we have the following collected data.
Collected Data:
x 1 2 3 4
y 8.1 22.1 60.1 165
Transformed Data:
x 1 2 3 4
lny 2.1 3.1 4.1 5.1
Since the data points are suspected to follow the form y = ce
x
, by taking the logarithmic function on each
side, we deduce
lny = x +lnc,
which is a line on the (x)(lny)plane such that its slope is 1 and the (lny)intercept is (x, lny) = (0, lnc).
Page 29 of 57
3.2 Analytic Methods of Model Fitting.
Topic I. Chebyshev Approximation Criterion.
Goal: For the collection of m data points (x
i
, y
i
), i = 1, 2, . . . , m and a certain function y = f (x) (given as
in the example below), we want to minimize the largest deviation between the data and the function, i.e.,
minimize
Maximum of |y
i
f (x
i
)|, i = 1, 2, . . . , m. (3.2.1)
In other words, if f (x) =ax+b, then we want to nd a and b which minimizing the maximumvalue in (3.2.1).
This criterion is often called the Chebyshev approximation criterion.
Rewriting Problem: Let
r
i
= |y
i
f (x
i
)|, i = 1, 2, . . . , m.
Then, we have
r = Maximum of |y
i
f (x
i
)| = Maximum of r
i
, i = 1, 2, . . . , m
and so r is the largest deviation and we want to minimize this r. Since r is the maximum value of all |r
i
|s,
i = 1, 2, . . . , m, it is easy to see
|r
i
| r = r r
i
r = 0 r r
i
and 0 r +r
i
i = 1, 2, . . . , m.
Thus, the whole problem can be rewritten as follows:
Minimize r (i.e., nd the minimum value of r)
subject to
0 r r
i
and 0 r +r
i
i = 1, 2, . . . , m.
This kind of problem (nding a maximum/minimum value) is called a linear program or optimization prob-
lem.
Strategy: Computer implementation of an algorithm known as Simplex Method. It will be discussed in
Chapter 7 later. In the examples below, we will see how to minimize the largest deviation for a given function
via Mathematica.
Example 3.2.1. For the following data set, formulate the mathematical model that minimize the largest
deviation between the data and the line y = ax +b. (We use Mathematica to estimate a and b.)
x 1.0 2.3 3.7 4.2 6.1 7.0
y 3.6 3.0 3.2 5.1 5.3 6.8
ANSWER. Let r be the largest absolute deviation between the data and f (x) = ax+b. Then for the following
absolute deviations, |y
i
f (x
i
)|, we have
|3.6 f (1.0)| = |3.61.0ab| r
= r 3.61.0ab r = 0 r 1.0ab+3.6 and 0 r +1.0a+b3.6
|3.0 f (2.3)| = |3.02.3ab| r
= r 3.02.3ab r = 0 r 2.3ab+3.0 and 0 r +2.3a+b3.0
|3.2 f (3.7)| = |3.23.7ab|
= r 3.23.7ab r = 0 r 3.7ab+3.2 and 0 r +3.7a+b3.2
|5.1 f (4.2)| = |5.14.2ab|
= r 5.14.2ab r = 0 r 4.2ab+5.1 and 0 r +4.2a+b5.1
|5.3 f (6.1)| = |5.36.1ab|
Page 30 of 57
= r 5.36.1ab r = 0 r 6.1ab+5.3 and 0 r +6.1a+b5.3
|6.8 f (7.0)| = |6.87.0ab| r
= r 6.87.0ab r = 0 r 7.0ab+6.8 and 0 r +7.0a+b6.8
So the constraints are
0 r 1.0ab+3.6 0 r +1.0a+b3.6
0 r 2.3ab+3.0 0 r +2.3a+b3.0
0 r 3.7ab+3.2 0 r +3.7a+b3.2
0 r 4.2ab+5.1 0 r +4.2a+b5.1
0 r 6.1ab+5.3 0 r +6.1a+b5.3
0 r 7.0ab+6.8 0 r +7.0a+b6.8.
We want to nd a and b, i.e., f (x) = ax +b minimizing the largest absolute deviation r subject to the 12
constraints above.
By the computer, we obtain
f (x) = 0.533333x +2.14667
and the largest absolute deviation is r = 1.45333 which occurs at the last data (7.0, 6.8).
Topic II. Minimizing the Sum of the Absolute Deviations.
Consider a given data (x
i
, y
i
), i = 1, 2, . . . , m, and the model y = f (x). Let r
i
= |y
i
f (x
i
)|. Then the sum
of the absolute deviations is
m
i=1
r
i
. Let us consider the case of m = 2 so that we have only two absolute
deviations r
1
and r
2
. The sum r
1
+r
2
. If we plot the points (r
1
, 0) and (r
1
+r
2
, 0) on the line, we observe that
minimizing the sum r
1
+r
2
can be interpreted as minimizing the length of the line formed by adding together
the numbers r
i
.
To solve this optimization problem using the calculus, the differentiability of the absolute deviation in terms
of the parameter should be guaranteed so that its critical number can be found. However, since an absolute
function is not differentiable at the cusp, the calculus technique may not be applied to the sum of the absolute
deviation. Because of this drawback, the following technique is considered.
Topic III. LeastSquares Criterion.
Currently, the most frequently used curvetting criterion is the leastsquares criterion. Consider a given
data (x
i
, y
i
), i = 1, 2, . . . , m, and the model y = f (x). Let r
i
= |y
i
f (x
i
)|. Then the sum of the squares
of absolute deviations is
m
i=1
r
2
i
. Let us consider the case of m = 3. If we introduce a vector in the three
dimensional space,

r = r
1
, r
2
, r
3
, we observe that the sum of the squares of the absolute deviations is in
fact
m
i=1
r
2
i
= r
1
, r
2
, r
3
2
=

r
2
.
That is, we may interpret the leastsquares criterion as minimizing the magnitude of the vector whose coor-
dinates represent the absolute deviation between the observed and predicted values.
Topic IV. Relating the Criteria.
In the previous topics, we have discussed the geometric interpretations. Now let us compare the criteria
analytically.
Suppose m data, (x
i
, y
i
), i = 1, 2, . . . , m, are given and Chebyshev and leastsquares criterion give the model
y = f
C
(x) and y = f
L
(x), respectively. Let
c
i
= |y
i
f
C
(x
i
)|, c
max
= max{c
i
: i = 1, 2, . . . , m},
Page 31 of 57
d
i
= |y
i
f
L
(x
i
)|, d
max
= max{d
i
: i = 1, 2, . . . , m}.
Then we observe
1. c
max
d
max
.
Proof. Because of the parameters of the function y = f
C
(x) are determined so as to minimize the value
of c
max
, it is the minimal largest absolute deviation obtainable.
2. Letting
D =
m
i=1
d
2
i
m
,
we have D c
max
d
max
.
Proof. Since y = f
L
(x) gives the minimal sum of the squares obtainable, so we have
m
i=1
d
2
i
= d
2
1
+d
2
2
+ +d
2
m
c
2
1
+c
2
2
+ +c
2
m
c
2
max
+c
2
max
+ +c
2
max
= mc
2
max
m
i=1
d
2
i
m
c
2
max
= D =
m
i=1
d
2
i
m
c
max
.
With the observation 1, we have D c
max
d
max
.
Through an example, one can apply the criteria and compare the values D, c
max
and d
max
, which will be
studied in SECTION 3.4 CHOOSING A BEST MODEL3.4.
Page 32 of 57
3.3 Applying the LeastSquares Criterion.
In this section we study the leastsquares criterion to estimate the parameters for several types of curves. We
discuss the topics analytically rather than graphically.
Topic I. Fitting a Straight Line.
Suppose a model of the form y = ax +b is expected and m data points (x
i
, y
i
), i = 1, 2, . . . , m, are given.
The leastsquares criterion is minimizing the sum of the squares of the largest deviations, i.e., minimizing S
dened by
S =
m
i=1
(y
i
ax
i
b)
2
. (3.3.1)
Considering S as a function of two independent variables a and b, nding the minimum value of S is the
problem on the minimum value of S(a, b) in Calculus. From Calculus, we recall
1. A point (a
0
, b
0
) is called a critical point of S(a, b) if
(i) (a
0
, b
0
) is in the domain of S(a, b) and
(ii) either S
a
(a
0
, b
0
) = 0 or S
b
(a
0
, b
0
) = 0 and
(iii) one of both of S
a
(a
0
, b
0
) and S
b
(a
0
, b
0
) do not exist.
(Here S
a
means the partial derivative of S(a, b) with respect to a.)
2. If S(a, b) has a local extremumat (a, b) = (a
0
, b
0
), then (a, b) = (a
0
, b
0
) must be a critical point of S(a, b).
(However, the converse is not generally true.)
3. (SECOND DERIVATIVE TEST) Suppose that S(a, b) has continuous secondorder partial derivatives in
some open disk containing the point (a
0
, b
0
) and that S
a
(a
0
, b
0
) = 0 = S
b
(a
0
, b
0
). For the discriminant
D(a, b) for the point (a, b) dened by
D(a, b) = S
aa
(a, b)S
bb
(a, b) [S
ab
(a, b)]
2
,
(i) if D(a
0
, b
0
) > 0 and S
aa
(a
0
, b
0
) > 0, then S has a local minimum at (a
0
, b
0
),
(ii) if D(a
0
, b
0
) > 0 and S
aa
(a
0
, b
0
) < 0, then S has a local maximum at (a
0
, b
0
),
(iii) if D(a
0
, b
0
) < 0, then S has a saddle point at (a
0
, b
0
),
(iv) if D(a
0
, b
0
) = 0, then no conclusion can be drawn.
Let us nd the local minimum of S(a, b) dened in (3.3.1). To nd the critical point of S, we compute
0 =
S
a
= 2
m
i=1
(y
i
ax
i
b)x
i
= 2
m
i=1
(
x
i
y
i
ax
2
i
bx
i
)
= 2
[
m
i=1
(x
i
y
i
) a
m
i=1
x
2
i
b
m
i=1
x
i
]
,
0 =
S
b
= 2
m
i=1
(y
i
ax
i
b) = 2
[
m
i=1
y
i
a
m
i=1
x
i
b
m
i=1
1
]
.
For a simple computation, we introduce vectors x=x
1
, x
2
, . . . , x
m
and y =y
1
, y
2
, . . . , y
m
and i =1, 1, . . . , 1.
Then we observe
m
i=1
x
2
i
= x
2
1
+x
2
2
+ +x
2
m
= x
2
=x x,
m
i=1
x
i
= x
1
(1) +x
2
(1) + +x
m
(1) =x i,
m
i=1
y
i
= y
1
(1) +y
2
(1) + +y
m
(1) =y i,
m
i=1
(x
i
y
i
) = x
1
y
1
+x
2
y
2
+ +x
m
y
m
=x y,
Page 33 of 57
m = 1+1+ +1 = i i,
where x y is the dot product between vectors x and y. So those equations on S
a
and S
b
for the critical points
become
0 =x yax xbx i = ax x+bx i = x y (3.3.2)
0 =y i ax i bi i = ax i +bi i = y i. (3.3.3)
The resulting two equations (3.3.2) and(3.3.3) are called the normal equations. Simply,
x (yaxbi) = 0 and i (yaxbi) = 0.
(Be careful! In general, u v = 0 implies neither u =0 nor v =0. So we should not say yaxbi =0.)
To nd the critical points of S, we should solve the normal equations (3.3.2) and (3.3.3) for a and b.
(1) (3.3.2) (i i) (3.3.3) (x i) implies
a[(x x)(i i) (x i)(x i)] = (x y)(i i) (y i)(x i), i.e., a =
(x y)(i i) (x i)(y i)
(x x)(i i) (x i)
2
.
(2) (3.3.3) (x x) (3.3.2) (x i) implies
b[(x x)(i i) (x i)(x i)] = (x x)(y i) (x y)(x i), i.e., b =
(x x)(y i) (x y)(x i)
(x x)(i i) (x i)
2
.
Thus, S(a, b) dened in (3.3.1) has the critical point
(a
0
, b
0
) =
(
(x y)(i i) (x i)(y i)
(x x)(i i) (x i)
2
,
(x x)(y i) (x y)(x i)
(x x)(i i) (x i)
2
)
=
(
(x y)i
2
(x i)(y i)
x
2
i
2
(x i)
2
,
(y i)x
2
(x y)(x i)
x
2
i
2
(x i)
2
)
(3.3.4)
which is clearly in the domain of S(a, b), i.e., RR under the assumption (x x)(i i) (x i)
2
= 0. In fact,
the denominator (x x)(i i) (x i)
2
cannot be zero. See below (3.3.5).
Now we use the SECOND DERIVATIVE TEST to classify the local extremum at the found critical point
(a
0
, b
0
).
S
a
(a, b) = 2
m
i=1
(y
i
ax
i
b)x
i
= 2
m
i=1
(
x
i
y
i
ax
2
i
bx
i
)
, S
aa
(a, b) = 2
m
i=1
x
2
i
= 2x x
S
b
(a, b) = 2
m
i=1
(y
i
ax
i
b), S
bb
(a, b) = 2
m
i=1
1 = 2i i, S
ab
(a, b) = 2
m
i=1
x
i
= 2x i,
D(a, b) = S
aa
(a, b)S
bb
(a, b) S
2
ab
(a, b) = 4(x x)(i i) 4(x i)
2
= 4
[
(x x)(i i) (x i)
2
]
.
We observe D(a, b)/4 = (x x)(i i) (x i)
2
is the denominator of the critical points. So it cannot be zero.
See below (3.3.5).
As we can see, all of S
aa
and S
bb
and D(a, b) are constants, because they dont have the variables a and b.
Moreover, the CAUCHYSCHWARTZ INEQUALITY
in Calculus says
|u v| uv,
which implies
|u v|
2
u
2
v
2
, 0 u
2
v
2
|u v|
2
, 0 (u u)(v v) (u v)
2
, (3.3.5)
Personally, I do believe that it is one of THE MOST IMPORTANT inequalities in MATHEMATICS.

Page 34 of 57
where the equality holds for the cases: Case 1. u =0 or v =0 and Case 2. u and v are parallel, i.e., u = sv
for some scalar s.
Since x and i are not parallel, hence, we deduce D(a, b) = 4
[
(x x)(i i) (x i)
2
]
> 0 for all (a, b) and also
S
aa
(a, b) = 2x x = 2x
2
> 0 for all (a, b). Therefore, by the SECOND DERIVATIVE TEST, the function S
has the local minimum at the critical point (a
0
, b
0
) found in (3.3.4).
Before we nd the local minimum value of S, let us modify the function S using the vectors:
S(a, b) =
m
i=1
[y
i
ax
i
b]
2
=
m
i=1
[
y
2
i
+a
2
x
2
i
+b
2
2(ax
i
y
i
abx
i
+by
i
)
]
=
m
i=1
y
2
i
+a
2
m
i=1
x
2
i
+b
2
m
i=1
12
(
a
m
i=1
(x
i
y
i
) ab
m
i=1
x
i
+b
m
i=1
y
i
)
=y y+a
2
x x+b
2
i i 2(ax yabx i +by i)
= (ax+bi) (ax+bi) 2(ax+bi) y+y y
= (ax+bi y) (ax+bi y) = yaxbi
2
,
which is amazingly nice, because the function S dened by the sum is rewritten as the norm of the vector
having the same form as in the sum denition (3.3.1).
Therefore, the local minimum value of S is obtained by putting (a, b) = (a
0
, b
0
) into the result:
S(a
0
, b
0
) = ya
0
xb
0
i
2
. (3.3.6)
Example 3.3.1. For the given data,
x 1 5 8
y 1 10 6
estimate the parameters of the tting model y = ax +b by using the LeastSquares criterion.
ANSWER. We use the formulas deduced above. Let x = 1, 5, 8 and y = 1, 10, 6 and i = 1, 1, 1. Then the
objective function S is
S(a, b) = yaxbi
2
,
and by the result (3.3.4), it has the following critical point (a
0
, b
0
):
(a
0
, b
0
) =
(
(x y)i
2
(x i)(y i)
x
2
i
2
(x i)
2
,
(y i)x
2
(x y)(x i)
x
2
i
2
(x i)
2
)
=
(
99(3) 14(17)
90(3) 14
2
,
17(90) 99(14)
90(3) 14
2
)
=
(
59
74
,
72
37
)
= (0.797297, 1.94595).
Putting it into the formula (3.3.6) on the minimum value, we have the minimum value
S
(
59
74
,
72
37
)
=
y
59
74
x
72
37
i
2
=
1
59
74

72
37
, 10
5(59)
74

72
37
, 6
8(59)
74

72
37
2
=
1849
74
= 24.9865.
Thus, we conclude that the model y = 0.797297x +1.94595 gives the minimum value of the sum of the
squares of the absolute deviations and the minimum value is 24.9856.
Remark 3.3.2. 1. The minimum value 24.9856 is obtained analytically by the formula (3.3.6). When we use
the model y = 0.797297x +1.94595 and the data, we can make the following table.
Page 35 of 57
x 1 5 8
y 1 10 6
y
i
0.797297x
i
1.94595 1.74324 4.06757 2.32432
and the sum of the squares of all deviations becomes
3
i=1
(y
i
0.797297x
i
1.94595)
2
= (1.74324)
2
+(4.06757)
2
+(2.32432)
2
= 24.9856,
which is exactly same as the one obtained by the formula (3.3.6).
2. When we use Mathematica, we obtain the result as in the gure 3.6, which is exactly same as we did
above.
Figure 3.6: Mathematica Results
Topic II. Fitting a Power Curve.
To the given m data points (x
i
, y
i
), i = 1, 2, . . . , m, suppose we t the model y = ax
n
by the leastsquares
criterion, where n is xed and the parameter a will be determined. In this case, the sum S of the squares of
the absolute deviations becomes
S =
m
i=1
(y
i
ax
n
i
)
2
. (3.3.7)
By the similar argument as discussed in Topic I above, we introduce vectors x =
x
n
1
, x
n
2
, . . . , x
n
m
and y =
y
1
, y
2
, . . . , y
m
. Then S becomes the function of one variable a and its critical point is found by
0 =
S
a
= 2
m
i=1
(y
i
ax
n
i
)x
n
i
= 2
[
m
i=1
(x
n
i
y
i
) a
m
i=1
x
2n
i
]
= 2
[
x yax
2
]
= a =
x y
x
2
.
That is, S has the critical point
a
0
=
x y
x
2
.
Using the vectors, the function S in (3.3.7) turns to be
S =
m
i=1
(y
i
ax
n
i
)
2
=
m
i=1
(
y
2
i
+a
2
x
2n
i
2ax
n
i
y
i
)
=
m
i=1
y
2
i
+a
2
m
i=1
x
2n
i
2a
m
i=1
(x
n
i
y
i
) = y y+a
2
x x2x y = yax
2
.
Putting the critical point a = a
0
into the resulting function S(a), we have the minimum value
S(a
0
) = ya
0
x
2
.
Page 36 of 57
Example 3.3.3 (LEASTSQUARES WITH FIXED POWER n = 2). For the given data,
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
estimate the parameters of the tting model y = ax
2
by using the LeastSquares criterion.
ANSWER. Let x =
0.5
2
, 1.0
2
, 1.5
2
, 2.0
2
, 2.5
2
and y =0.7, 3.4, 7.2, 12.4, 20.1. Then the objective function
S is
S(a) = yax
2
,
and it has the following critical point a = a
0
:
a
0
=
x y
x
2
= 3.18693.
Putting it into the formula, we have the minimum value
S(3.18693) = y3.18693x
2
= 0.20954
Thus, we conclude that the model y = 3.18693x
2
gives the minimum value of the sum of the squares of the
absolute deviations and the minimum value is 0.20954.
Remark 3.3.4. Check with Mathematica: See the gure 3.7.
Figure 3.7: Mathematica Results
Topic III. Transformed LeastSquares Fit.
When the data can be approximated by a linear function y = ax +b (Topic I) or a curve with xed degree
y = ax
n
, where n is xed (Topic II), it is not difcult to estimate the parameters. In this topic, we consider a
simple case of the tting model y = ax
n
, where a and n are parameters to be determined.
If we apply the leastsquares criterion on y =ax
n
directly to the m data, we have to differentiate
m
i=1
(y
i
ax
n
i
)
2
with respect to a and n to get the critical point. However, it is not easy to get the critical point of such a
function. (One may try to nd the critical point!)
The strategy to estimate the parameters of the model y = ax
n
is using the independent substitution on x and y
or simply transformation on the data, Y = lny and X = lnx. Taking the natural logarithmic function on data
(x, y), we have the transformed data (X,Y) = (lnx, lny) and the tting model becomes
lny = ln(ax
n
) = lna+lnx
n
= lna+nlnx = Y = nX +A, (A = lna) (3.3.8)
which is linear in terms of X and Y and whose parameters n and lna can be estimated by the technique
discussed in Topic I above. The reason why we transform the data and the model by the natural logarithmic
function mainly lies on the properties of the logarithmic function: explicitly, we can pull down the power of
Page 37 of 57
an exponential function so that the exponential model y =ax
n
becomes a linear one Y =nX +lna. Moreover,
since a logarithmic function is a onetoone correspondence (or a bijection) and a conformal mapping (i.e.,
anglepreserving mapping), the transformation via the function does not change the critical properties (such
as the absolute deviations) inherited in the original data.
Now let us estimate a and n in the exponential model y =ax
n
. We apply the leastsquares criterion developed
in Topic I to the transformed model (3.3.8). Then the objective function S on the transformed data (X,Y) =
(lnx, lny) has the independent variables n and A, i.e.,
S(n, A) =
m
i=1
(lny
i
nlnx
i
lna)
2
=
m
i=1
(Y
i
nX
i
A)
2
,
where X
i
= lnx
i
and Y
i
= lny
i
. As we did in Topic I, let us introduce vectors
X = X
1
, X
2
, . . . , X
m
= lnx
1
, lnx
2
, . . . , lnx
m
,
Y = Y
1
,Y
2
, . . . ,Y
m
= lny
1
, lny
2
, . . . , lny
m
, i = 1, 1, . . . , 1.
Then the objective function S(n, A) becomes
S(n, A) = YnXAi
2
.
(See the argument preceding the result (3.3.6).) It has the normal equations,
nX X+AX i =X Y, and nX i +Ai i =Y i.
(See the equations (3.3.2) and (3.3.3).) Solving the equations for n and A, we nd the critical point (n, A) =
(n
0
, A
0
),
(n
0
, A
0
) =
(
(X Y)i
2
(X i)(Y i)
X
2
i
2
(X i)
2
,
(Y i)X
2
(X Y)(X i)
X
2
i
2
(X i)
2
)
(3.3.9)
(See the result (3.3.4).) Thus S(n, A) has the minimum value
S(n
0
, A
0
) = Yn
0
XA
0
i
2
. (3.3.10)
Since now we know the critical point (n
0
, A
0
), those parameters of the exponential model y =ax
n
are obtained
by
n = n
0
, and a = e
A
0
, i.e., y = e
A
0
x
n
0
.
Example 3.3.5 (TRANSFORMED LEASTSQUARES WITH UNFIXED POWER n (SAME DATA AS IN EXAM-
PLE 3.3.3)). For the given data,
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
estimate the parameters of the tting model y =ax
n
by using the Transformed LeastSquares criterion. (Here
we have two parameters a and n to estimate.)
ANSWER. Since the model is an exponential function, we transform the data and the model by the logarith-
mic function.
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
X = lnx 0.693147 0 0.405465 0.693147 0.916291
Y = lny 0.356675 1.22378 1.97408 2.5177 3.00072
Page 38 of 57
Then the model becomes
lny = nlnx +lna = Y = nX +A,
by setting X = lnx and Y = lny and A = lna. Using the results deduced above, the objective function of the
transformed data becomes
S(n, A) = YnXAi
2
,
where X, Y, and i are vectors as in the argument above. By the formulas (3.3.9) and (3.3.10), the function S
has the critical point (n
0
, A
0
) and the minimum value S(n
0
, A
0
):
(n
0
, A
0
) =
(
(X Y)i
2
(X i)(Y i)
X
2
i
2
(X i)
2
,
(Y i)X
2
(X Y)(X i)
X
2
i
2
(X i)
2
)
= (2.06281, 1.12661),
S(n
0
, A
0
) = Yn
0
XA
0
i
2
= Y2.06281X1.12661i
2
= 0.014179.
Therefore, we deduce the tting model y = e
A
0
x
n
0
= e
1.12661
x
2.06281
= 3.08519x
2.06281
with the minimum
value 0.014179 of the sum of the squares of the absolute deviations.
Example 3.3.6 (TRANSFORMED LEASTSQUARES WITH FIXED POWER n = 2 (SAME DATA AS IN EX-
AMPLE 3.3.3)). For the given data,
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
estimate the parameters of the tting model y =ax
2
by using the Transformed LeastSquares criterion. (Here
we have two parameters a and n to estimate.)
ANSWER. We follow the exactly same argument in the previous example. The model y = ax
2
becomes
lny = 2lnx +lna = Y = 2X +A.
The objective function of the transformed data becomes
S(A) = Y2XAi
2
,
The function S(A) has the critical point A = A
0
and the minimum value S(A
0
):
A
0
=
Y i 2(X i)
i
2
= 1.14322,
S(A
0
) = Y2XA
0
i
2
= Y2X1.14322i
2
= 0.0205521.
Therefore, we deduce the tting model y =e
A
0
x
2
=e
1.14322
x
2
=3.13684x
2
with the minimumvalue 0.0205521
of the sum of the squares of the absolute deviations.
Remark 3.3.7. For the data given in Example 3.3.3, we have tested three models with different criteria:
1. y = ax
2
with LeastSquares (Example 3.3.3): We have deduced y = 3.18693x
2
with the minimum value
0.20954.
2. y = ax
n
with Transformed LeastSquares (Example 3.3.5): We have deduced y = 3.08519x
2.06281
with the
minimum value 0.014179.
3. y = ax
2
with Transformed LeastSquares (Example 3.3.6): We have deduced y = 3.13684x
2
with the
minimum value 0.0205521.
Let us make a table on the deviations and compare them.
Page 39 of 57
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
1. y
i
3.18693x
2
i
0.0967325 0.21307 0.0294075 0.34772 0.181688
2. y
i
3.08519x
2.06281
i
0.0384383 0.31481 0.0792666 0.489902 0.32474
3. y
i
3.13684x
2
i
0.08421 0.26316 0.14211 0.14736 0.49475
Model
m
i=1
(y
i
y(x
i
))
2
max|y
i
y(x
i
)|
1. y
i
3.18693x
2
i
0.20954 0.34772
2. y
i
3.08519x
2.06281
i
0.452326 0.489902
3. y
i
3.13684x
2
i
0.363032 0.49475
In the case of the given data, we observe the rst model y = 3.18693x
2
makes the smallest sum of the
absolute deviations and also smallest sum of the squares of the deviations. So we are allowed to say the
best one between those three models is the rst one. We may guess this result before going through all
the computations in those three examples. Simply speaking, when we use the transformation, the data get
damaged and the transformed model also loose some information. Due to these loss, the transformed model
may not be better than the one obtained from the original data. However, such as in the exponential model
y = ax
n
with unxed power n, we need to transform the data and the model anyway.
We end this section by copying one paragraph in the textbook. The preceding examples illustrate two
facts.
1. If an equation can be transformed to yield an equation of a straight line (Topic I) in the transformed
variable, equations (3.3.2) and (3.3.3) can be used directly to solve for the slope and intercept of the
transformed graph.
2. The leastsquares best t to the transformed equations does not coincide with the leastsquares best
t of the original equations. The reason for this discrepancy is that the resulting optimization problems
are different. In the case of the original problem, we are nding the curve that minimizes the sum of the
squares of the deviations using the original data, whereas in the case of the transformed problem we are
minimizing the sum of the squares of the deviations using the transformed variables.
Page 40 of 57
3.4 Choosing a Best Model.
We start with the example from the previous section. The data is given and approximate by the model y =ax
2
.
x 0.5 1.0 1.5 2.0 2.5
y 0.7 3.4 7.2 12.4 20.1
Table 3.1: Collected Data
LeastSquares Criterion: We have y = 3.1869x
2
.
Transformed LeastSquares Fit: We have y = 3.1368x
2
.
Chebyshev Criterion: We have y = 3.17073x
2
.
We can summarize the deviations by each method as follows:
x
i
y
i
y
i
3.1869x
2
i

y
i
3.1368x
2
i

y
i
3.17073x
2
i

0.5 0.7 0.0967 0.0842 0.0927
1.0 3.4 0.2131 0.2632 0.2293
1.5 7.2 0.029475 0.1422 0.0659
2.0 12.4 0.3476 0.1472 0.2829
2.5 20.1 0.181875 0.4950 0.28293
Table 3.2: Summary of deviations for each model y = ax
2
Criterion Model [y
i
y(x
i
)]
2
max|y
i
y(x
i
)|
LeastSquares y = 3.1869x
2
0.2095 0.3476
Transformed LeastSquares y = 3.1368x
2
0.3633 0.4950
Chebyshev y = 3.17073x
2
0.2256 0.28293
Table 3.3: Summary of results for the three models
Which model is best? The model obtained by the leastsquares criterion is better than the one by Chebyshev
in the sense that the former gives smaller sum of the squares of the deviations. However, under the purpose of
minimizing the largest absolute deviation, the model obtained by Chebyshev is better. Thus, the best criterion
depends on the case, judged by the purpose of modeling and so on.
Even if the sum of the squares of deviations is small, we need to be careful before we jump into making a
decision on the trend of the data. Its because we may be misled into a wrong prediction. See the gure 3.15
on page 121 in the textbook. The model y = x has the same sum of squared deviations. However, as we
can see, they give signicantly different prediction on the trend of the data. In order to prevent this kind of
misleading, we need to plot the model and the data together and compare them.
Example 3.4.1. 1. Find a model using the leastsquares criterion either on the data or on the transformed
data (as appropriate).
2. Compare your results with the graphical ts obtained in the problem set 3.1 by computing the devia-
tions, maximum absolute deviations and the sum of the squared deviations for each model.
3. Find a bound on c
max
if the model was t using the leastsquares criterion.
Problem 3 in problem set 3.1. In the following data, x is the diameter of a ponderosa pine in inches measured
at breast height and y is a measure of volumenumber of board feet divided by 10.
(1) Test the model y = ax
b
by plotting the transformed data.
Page 41 of 57
(2) If the model seems reasonable, estimate the parameters a and b of the model graphically.
Figure 3.8: Data and model y = 0.00282062x
3.11139
by transforming data
ANSWER TO QUESTIONS (1) AND (2) IN SECTION 3.1. Taking the natural logarithmic function on both
sides of y = ax
b
, we have
lny = blnx +lna,
of which graph on the lny versus lnx is a line with the slope b and the (lny)intercept (0, lna). A simple
computation gives the transformed data (lnx, lny). Plotting them, we observe a line. Using two points
(ln17, ln19) and (ln41, ln294), we have the slope b = 3.11139 and the (lny)intercept (0, 5.8708), i.e.,
a = 0.00282062 and so we deduce
y = 0.00282062x
3.11139
.
See the gure 3.8.
ANSWER TO QUESTIONS 1, 2, AND 3 IN SECTION 3.4. For the model y = ax
b
, we transform the data and
apply the transformed leastsquares t:
lny = blnx +lna,
which gives the normal equations with m = 15,
b =
m
m
i=1
lnx
i
lny
i
m
i=1
lnx
i
m
i=1
lny
i
m
m
i=1
(lnx
i
)
2
(
m
i=1
lnx
i
)
2
lna =

m
i=1
(lnx
i
)
2
m
i=1
lny
i
m
i=1
lnx
i
m
i=1
lnx
i
lny
i
m
m
i=1
(lnx
i
)
2
(
m
i=1
lnx
i
)
2
.
By the equations, we deduce b = 3.09187 and a = 0.00320603 and
y = 0.00320603x
3.09187
.
x 17 19 20 22 23 25 28 31 32 33 36 37 38 39 41
y 19 25 32 51 57 71 113 141 123 187 192 205 252 259 294
Page 42 of 57
Figure 3.9: Data and model y = 0.00320603x
3.09187
by transformed leastsquares t
See the gure 3.9.
As we can observe from the table 3.5, in this problem, the transformed leastsquares model gives a better
approximation than the one obtained by transforming the data and the linearity.
Page 43 of 57
x
i
1
7
1
9
2
0
2
2
2
3
2
5
2
8
y
i
1
9
2
5
3
2
5
1
5
7
7
1
1
1
3
y
i
0
.
0
0
2
8
x
3
.
1
1
1
4
i
0
.
0
0
0
1
1
.
8
5
6
3
0
.
4
9
6
7
8
.
6
2
1
5
8
.
3
3
5
6
7
.
9
2
1
5
2
3
.
2
5
3
5
y
i
0
.
0
0
3
2
x
3
.
0
9
1
9
i
1
.
4
3
4
1
3
.
8
2
0
9
1
.
7
7
4
1
5
.
6
5
1
4
4
.
9
7
0
1
3
.
6
6
8
8
1
7
.
4
1
4
5
x
i
3
1
3
2
3
3
3
6
3
7
3
8
3
9
4
1
y
i
1
4
1
1
2
3
1
8
7
1
9
2
2
0
5
2
5
2
2
5
9
2
9
4
y
i
0
.
0
0
2
8
x
3
.
1
1
1
4
i
1
7
.
8
1
6
5
1
2
.
9
7
3
2
3
7
.
3
6
4
8
4
.
1
5
9
2
8
.
6
1
5
0
1
9
.
9
0
4
1
7
.
3
6
7
3
0
.
0
0
2
2
y
i
0
.
0
0
3
2
x
3
.
0
9
1
9
i
1
0
.
0
6
2
5
2
1
.
4
4
2
7
2
8
.
1
3
9
7
1
5
.
8
9
9
1
2
1
.
2
7
8
6
6
.
2
7
2
9
7
.
2
7
6
3
1
6
.
8
0
3
3
T
a
b
l
e
3
.
4
:
S
u
m
m
a
r
y
o
f
t
h
e
d
e
v
i
a
t
i
o
n
s
f
o
r
e
a
c
h
m
o
d
e
l
C
r
i
t
e
r
i
o
n
M
o
d
e
l
[
y
i
y
(
x
i
)
]
2
m
a
x
|
y
i
y
(
x
i
)
|
T
r
a
n
s
f
o
r
m
e
d
L
i
n
e
a
r
i
t
y
y
=
0
.
0
0
2
8
2
0
6
2
x
3
.
1
1
1
3
9
3
1
7
4
.
8
1
3
7
.
3
6
4
8
T
r
a
n
s
f
o
r
m
e
d
L
e
a
s
t
S
q
u
a
r
e
s
y
=
0
.
0
0
3
2
0
6
0
3
x
3
.
0
9
1
8
7
2
8
2
6
.
2
6
2
8
.
1
3
9
7
T
a
b
l
e
3
.
5
:
S
u
m
m
a
r
y
o
f
r
e
s
u
l
t
s
f
o
r
t
h
e
t
w
o
m
o
d
e
l
s
Page 44 of 57
Chapter 4
Chapter 7 Discrete Optimization Modeling
Section 7.4 Linear Programming III: Simplex Method.
PROBLEM: Use the simplex method
to solve the following optimization problem.

Maximize 3x
1
+x
2
subject to 2x
1
+x
2
6 (Original Pr)
x
1
+3x
2
9
x
1
, x
2
0.
ANSWER. We introduce variables y
1
and y
2
and convert the problem as follows:
Maximize 3x
1
+x
2
subject to 2x
1
+x
2
+y
1
= 6
x
1
+3x
2
+y
2
= 9
x
1
, x
2
, y
1
, y
2
0,
which is called canonical slack maximization and the variables y
1
and y
2
are called slack
variables.
Since x
1
0 and x
2
0 are required by the condition, so the objective function should satisfy
3x
1
x
2
0
and we introduce another slack variable z so that
3x
1
x
2
+z = 0 and z 0.
Hence, we can rewrite the original problem (Original Pr) as follows:
2x
1
+x
2
+y
1
= 6 (4.0.1)
x
1
+3x
2
+y
2
= 9 (4.0.2)
3x
1
x
2
+z = 0 (4.0.3)
x
1
, x
2
, y
1
, y
2
, z 0,
and we will nd the largest value of z and the pair (x
1
, x
2
) which gives the largest value of z.
By collecting the coefcients, we record the problem into the socalled Tucker tableau
.
In the objective function constraint (4.0.3), we compare the absolute value of the coefcients. Since the
coefcient of x
1
has the largest absolute value 3, so we choose x
1
as the entering variable.
Compute the ratio of the RHS divided by the column labeled x
1
to determine the minimum positive ratio.
Choose y
1
corresponding to the minimum positive ratio 3 as the exiting variable.
Pivot
Divide the row containing the exiting variable (the rst row in this case) by the coefcient of the
Simplex algorithm was developed in the 1940s by George B. Dantzig. We will employ certain renements in Dantzigs
original technique developed in the 1960s by A.W. Tucker.
slack: wanting in activity; lacking in completeness, nish, or perfection
Tableau: picture, painting, representation, illustration, image
Pivot: a shaft or pin on which something turns

45
x
1
x
2
y
1
y
2
z RHS
2 1 1 0 0 6
1 3 0 1 0 9
3 1 0 0 1 0
Table 4.1: Original Tucker Tableau
x
1
x
2
y
1
y
2
z RHS
2 1 1 0 0 6
1 3 0 1 0 9
3 1 0 0 1 0
Table 4.2: Entering Variable x
1
x
1
x
2
y
1
y
2
z RHS Ratio
2 1 1 0 0 6 3 (= 6/2)
1 3 0 1 0 9 9 (= 9/1)
3 1 0 0 1 0
1
and Exiting Variable y
1
entering variable in that row (the coefcient of x
1
in this case), giving a coefcient of 1 for the entering
variable in this row. Then eliminate the entering variable x
1
from the remaining rows (which do not contain
the exiting variable y
1
and have a zero coefcient for it).
x
1
x
2
y
1
y
2
z RHS
1 1/2 1/2 0 0 3
1 3 0 1 0 9
3 1 0 0 1 0
x
1
x
2
y
1
y
2
z RHS
1 1/2 1/2 0 0 3
11 31/2 01/2 1 0 93
3+3 1+3/2 0+3/2 0 1 0+9
Simply,
Since there are no negative coefcients in the bottom row, thus x
1
= 3 and y
2
= 6 (i.e., x
2
= 0 by (4.0.2))
gives the extreme point (x
1
, x
2
) = (3, 0) at which the optimal objective function value z = 9.
Page 46 of 57
x
1
x
2
y
1
y
2
z RHS
1 1/2 1/2 0 0 3 (= x
1
)
0 5/2 1/2 1 0 6 (= y
2
)
0 1/2 3/2 0 1 9 (= z)
Table 4.4: Tableau giving Extreme Point and Optimal Value
Example 4.0.2. Solve Carpenters problem.
Maximize 25x
1
+30x
2
subject to 20x
1
+30x
2
690
5x
1
+4x
2
120
x
1
, x
2
0.
ANSWER. By introducing slack variables y
1
, y
2
and z, we convert the problem as follows:
20x
1
+30x
2
+y
1
= 690 (4.0.4)
5x
1
+4x
2
+y
2
= 120 (4.0.5)
25x
1
30x
2
+z = 0 (4.0.6)
x
1
, x
2
, y
1
, y
2
, z 0.
By collecting the coefcients, we record the problem into the Tucker tableau:
x
1
x
2
y
1
y
2
z RHS
20 30 1 0 0 690
5 4 0 1 0 120
25 30 0 0 1 0
coefcient of x
2
2
x
1
x
2
y
1
y
2
z RHS
20 30 1 0 0 690
5 4 0 1 0 120
25 30 0 0 1 0
2
2
Page 47 of 57
x
1
x
2
y
1
y
2
z RHS Ratio
20 30 1 0 0 690 23 (= 690/30)
5 4 0 1 0 0 30 (= 120/4)
25 30 0 0 1 0
2
1
Choose y
1
corresponding to the minimum positive ratio 23 as the exiting variable.
Pivot Divide the row containing the exiting variable (the rst row in this case) by the coefcient of the
2
2
1
x
1
x
2
y
1
y
2
z RHS
2/3 1 1/30 0 0 690/30
5 4 0 1 0 120
25 30 0 0 1 0
x
1
x
2
y
1
y
2
z RHS
2/3 1 1/30 0 0 690/30
58/3 44 04/30 1 0 1204(690/30)
25+30(2/3) 30+30 0+30/30 0 1 0+30(690/30)
Simply,
x
1
x
2
y
1
y
2
z RHS
2/3 1 1/30 0 0 23
7/3 0 2/15 1 0 28
5 0 1 0 1 690
Since we have a negative coefcient in the bottom row, we repeat the work by comparing the coefcients of
the absolute values in the bottom row.
Simply,
Page 48 of 57
x
1
x
2
y
1
y
2
z RHS
2/3 1 1/30 0 0 23
7/3 0 2/15 1 0 28
5 0 1 0 1 690
1
2
x
1
x
2
y
1
y
2
z RHS
2/3 1 1/30 0 0 23
1 0 2/35 = (2/15)(3/7) 3/7 0 12 = 28(3/7)
5 0 1 0 1 690
x
1
x
2
y
1
y
2
z RHS
2/32/3 1 1/30(2/3)(2/35) 0(2/3)(3/7) 0 23(2/3)12
1 0 2/35 3/7 0 12
5+5 0 1+5(2/35) 0+5(3/7) 1 690+5(12)
x
1
x
2
y
1
y
2
z RHS
0 1 1/14 2/7 0 15 (= x
2
)
1 0 2/35 3/7 0 12 (= x
1
)
0 0 5/7 15/7 1 750 (= z)
Since there are no negative coefcients in the bottom row, thus by choosing y
1
= 0 = y
2
, we get x
1
= 12
and x
2
= 15 which gives the extreme point (x
1
, x
2
) = (12, 15) and the optimal objective function value z =
750.
Example 4.0.3. An electrical rm manufactures circuit boards in two congurations, say conguration #1
and conguration #2. Each circuit board in conguration #1 requires 1A component, 2B component, and 2C
component; each circuit board in conguration #2 requires 2A component, 2B component, and 1C compo-
nent. The rm has 20A components, 30B components and 25C components available. If the prot realized
upon sale is $200 per circuit board in conguration #1 and $150 per circuit board in conguration #2, how
many circuit boards of each conguration should the electrical rm manufacture so as to maximize prots?
ANSWER. Let x
1
and x
2
be respectively the number of circuit boards in conguration #1 and #2. Then the
mathematical formulation of the problem is
Maximize 200x
1
+150x
2
subject to x
1
+2x
2
20
2x
1
+2x
2
30
2x
1
+x
2
25.
Using the slack variables y
1
, y
2
, y
3
and z, we can rewrite the original problem as follows:
x
1
+2x
2
+y
1
= 20 (4.0.7)
Page 49 of 57
2x
1
+2x
2
+y
2
= 30 (4.0.8)
2x
1
+x
2
+y
3
= 25 (4.0.9)
200x
1
150x
2
+z = 0 (4.0.10)
x
1
, x
2
, y
1
, y
2
, y
3
, z 0,
and we will nd the largest value of z and the pair (x
1
, x
2
) which gives the largest value of z.
By collecting the coefcients, we record the problem into the Tucker tableau.
x
1
x
2
y
1
y
2
y
3
z RHS
1 2 1 0 0 0 20
2 2 0 1 0 0 30
2 1 0 0 1 0 25
200 150 0 0 0 1 0
coefcient of x
1
1
x
1
x
2
y
1
y
2
y
3
z RHS
1 2 1 0 0 0 20
2 2 0 1 0 0 30
2 1 0 0 1 0 25
200 150 0 0 0 1 0
1
1
x
1
x
2
y
1
y
2
y
3
z RHS
1 2 1 0 0 0 20 (= 20/1)
2 2 0 1 0 0 15 (= 30/2)
2 1 0 0 1 0 25/2 (= 25/2)
200 150 0 0 0 1 0
1
3
Choose y
3
corresponding to the minimum positive ratio 25/2 = 12.5 as the exiting variable.
Pivot Divide the row containing the exiting variable (the third row in this case) by the coefcient of the
1
1
3
Page 50 of 57
x
1
x
2
y
1
y
2
y
3
z RHS
1 2 1 0 0 0 20
2 2 0 1 0 0 30
1 1/2 0 0 1/2 0 25/2
200 150 0 0 0 1 0
x
1
x
2
y
1
y
2
y
3
z RHS
11 21/2 1 0 01/2 0 2025/2
22 21 0 1 01 0 3025
1 1/2 0 0 1/2 0 25/2
200+200 150+100 0 0 100 1 0+(25/2)200
Simply,
x
1
x
2
y
1
y
2
y
3
z RHS
0 3/2 1 0 1/2 0 15/2
0 1 0 1 1 0 5
1 1/2 0 0 1/2 0 25/2
0 50 0 0 100 1 2500
Since there are no negative coefcients in the bottom row, thus x
1
= 3 and y
2
= 6 (i.e., x
2
= 0 by (4.0.2))
gives the extreme point (x
1
, x
2
) = (3, 0) at which the optimal objective function value z = 9.
Page 51 of 57
Page 52 of 57
Chapter 5
Chapter 8 Dimensional Analysis
5.1 Introduction.
Read the textbook. Studied in class but the lecture note has not been typed.
5.2 Dimensions as Product.
53
Page 54 of 57
Chapter 6
Chapter 10 Modeling with a Differential Equation
10.5 Numerical Approximation Method.
55
Page 56 of 57
Chapter 7
Chapter 11 Modeling with Systems of Differential Equations
11.1 Graphical Solutions of Autonomous Systems of FirstOrder Differential Equations.
57

Mathematical Modelling

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Mathematical Modelling

Caricato da

Copyright:

Formati disponibili

Mathematical Modeling

Culture). By an experiment measuring the growth of a yeast culture, we

Figure 1.5: Red #: Experimental Result, Blue : Model Result

Sewage: refuse liquids or waste matter usually carried off by sewers

, where each one has length l, l

are geometrically similar so that there is a onetoone

, and other points and the ratio of the distances

Figure 2.5: Two geometrically similar objects X and X

for some constant k > 0.

in the boxes X and X

, respectively, we observe that the angles are

. It is easy to understand that the boxes X and X

above, the volumes of X and X

between the dimensions of the boxes. Since l/l

Remark 2.3.3 (ASIDE). From S

). Droplets falling in a motionless air can be modeled by the differ-

From the book, Concepts of Mathematical Modeling, written by Walter J. Myer

Personally, I do believe that it is one of THE MOST IMPORTANT inequalities in MATHEMATICS.

to solve the following optimization problem.

slack: wanting in activity; lacking in completeness, nish, or perfection

Tableau: picture, painting, representation, illustration, image

Pivot: a shaft or pin on which something turns

Potrebbero piacerti anche