Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Dmitriy Leykekhman
Fall 2008
Goals
Orthogonal matrices. QR-decomposition. Solving LLS with QR-decomposition.
D. Leykekhman - MATH 3795 Introduction to Computational Mathematics Linear Least Squares 1
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one.
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one. If Q Rnn is orthogonal, then QT Q = I implies that Q1 = QT .
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one. If Q Rnn is orthogonal, then QT Q = I implies that Q1 = QT . If Q Rnn is an orthogonal matrix, then QT is an orthogonal matrix.
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one. If Q Rnn is orthogonal, then QT Q = I implies that Q1 = QT . If Q Rnn is an orthogonal matrix, then QT is an orthogonal matrix. If Q1 , Q2 Rnn are orthogonal matrices, then Q1 Q2 is an orthogonal matrix.
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one. If Q Rnn is orthogonal, then QT Q = I implies that Q1 = QT . If Q Rnn is an orthogonal matrix, then QT is an orthogonal matrix. If Q1 , Q2 Rnn are orthogonal matrices, then Q1 Q2 is an orthogonal matrix. If Q Rnn is an orthogonal matrix, then (Qx)T (Qy) = xT y x, y Rn
i.e. the angle between Qx and Qy is equal to the angle between x and y
Orthogonal matrices.
A matrix Q Rmn is called orthogonal if QT Q = In , i.e., if its columns are orthogonal and have 2-norm one. If Q Rnn is orthogonal, then QT Q = I implies that Q1 = QT . If Q Rnn is an orthogonal matrix, then QT is an orthogonal matrix. If Q1 , Q2 Rnn are orthogonal matrices, then Q1 Q2 is an orthogonal matrix. If Q Rnn is an orthogonal matrix, then (Qx)T (Qy) = xT y x, y Rn
i.e. the angle between Qx and Qy is equal to the angle between x and y As a result Qx
2
= x
Matrix Norms
Example
In two dimensions a rotation matrix Q= cos sin sin cos
is orthogonal matrix. This fact can easily be checked QT Q = = cos sin sin cos cos sin sin cos = 1 0 0 1 .
cos2 + sin2 0
0 cos2 + sin2
QR-Decomposition.
Let m n. For each A Rmn there exists a permutation matrix P Rmnn , an orthogonal matrix Q Rmm , and an upper triangular matrix R Rnn such that AP = Q R 0 } } n mn QR-decomposition.
QR-Decomposition.
Let m n. For each A Rmn there exists a permutation matrix P Rmnn , an orthogonal matrix Q Rmm , and an upper triangular matrix R Rnn such that AP = Q R 0 } } n mn QR-decomposition.
The QR decomposition of A can be computed using the Matlab command [Q, R, P ] = qr(A).
QR-Decomposition.
Let m n. For each A Rmn there exists a permutation matrix P Rmnn , an orthogonal matrix Q Rmm , and an upper triangular matrix R Rnn such that AP = Q R 0 } } n mn QR-decomposition.
The QR decomposition of A can be computed using the Matlab command [Q, R, P ] = qr(A). We will not go into the details of how Q, P, R are computed. If you interested check Chapter 5 of the book Gene Golub and Charles Van Loan, Matrix Computations
where R Rnn is upper triangular matrix. Since A has full rank n the matrix R also has rank n and, therefore, is nonsingular.
where R Rnn is upper triangular matrix. Since A has full rank n the matrix R also has rank n and, therefore, is nonsingular. Moreover, since Q is orthogonal it obeys QQT = I. Hence QT y
2
= y
y Rm .
= QT (Ax b)
2 2 2 2 T
= QT (AP P T x b) R 0
= (QT AP )P T x Q b = P T x QT b
2 2 2 2
c d
} }
n mn
R 0
y
2 2
c d
=
2
Ry c d
2 2
= Ry c Thus, min Ax b
x 2 2
+ d 2. 2 min Ry c
y 2 2
+ d
2 2
Thus, min Ax b
x 2 2
min Ry c
y
2 2
+ d
2 2
Thus, min Ax b
x 2 2
min Ry c
y
2 2
+ d
2 2
x = P y.
c.
10
= QT (AP P T x b) = R1 R2 0
2 2 2
P T x QT b
2
11
r nr mn
y1 y2
} }
r nr
Ax b
R1 y1 + R2 y2 c1 c2 = d = R1 y1 + R2 y2 c1
2 2
2 2 2
+ c2
+ d 2. 2
12
is equivalent to
2 2
+ c2
+ d 2, 2
13
is equivalent to
2 2
+ c2
+ d 2, 2
where R1 Rrr is nonsingular. Solution is 1 y1 = R1 (c1 R2 y2 ) for any y2 Rnr . Since y = P T x and P T P = I, x = Py = P
1 R1 (c1 R2 y2 ) y2
13
is equivalent to
2 2
+ c2
+ d 2, 2
where R1 Rrr is nonsingular. Solution is 1 y1 = R1 (c1 R2 y2 ) for any y2 Rnr . Since y = P T x and P T P = I, x = Py = P
1 R1 (c1 R2 y2 ) y2
We have innitely many solutions since y2 is arbitrary. Which one to choose? If we use Matlab x = A\b, then Matlab computes the one with y2 = 0 x=P
1 R1 c1 0
13
14
15
are given by x = Py = P
1 R1 (c1 R2 y2 ) y2
where y2 Rnr is arbitrary. Minimum norm solution: Of all solutions, pick the one with the smallest 2-norm. This leads to min P
y2 1 R1 (c1 R2 y2 ) y2 2 2
15
=
2 2
=
2 2
=
2 2 2
y2
1 R1 c1 0
which is another linear least squares problem with unknown y2 . This problem is n (n r) and it has full rank. It can be solved using the techniques discussed earlier.
16
18