CSC 5800:

Intelligent Systems:
Algorithms and Tools

Lecture 2:
Linear algebra Review

Acknowledgement: This lecture is partially based on the slides from

http://chortle.ccsu.edu/VectorLessons/index.html

Some Terminology
• Point — a geometric object; a location in 3D (or 2D) space.

• Vector — a geometric object that has properties of direction and

length, but not location.

• Column matrix — an ordered list of numbers arranged into a

column.

• Row matrix — an ordered list of numbers arranged into a row.

• Element — one of the numbers that makes up a column or row

matrix.

• Dimension — the number of elements in a column or row matrix.

1
 Matrices
• Matrix is a tabular representation of a set of numbers
as a collection of rows and columns

• “m by n matrix” means it has m rows and n columns

• Transpose of matrix is obtained by interchanging

rows by columns
– (AT)T =A

• A(2, :) means all elements of row 2

• A(:,3) means all elements of column 3

Matrix Addition
• Two matrices should be of same dimension

• Commutative (A+B = B+A)

• Associative ((A+B)+C=A+(B+C))

• Zero matrix is the identity element A+0=A

• Additive Inverse of A is –A

2
 Scalar Multiplication
• The product of a scalar α and matrix A, is the matrix B= αA

• Associativity – α(βA) = (αβ)A

• Distributivity (Addition over Multiplication)– (α+β)A= αA+βA

• Distributivity (Multiplication over Addition)–α(A+B)= αA+ αB

• Scalar Identity - if α=1, then for any A, αA=A.

Matrix Multiplication
• To multiply two matrices
– No. of columns in the First matrix must the same as the No. of
rows in the Second Matrix

• A (m X n) B (n X k) = C (m X k)
• Generally, it is not Commutative AB<>BA

• Associativity, (AB)C=A(BC)

• Distributivity,
– A(B+C)=AB+AC
– (B+C)A=BA+CA

• Identity Matrix I (n Xn) is a diagonal matrix with 1 in the

diagonal

3
 Matrix Properties
• The column rank of a matrix A is the maximal
number of linearly independent columns of A.
Likewise, the row rank is the maximal number
of linearly independent rows of A.

• The rank of an m-by-n matrix is at most the

lesser of m and n.

• A matrix that has as large a rank as possible is

said to have full rank; otherwise, the matrix is
rank deficient.

Matrices
• Singular Matrix have no inverse
– Determinant is zero

• Inverse of a Matrix

• AA-1 = I

• (ABC)-1 = C-1B-1A-1

4
Determinant

Determinant

5
 Statistical Operations (from Wikipedia.org)

• Mean is the sum of the observations divided by the number of

observations. It describes the central location of the data

• Variance is the average of the squared differences between data points

and the mean. Standard deviation is the square root of the variance. It is a
measure of the spread of the values.

• Median is a type of average that is described as the number dividing the

higher half of a sample (or a population), from the lower half.
– More robust to outliers

• Mode of a data sample is the element that occurs most often in the
collection.
– For example, the mode of the sample [1, 3, 6, 6, 6, 6, 7, 7, 12, 12, 17]
is 6.
– Given the list of data [1, 1, 2, 4, 4] the mode is not unique, unlike the
arithmetic mean.

Percentile
• A percentile is the value of a variable below which a
certain percent of observations fall.

• E.g. 20th percentile is the value (or score) below which 20

percent of the observations may be found.

• The 25th percentile is also known as the first quartile; the

50th percentile as the median. Decile is equal to 10th
percentile.

• pth percentile of n ordered values is obtained by first

calculating the rank k=p(n+1)/100 , rounded to the
nearest integer and then taking the value that
corresponds to that rank

6
 Harmonic Mean
• The harmonic mean H of the positive real numbers a1, ...,
an is defined to be

• If for half the distance of a trip you travel at 40 kilometres

per hour and for the other half of the distance you travel
at 60 kilometres per hour,

• Your average speed for the trip is given by the harmonic

mean of 40 and 60, which is 48; (not 50!!)

• i.e. the total amount of time for the trip is the same as if
you travelled the entire trip at 48 kilometres per hour.

Geometric Mean
• The nth root of the product of all the samples of the data
set, where n is the number of samples.

• The geometric mean is useful to determine "average

factors".
• For example, if a stock rose 10% in the first year, 20% in
the second year and fell 15% in the third year, then we
compute the geometric mean of the factors 1.10, 1.20 and
0.85 as (1.10 × 1.20 × 0.85)1/3 = 1.0391. We conclude
that the stock rose 3.91 percent per year, on average.

7
 Relationship of the means
• In case of two numbers

• For all other cases,

Arithmetic Mean > Geometric Mean > Harmonic Mean

chi-square statistic
• X2 is the chi-square statistic is given by

X2 = Sigma [ (O-E)2 / E ]

where, O is the observed frequency and E is the expected frequency.

• For Heads, X2= (47-50)2/50=0.18

• For Tails, X2= (53-50)2/50=0.18
• The sum of these categories is 0.18 + 0.18 = 0.36
• Assess the significance levels and see if it is a fair coin
• More details about chi-square test are available at :
– http://cnx.org/content/m13487/latest/