Tugas Mata Kuliah

Pengenalan Pola

Disusun Oleh :
Nama : Faransi Al-azdi
NIM : 09011181621029
Kelas : SK7A
Dosen Pengampuh : Dr. Erwin, S.Si., M.Si.

SISTEM KOMPUTER
FAKULTAS ILMU KOMPUTER
UNIVERSITAS SRIWIJAYA
2019
Exercise 1.3.1
Consider a 2-class classification task in the 2-dimensional space, where the data in both classes, ω1,
ω2, are distributed according to the Gaussian distributions N (m1,S1) and N (m2,S2), respectively. Let

1 0
m1 = [1, 1]T , m2 = [3, 3]T , S1 = S2 = [ ]
0 1

For P(ω1) = 1/6 and P(ω2) = 5/6, and

for P(ω1) = 5/6 and P(ω2) = 1/6

Solution. Utilize the function comp_ gauss_dens_val by typing

For P(ω1) = 1/6 and P(ω2) = 5/6, and
P1=1/6; P2=5/6;
m1=[1 1]'; m2=[3 3]'; S=eye(2); x=[1.8 1.8]';
p1=P1*comp_gauss_dens_val(m1,S,x);
p2=P2*comp_gauss_dens_val(m2,S,x);

Result

for P(ω1) = 5/6 and P(ω2) = 1/6

P1=5/6; P2=1/6;
m1=[1 1]'; m2=[3 3]'; S=eye(2); x=[1.8 1.8]';
p1=P1*comp_gauss_dens_val(m1,S,x);
p2=P2*comp_gauss_dens_val(m2,S,x);

Result
Exercise 1.4.1
Generate N 2-dimensional feature vectors from a Gaussian distribution N (m,S), where
0.9 0.2
m = [2,−2]T , S = [ ]
0.2 0.3
for N = 500 points and N = 5000 points
Let X be the resulting matrix, having the feature vectors as columns. Compute the ML estimate of the
mean value, m, and the covariance matrix, S, of N (m,S) and comment on the resulting estimates.

Solution. To generate X, type

N=500
randn('seed',0)
m = [2 -2]; S = [0.9 0.2; 0.2 .3];
X = mvnrnd(m,S,500)';
Result

N=5000
randn('seed',0)
m = [2 -2]; S = [0.9 0.2; 0.2 .3];
X = mvnrnd(m,S,5000)';

Result
Exercise 1.4.2
Generate two data sets, X (training set) and X1 (test set), each consisting of N = 1000 3-dimensional
vectors that stem from three equiprobable classes, ω1, ω2, and ω3. The classes are modeled by Gaussian
distributions with means m1 = [0, 0, 0]T , m2 = [1, 2, 2]T , and m3 = [3, 3, 4]T , respectively; their
covariance matrices are

0.8 0.2 0.1

S1 = S2 = S3 =[ 0.2 0.8 0.2 ]
0.1 0.2 0.8

Solution. To generate X, use the function generate_gauss_classes by typing

m=[0 0 0; 1 2 2; 3 3 4]';
S1 = [0.8 0.2 0.1;0.2 0.8 0.2;0.1 0.2 0.8]
S(:,:,1)=S1;
S(:,:,2)=S1;
S(:,:,3)=S1;
P=[1/3 1/3 1/3]';
N=1000;
randn('seed',0)
[X,y]=generate_gauss_classes(m,S,P,N);

Exercise 1.4.3
using P1 = 1/2, P2 = P3 = 1/4 to generate X and X1. For this case, because the a priori probabilities are
not equal, the Bayesian classifier should result in the best performance. Why?

Exercise 1.4.4
using P(ω1) = P(ω2) = P(ω3) = 1/3 and
0.8 0.2 0.1 0.6 0.01 0.01 0.6 0.1 0.1
S1 = [0.2 0.8 0.2], S2 = [0.01 0.8 0.01], S3 = [0.1 0.6 0.1]
0.1 0.2 0.8 0.01 0.01 0.6 0.1 0.1 0.6

Exercise 1.6.1

Exercise 1.7.1
Generate N = 1000 data points lying in the real axis, xi ∈ R, i = 1, 2,...,N, from the following pdf, and
plot p(x):
 1 1 𝑥2 2 1 (𝑥−2)2
P(x) = 3 exp(− )+
2𝜎12 3
exp(− 2𝜎22
)
√2𝜋𝜎 √2𝜋𝜎22

where σ2 1 = σ2 2 = 0.2.
Use the Parzen windows approximation of Eq. (1.9), with h = 0.1, and plot the obtained estimate.
with h = 0.01, N = 1000 and h = 0.1, N = 10,000. Comment on the results. The choice of h for a given
N needs careful consideration. Tips related to this choice are provided in [Theo 09, Section 2.5.6] and
the references therein.

Exercise 1.7.2
Generate N = 1000 data points from the following 2-dimensional pdf.

1 1 𝑥(1)2 +𝑥(2)2 2 1 𝑥(1)2 +(𝑥(2)−2)2

P(x) ≡ p(x(1), x(2)) = 3 √2𝜋𝜎
exp(− 2𝜎12
)+3 exp(− 2𝜎22
)
√2𝜋𝜎22

Exercise 1.7.3
Use the setup for the classification task in Example 1.4.3. Classify the data points of the set X1 using
the Bayesian classifier, where the estimate of the required values p(x|ω1), p(x|ω2) for each point in X1
is obtained via the Parzen window estimation method. Use different values of h and choose the one that
results in the best error performance of the classifier.

Exercise 1.8.1
Consider the data set generated in Example 1.7.1 and use the k-nearest neighbor density estimator to
estimate the required pdf with k = 21.
1 for k = 5,100. Repeat with N = 5000.

Exercise 1.8.2
Use the setup for the classification task in Example 1.4.3. Classify the data points of set X1 using the
Bayesian classifier. Estimate the required values p(x|ω1), p(x|ω2) for each point in X1 via the k-nearest
neighbor density estimation method. Use different values of k and choose the one that results in the best
error performance of the classifier.

Exercise 1.9.1
Generate a set X1 that consists of N1 = 50 5-dimensional data vectors that stem from two equiprobable
classes, ω1 and ω2. The classes are modeled by Gaussian distributions with means
m1 = [0, 0, 0, 0, 0]T and m2 = [1, 1, 1, 1, 1]T and respective covariance matrices.

1. Classify the points of the set X2 in Example 1.9.1, adopting the optimal Bayesian classifier.
That is, use the true values of the means and covariance matrices associated with the 5-
dimensional Gaussian pdfs. Compare the results with those obtained in Example 1.9.1.
2. Repeat Example 1.9.1 with X1 consisting of N1 = 1000 data vectors.

Exercise 1.10.1
1. Consider a 2-dimensional classification problem where the data vectors stem from two
equiprobable classes, ω1 and ω2. The classes are modeled by Gaussian distributions with means
m1 = [0, 0]T , m2 = [1, 2]T , and respective covariance matrices
 0.8 0.2
S1=S2=[ ]
0.2 0.8

2. Generate two data sets X1 and X2 consisting of 1000 and 5000 points, respectively. 2. Taking
X1 as the training set, classify the points in X2 using the k-NN classifier, with k = 1,7,15 and
adopting the squared Euclidean distance. Compute the classification error.

Exercise 1.10.2

Compose your own example of a 2-class classification task in the 5-dimensional space. Assume the
data to follow the Gaussian pdf in both classes. Choose the mean values and covariance matrices.
Produce two data sets, one for training and one for testing. Use the nearest neighbor classifier.
Experiment with different values of the mean values, the covariance matrix, the parameter k, and the
length of the training data set. Comment on the obtained results as well as the computational time
involved.