Project: Skills: Type: Status: Co-Author: Link: Multi-layer Perceptron Pattern Classification Neural Networks, MATLAB, Probability Theory

Artificial Intelligence Assignment2 - Programming Completed on June 2002 Kehinde Adekunl A comprehensive report is available in hard copy. -------------------------------------------------------------------------------Introduction: Artificial neural network can classify groups. Multi-layer perceptron (back prop agation) is suitable and widely used to solve complex classification problems. T he aim of the project was to "illustrate the learning behavior of back-propagati on neural network for pattern classification". In particular, the task was to wr ite a MATLAB program to implement the back propagation algorithm to classify two classes defined by the following conditional probability density functions.

Procedure: First, a training data was generated using MATLAB function randn and variances. Second, a feed forward network of one input layer (input themselves), two hidden layers, and one output layer was constructed. Then, the neural network was opti mized by appropriately selecting the number of neurons on each layer, the learni ng rate, and the momentum rate by trial and error. The Neural Network Implementation Code: %Lab2 format long % intialize variables data_size = 11000; % variance and mean vectors for classes variance_class_1 = 1 ; mean_vector_class_1 = [0;0] ; variance_class_2 = 4 ; mean_vector_class_2 = [2;0] ; % random data x for clases 1 & 2 C1 = zeros(2,data_size); C2 = zeros(2,data_size); for k=1:data_size C1(1,k) = mean_vector_class_1(1,1) ; C1(2,k) = mean_vector_class_1(2,1) ; C2(1,k) = mean_vector_class_2(1,1) ; C2(2,k) = mean_vector_class_2(2,1) ; end C1_x = C1 + sqrt(1)*randn(2, data_size) ;

C2_x = C2 + sqrt(2)*randn(2, data_size) ; % Back Propagation Neural Network net = newff([-10 10; -10 10], [8, 4, 1], {'tansig','tansig','purelin'}); net.trainParam.show = 50; % Show results every 50 iterations net.trainParam.epochs = 200; % Max number of iterations net.trainParam.goal = 1e-2; % Error tolerance stopping criterion net.trainParam.mu = 0.9; % Momentum net.trainParam.lr = 0.01; % Learning rate % Train network '0.5' output for class C1 and '0.95' for class C2 for i=1:1000 targets_C1(i) = 0.05; targets_C2(i) = 0.95; end targets = [targets_C1 targets_C2]; train_data = [C1_x(1,1:1000) C2_x(1,1:1000); C1_x(2,1:1000) C2_x(2,1:1000)]; net = train(net, train_data, targets); classification_boundary = 0.5; % Accuracy calculation for test data network_output_C1 = sim(net, [C1_x(1,1000:11000); C1_x(2,1000:11000)]); network_output_C2 = sim(net, [C2_x(1,1000:11000); C2_x(2,1000:11000)]); correct_C1 = 0; correct_C2 = 0; for i=1:10000 if (network_output_C1(i) < classification_boundary ) correct_C1 = correct_C1 + 1; end end for i=1:10000 if (network_output_C2(i) > classification_boundary ) correct_C2 = correct_C2 + 1; end end accuracy_for_C1 = (correct_C1 / 10000)* 100 accuracy_for_C2 = (correct_C2 / 10000)* 100 % The two nets are equally probable accuracy_net = 0.5*accuracy_for_C1 + 0.5*accuracy_for_C2 Primary Conclusion: The final network can be characterized as shown in the graph. The optimum learni ng rate was 0.01 and momentum rate was 0.5. The best average accuracy of classif ication for both classes was 78%. http://www.natkeeran.ca/AJhtmlport/AI.html