NEURAL NETWORKS

ABA


10 )(107



1/1000 .

100 ) (1000/10
.

ABA

) (

)(Information Processing

) (Parallel

" "

ABA

ABA

)(1012

10000

ABA


)(Artificial Neuron

" "

) (Processing Unit
)(Perceptron

ABA

Wi xi >
i=1

If

Otherwise

1
=Y
0

W1

W2

+
Wn

X1
X2
.
.
.

Xn

" )(OR
" )(AND
" " )(NAND
" " )(NOR
ABA

:
Wi new = Wi old + a (O desired O) xi

:
Wi
=
O
=
xi
=
= a

ABA

"" )(OR

1
Odesired

x2

x1

W1
O

x1

W2

x2

ABA

:
W1 new = W1 old + (O desired O) x1
W2 new = W2 old + (O desired O) x2

ABA

10

x1

x2 W1old W2old Odesired O error W1

new

W2
new

W2 = 1 = W1 :
11

ABA

) (Classification
)(Separation

""
Boundary

X2
1
X1

0
)(1,0

(0,1) 0
0
)(0,0

ABA

12

" " XOR

!!!!!

X2
0

X1

1
)(1,0

(0,1) 1
0
)(0,0

ABA

13

" "

X2
0
X1

1
)(1,0

(0,1) 1
0
)(0,0

ABA

14

)(Feed forward networks

ABA

15

) (Input Layer )(Output Layer

) (Hidden Layer

ABA

16

O1

O2

O3

W56

W57

W48

W14

W46

W47

W58

x1

W15
W24
W25
W34
W35

2
3

x2

x3

ABA

17

""
) (Threshold

ABA

18


Supervised Neural Networks

) (Weights

ABA

19

" "
)(Error Back Propagation

ABA

20

)(Feedforward Networks
:

" "

" "

ABA

21

 Activation Functions

:
:
: ) (Differentiation

ABA

22

)(Activation Function
)( ) (

ABA

23

f (x)=x

)f (x

1
1+ e x

= )f ( x

1
x

)f (x

ABA

24

w11=3
x1 = 0.1

w12=4

w10=7
w21=5

x2 = 0.2

w20=8

w22=6
x1 = 0.1

w11=3

w12=4

x2 = 0.1
ABA

1.3

1
= 0.79
1
.
3

1+ e

w10=7

w21=5

w22=6

25

1.6

w20=8

1.62

1
= 0.83
1
.
6

1+ e

o=1.622

 )(Training Algorithm

)(Back propagation

) ( .

" " ) ( Least Mean Squares

ABA

26

:
1
1 + e x

= )f (x

) df ( x
]) = f ( x )[1 f ( x
dx

:




ABA

27

ABA

28

t (Target) :
O (Output) :
f (Activ. Func.) :
(Learning Rate) :
W (Weight) :
(Error signal) :

)(Subscripts
.
ABA

29

.
.

Ok

k
.
.

wjk

oj

wij

oi

.
.

i
.
.

ABA

30

 :

:
N k = W jk O j
j

:
1
) = f ( Nk
1 + e Nk

= Ok

ABA

31

 :
) ( :
) k = ( t k O k )f ' ( N k

:
) k = ( t k O k ) O k (1 O k

.
:
W jk W jk + k j
ABA

32

 :

:
W jk k
k

j = j 1 O j

:
W ij W ij + j i

ABA

33

ABA

34

Input Layer
(i)

x1

Hidden
(h)
W11
W12

x2

W22

W21

h1
h2

W10
Output (O)
W20

35

ABA



)Target (t

x2

x1

W 20
1

W 11 W 12 W 21 W 22 W 10
1

x2

x1

ABA

36

= hi1

= hi2

= ho1

= ho2

=N

=O

ABA

37

 :
= W11 x1 + W21 x2
)= (1) (0) + (0) (0
=0

hi1

= W12 x1 + W22 x2
)= (0) (0) + (1) (0
=0

hi2

ABA

38

1
h O1 =
1 + e h i1

hO 2 =
=

39

ABA

1 + e 0

= 0 .5

1
1 + ehi2
1

1 + e 0

= 0 .5


:
N = W10 hO1 + W20 hO2
)= (1) (0.5) + (1) (0.5
= 1

:
1
1
=
1 + e N 1 + e 1

=O

= 0.73106

ABA

40

:
) O = (t O )O (1 O

)= (0-0.73106) (0.73106) (1-0.73106

= -0.14373

ABA

41

W10 W10 + OhO1

)= 1 + (1) (-.14373) (0.5

= 0. 92813

W20 W20 + OhO2

)= 1 + (1) (-.14373) (0.5

= 0. 92813

ABA

42

.
:
h1 = hO1(1 hO1 )W10O
)= (0.5) (1-0.5) (.92813) (-0.14373
= -0.03335
h2 = hO2 (1 hO2 )W20O
)= (0.5) (1-0.5) (0.92813) (-0.14373
= -0.03335

ABA

43

:
W11 = W11 + h1 x1
)= 1 + (1) (-0.03335) (0
=1

W12 = W12 + h 2 x1

)= 0 + (1) (-0.03335) (0
=0

W21 = W21 + h 1 x2

)= 0 + (1) (-0.03335) (0
=0

W22 = W22 + h 2 x2

)= 1 + (1) (-0.03335) (0
=1

. .

ABA

44


W 20

W 10

0.92813 0.92813

t W 11 W 12 W 21 W 22

x1 x2

:
x1 = 0 , x2 = 1 , t = 1

ABA

45

W 20

W 10

W 22

W 21

0 .0 1 0 5 4 1 .0 0 8 3 8 0 .9 5 0 3 0 .9 6 0 5 6

x 1 x 2 t W 11 W 12
0

1 1

ABA

46

W20

W10

W22

W21

W12

W11

-3.5402 4.0244 -3.5248 4.5814 -11.9103 4.6940

ABA

47

)x 2 T arget (t

x1

) O utput (O
0.0264

0.9867

0.9863

0.9908

ABA

48

 :



. :

)(Overfitting
)(Underfitting

)(Normalization
)(Learning Rate

ABA

49

ABA

50

) (Early Stopping.
:
)(Training
)(Validation
)(Testing

ABA

51

Validation

Training

ABA

52

)(Trial and Error

ABA

53

MATLAB

ABA

54

 Normalization

.

:
:
: ) 1 (-1

ABA

55

Learning Rate

ABA

56

(
)

ABA

57

) (
) (



ABA

58

59

ABA