Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
There are three hidden units, units 3, 4 and 5, connected to the output unit, unit 6. The output of the network, y6 , is connected to a known, but nonlinear process:
y = y6 3 A distal teacher provides a training signal y, which is compared to the above estimate y . The weights of the network are corrected based on the error back propagation algorithm with learning rate to reduce the squared error given by: 1 2 E = (y y) . 2 Answer the following questions.
a). Using the chain rule, compute the incremental weight change to be made to w5 4 when inputs x1 and x2 and the corresponding distal teacher signal y are presented. b). Compute the incremental weight change w41 for the same input-output training data as part a). c). In computing w41 , discuss how the result of computing w64 as well as that of
w5 4 can be used for streamlining the computation.
d). For a particular pair of input data, z6 became very large, z6 1 . Is the weight change w31 large for this input? For another input, z5 became very large z5 1 , and z6 and z3 became 0. Are weight change w31 and w53 likely to be large when E is large? Explain why.
Unit 3 Unit 1
w31
x1
z3 g 3 ( z3 ) y3
Unit 5
w3 2 w5 3
Distal Teacher
w6 3 Unit 6
y5
y6
z6 g6 ( z6 )
z5 g 5 ( z5 )
w6 5
y = y6
y
3
x2
Unit 2
w41
w5 4
z 4 g 4 ( z 4 ) y4
w6 4
Nonlinear Process
w4 2
E=
Unit 4
1 ( y y )2 2
Problem 2 (60 points) Consider the following true system and model structure with parameter vector ,
S:
= ( a1 , a2 , c )T
where input sequence {u(t)} is white noise with variance and {e0(t)} is white noise with variance . The input {u(t)} is uncorrelated with noise {e0(t)} and {e(t)}. Answer the following questions. 1). Obtain the one-step-ahead predictor of y (t ) . 2). Compute covariances Rye (0) = E[ y (t )e0 (t )], and Rye (1) = E[ y (t )e0 (t 1)].
5). After identifying the parameters a1 and a2 , the true system has changed to: S: y (t ) + 0.3 y (t 1) + 0.1 y (t 2) = u(t 1) + e0 (t ) + 0.2e0 (t 1) .
Now consider the model structure: M ( ) : y (t ) + 0.3 y (t 1) + 0.1 y (t 2) = u(t 1) + e(t ) + ce(t 1) with only one unknown parameter = ( c ) . Obtain the asymptotic error covariance using the frequency-domain expression of Cov . [Hint, obtain the function A( ) involved in
N
What are the two most important or most inspiring things that you have learned in 2.160 that you think you should not forget even ten years after your graduation from MIT?