Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
NEURAL DYNAMIC1:
ACTIVATIONHS AND SIGNALS
MAIN POINTS:
NEURONS AS FUNCTIONS()
SIGNAL MONOTONICITY
BIOLOGICAL ACTIVATIONS AND SIGNALS
NEURON FIELDS
NEURONAL DYNAMICAL SYSTEMS
COMMON SIGNAL FUNCTION
PULSE-CODED SIGNAL FUNCTION
NEURONS AS FUNCTION
Ii
j 1
ji
x j i
y f I j
NEURONS AS FUNCTION
Common nonlinear transduction description: a sigmoidal or S-shaped curve
Signal Function:
1
S ( x)
1 e cx
dS
S'
cS (1 S ) 0
dx
( c 0)
SIGNAL MONOTONICITY
In general, signal functions are monotone nondecreasing S>=0. In
practice this means signal functions have an upper bound or saturation
value.
An important exception: bell-shaped signal function or Gaussian signal
functions
S ( x) e
c0
cx 2
S ' 2cxe cx ,
2
S' x
The sign of the signal-activation derivation s is opposite the sign of the activation
x. We shall assume signal functions are monotone nondecreasing unless stated
otherwise.
SIGNAL MONOTONICITY
Generalized Gaussian signal function define potential or radial basis
function:
1
Si ( x) exp[ 2
2 i
input activation vector:
:
variance:
mean vector:
i 2
(
x
j j) ]
j
x ( x1 , , xn ) R n
i2
i ( 1i , , ni )
Because the function depend on all neuronal activations not just the ith
activation, we shall consider only scalar-input signal functions : S ( x )
i
SIGNAL MONOTONICITY
A property of signal monotonicity: semi-linearity
Comparation:
a. Linear signal functions:
computation and analysis is comparatively easy; do not suppress
noise.
b. Nonlinear signal functions:
SIGNAL MONOTONICITY
Signal and activation velocities
S =dS/dt
S dS dx S ' x
dx dt
1
s x
0
x T
1
s x
1
x T
x T
x T
NEURON FIELDS
In general, neural networks contain many fields of neurons. Neurons
within a field are topological.
Denotation:
FX
F X FY FZ
: input field
Fz : output field
s x
pairs x , y , x , y ...... x , y
1
1
2
2
m
m
Classification:
x g FX , FY ,......
y h FX , FY ,......
xi , yi
x x1 , x2 .....
y y1 , y 2 .....
FY
X t x1 t ,..., x x t R n
Y t y1 t ,... y n t R p
n
p
,
F
R
X
Y
Augmentation :
[ FX | FY ] FZ
Fz R n p
c>0
S
k
o
S(x)= cx
if |x|<
-r
if x-
r>0, r is a constant.
S
-r
S ( x) 0
cx
Another form:
if
if
cx 1
cx 0
else
S ( x ) min(1, max(0, cx ))
S' c 0
1
S ( x)
cx
1 e
e
e
c
x
2
c
x
2
c
x
2
Where c>0.
S ' cS (1 S ) 0
So the logistic signal function is monotone increasing.
S ( x k 1 ) S ( x k )
0
if x k 1 T
if x k 1 T
if x k 1 T
S ( x ) tanh( cx )
2
S ' c(1 S ) 0
Another form:
tanh( cx )
e cx e cx
e cx e cx
S ( x ) min(1, e cx )
When
e cx 1
S' ce cx 0
S ' ' c 2 e cx 0
S ( n ) c n e cx 0
S ( x ) max(0,1 e cx )
When
x0
S' ce cx 0
S ' ' c 2 e cx 0
S ( x ) max(0,
For
xn
cx
n 1
S'
cnx n 1
n 2
(c x )
S i (t ) xi ( s )e st ds
S j (t ) y j ( s)e st ds
where
1
xi (t )
0
if a pulse occurs at t
if no pulse at t
S i (t ) xi (t ) S i (t )
S j (t ) y j (t ) S j (t )
Current pulse and current signal or expected pulse frequency are
available quantities.
Another computational advantage:
1
0
If xi (t )
if a pulse occurs at t
if no pulse at t
1 s t 0
s t 0