1

Electrical and Computer Engineering Department

University of Maryland College Park
ENEE 660
System Theory
Fall 2008
Professor John S. Baras

Solutions to Homework Set #1






the binary representation. Let us suppose that #U

/E
X
B
is nite and equals n. Then let r be a
Fibonacci (in binary) number with binary string length larger than n. We can nd such a Fibonacci
number no matter what n is because the recursion shows that Fibonacci numbers can be arbitrarily
large. Then we can write r as r = wvz where w, v, z are binary numbers. If v
i
is concatenation i
times, then wv
i
z will also be in the Fibonacci sequence for all i, because the states have to cycle
through a loop since the input string length is longer than n.
Consider r = wvz and r

= wv
i
z. Let n
w
, n
v
, n
z
be the value of w, v and z treated as binary
numbers. The numerical value of r is
n
w
2
|z|+|v|
+ n
v
2
|z|
+ n
z
and that of r

is
n
w
2
|z|+i|v|
+ n
v
2
|z|
(1 + 2
|v|
+ + 2
(i1)|v|
) + n
z
Then both r, r

are Fibonacci. And r

is Fibonacci for all i. Clearly given the recursion that

Fibonacci numbers satisfy this cannot be true. Given any n, I can nd i for which the recursion
will not be satised.
However, given the recursion it will become clear in a couple of lectures in the course that I can
indeed realize this sequence using a two dimensional linear system. But the two dimensional linear
system uses innite number of states as an automaton.
(b) We want to examine if the sequence of prime numbers
P = 2, 3, 5, 7, 11, 13,
is realizable. We formulate the problem as follows. First we represent each prime number in binary
representation. This gives us the set of strings of 0s and 1s, beginning with a 1, whose value is
interpreted as the binary representation of a prime. So let U = 0, 1, consider U

R = u U

, rst element of u is 1 , u is the binary representation of a prime

2 is the string 10
3 is the string 11
5 is the string 101
7 is the string 111
11 is the string 1011
13 is the string 1101
etc.etc.
_

_
This set of strings from
U

form R
Take as output set Y = 0, 1. We can now state the problem as:
Has the I/O map:
2
A
R
: U

Y
a nite set of Nerode equivalent classes? The answer is no. We prove this below.
To prove the result we need two results from number theory. First, it is a fact that the number of
primes is innite and therefore there are arbitrarily large primes. Second, it is also a fact (proven
by Fermat) that 2
p1
1 is divisible by p for any prime p > 2. Or, what is the same, 2
p1
1 mod
p.
We proceed by contradiction. Suppose #U

/E
X
R
= n. Let z be the binary representation of a
prime p such that p > 2
n
. Such a prime exists since there are innitely many primes. Then we can
write z = uvw, where the length of v is 1 and uv
i
w (where v
i
is the concatenation of v , i times)
is the binary representation of a prime for all i. This is true because the length of z is longer than
n (the number of states) and so there must be a piece of z that creates a loop in the state set.
Let n
u
, n
v
, n
w
be the value of u, v, and w treated as binary numbers. If u or w are , the n
u
or n
w
,
is 0. Let i = p. Then uv
p
w is the binary representation of a prime q. The numerical value of q is
n
u
2
|w|+p|v|
+ n
v
2
|w|
(1 + 2
|v|
+ + 2
(p1)|v|
) + n
w
,
where [ [ is the binary strings length.
By Fermats theorem 2
(p1)
1 mod p. We raise both sides to the power [v[ and get 2
(p1)|v|
1
mod p. Then 2
p|v|
= 2
(p1)|v|
2
|v|
2
|v|
mod p.
Let s = 1 +2
|v|
+ +2
(p1)|v|
. Then (2
|v|
1)s = 2
p|v|
1 = 2
|v|
1 mod p. So (2
|v|
1)(s 1) is
divisible by p. But 1 [v[ n and so 2 2
|v|
2
n
< p. So p cannot divide 2
|v|
1, and therefore
it must divide s 1. Or s 1 mod p. But
q = n
u
2
|w|+p|v|
+ n
v
2
|w|
s + n
w
and therefore
q n
u
2
|w|+|v|
+ n
v
2
|w|
+ n
w
mod p ,
where the r.h.s. is the numerical value of p. Therefore q p mod p; or p divides q. Since q > p > 1, q
cannot be prime. But we started from a q that was prime. So contradiction.
Problem 3
(a) Since U

/
f
F(U

/E
f
) (i.e., U

/
f
is a subset of the set of functions of U

/E
f
into itself)
we have
n

#F(U

/E
f
) = n
n
Since x
f
y implies xE
f
y we also have #U

/
f
#U

/E
f
or n

n.
3
(b) Here U = u
1
, u
2
, , u
n1
= Y . All inputs are reset inputs i.e. f(zu
k
) = u
k
for all z U

f() = u
1
We know in general u

f
u

implies u

E
f
u

. Now suppose u

E
f
u

. Then f(vu

z) = u

=
last element of the string z from U = f(vu

z).
So u

f
u

also.
Therefore #U

/
f
= #U

/E
f
.
(c) Here U = u
1
, u
2
, , u
n
n
, Y = y
1
. , y
n
. Let g
1
, g
2
, g
n
n
be the set of functions of
Y into itself. We can re-label them by the inputs, since their numbers are the same. So we
let g
u
1
, g
u
2
, , g
u
n
n
be these functions.
The I/O map is
f() = y
1
Y
f(zu
j
) = g
u
j
(f(z))
Now u

E
f
u

= f(u

z) = f(u

z), z U

= g
u
z
(f(u

)) = g
u
z
(f(u

))
where u
z
is the element of U
uniquely dened by the string z
= f(u

) = f(u

)
Here u
z
is an element of U such that g
u
z
= g
z
l
(g
z
l1
(. . . (g
z
1
(.)))) as functions, where the string
z has the elements z = z
1
z
2
. . . z
l
. So the elements of U

/E
f
are in one to one correspondence
with the distinct images of elements of U

, which are obviously n, since Y has n elements.

The elements of U

/
f
are therefore the maps of Y into itself induced by elements of U

.
By denition of f these include all the functions g
u
k
. Thus n

= n
n
.
Problem 4
The input-output description is
U = 0, 1, Y = 0, 1, 2, 3
| = U

and
f

(u) = f

(u
i
; i = 1, n) = (
n

i=1
u
i
) mod 4
Nerode equivalence classes
4
x
0
= [0] = input strings with 4 1

s
x
1
= [1] = input strings with 4 + 1 1

s
x
2
= [11] = input strings with 4 + 2 1

s
x
3
= [111] = input strings with 4 + 3 1

s
where is an arbitrary integer. Now the Nerode equivalence relation of f

is actually a congruence.
Indeed, if u
1
E
f
u
2
we have
_
_
n
1

i=1
u
1
i
_
_
mod 4 =
_
_
n
2

k=1
u
2
k
_
_
mod 4.
Let v, w be strings of length n
v
, n
w
respectively. Then
f

(vu
1
w) =
_
_
n
v

k=1
v
k
+
n
1

i=1
u
1
i
+
n
w

=1
w

_
_
mod 4
=
_
_
n
v

k=1
v
k
+
n
2

i=1
u
2
i
+
n
w

=1
w

_
_
mod 4
= f

(vu
2
w) .
Therefore, the Nerode and Myhill equivalence classes coincide! So, the Nerode and Myhill realiza-
tions coincide as well! The state transition map is
a([x], u) = [xu] = [x] + [u]
where addition is mod 4. Recall from above that [u] = # of 1s in the string u. The read-out map
is
c([x], u) = [x] + [u] .
The number of output values is equal to the number of states. Diagramatically, we have
0,2
0,1 1,1
1,2 1,0
0,0
0,3
1,3
x
x
x
x
0 1
2 3
The construction for counting 1s mod 5 is identical. We now have 5 states instead of 4. Yes, the
Myhill semigroup is a group. Namely, the additive group Z
5
. Similarly, for any integer p, counting
ones mod p leads to a machine with Myhill semigroup Z
p
(additive).
5
Problem 5
The input-output map of the system is
f
I/O
= A
L
with | = a, b

and Y = 0, 1. Here, L is the set a

n
b
n
: n = 1, 2, . The criterion for
realizability of this map by a nite state machine is that the set of Nerode equivalence classes must
be nite. We want to show that there are innite number of Nerode equivalence classes. Indeed,
we claim that a
k
and a
n
are in dierent Nerode equivalence classes for k ,= n. Here, by a
k
we
understand a a
. .
k
. Now, recall that
a
k
E
f
a
n
f(a
k
w) = f(a
n
w)
for all wU.
Let w = b
k
. Then, f(a
k
b
k
) = 1, while f(a
n
b
k
) = 0. So a
k
is not Nerode equivalent to a
n
for k ,= n.
Therefore, there are an innite number of Nerode equivalence classes.
Problem 6
We let U = R, Y = R, X = R.
The parameter set is 0, 1, 2, 3, , i.e. discrete time. The next state transition map is
(t + 1; t, x, ) = x + p
t+1
,
and the readout map is
(t, x, ) =
t,n+1
x(t)
where
i,j
is the Kronecker delta symbol. To represent the Horner rule we initialize the machine at
x(0) = 0.
Solving for the state sequence in Horners rule gives
x(t) =
t1

i=0

i
p
ti
which is a convolution! So in matrix notation
_

_
x(1)
x(2)
.
.
.
x(n + 1)
_

_
=
_

_
1 0 0
1 0
.
.
.
0
.
.
. 1
.
.
.
.
.
.

n1
1
_

_
_

_
p
1
p
2
.
.
.
p
n+1
_

_
The optimality of Horners rule is discussed in A. Borodins Horners rule is uniquely optimal,
in Theory of Machines and Computations (1971), A. Kohavi and A. Paz (editors). From the last
6
computation, we deduce that Horners rule is the optimal rst order recursion to compute the
product of a Toeplitz matrix of the above form with any vector!
The input-output map of the system here is the map which associates to a given value(s) of s
the corresponding value(s) of the polynomial. Clearly, if we compute the value of the polynomial
naively (non-recursively) we will need (n + (n 1) + + 1) multiplications
_
=
n(n+1)
2
_
and n
additions. The use of recursions reduces the number of multiplications. Note that when we consider
the simultaneous evaluation of the value of the polynomial for several values of , the reduction
obtained by the Horner rule is even more dramatic. Recursions are equivalent to introducing
memory in the computational algorithm. Horners rule requires just a scalar as the state. In
this sense, the recursion utilizes the minimum memory, i.e. one state variable only. Because of
the structure of the dynamics this leads to optimality with respect to number of additions and
multiplications, but this is tricky to prove.
Problem 7
You can nd more details about Strassens algorithm in V. Strassen, Gaussian Elimination is not
Optimal, Numerische Math, Vol. 13, pp. 354-356 (1969).
Let
A =
_
a
11
a
12
a
21
a
22
_
, B =
_
b
11
b
12
b
21
b
22
_
, C = AB =
_
c
11
c
12
c
21
c
22
_
Naive multiplication requires 8 multiplications. Strassens algorithm is:
Let
z
1
= (a
11
+ a
22
)(b
11
+ b
22
) 1 multiplication
z
2
= (a
21
+ a
22
)b
11
1 multiplication
z
3
= a
11
(b
12
b
22
) 1 multiplication
z
4
= (a
11
+ a
12
)b
22
1 multiplication
z
5
= a
22
(b
11
+ b
21
) 1 multiplication
z
6
= (a
11
+ a
21
)(b
11
+ b
12
) 1 multiplication
z
7
= (a
12
a
22
)(b
21
+ b
22
) 1 multiplication
Now verify that
c
11
= z
1
+ z
5
z
4
+ z
7
c
12
= z
3
+ z
4
c
21
= z
2
+ z
5
c
22
= z
1
z
2
+ z
3
+ z
6
You have used only 7 multiplications. In return, you have used many more additions. Since
multiplications are more costly i.e. take more time, people are interested in reducing the number
of multiplications. You can also verify that the algorithm works without assuming commutativity
7
in the scalar multiplication! As a consequence of this obervation, the algorithm remains valid even
if a
ij
, b
ij
are matrices.
We can, therefore, extend it to higher dimensional matrices with dramatic savings in the number
of multiplications. For example, for 4 4 matrices A, B, naive multiplication C = AB, requires 64
multiplications. Now use Strassens algorithm. Partition to block 2 2 matrices
A =
_
A
11
A
12
A
21
A
22
_
, B =
_
B
11
B
12
B
21
B
22
_
.
The product C requires 7 2 2 matrix multiplications. Each 2 2 matrix multiplication requires
7 multiplications. So, total required is 7 7 = 49. Think of the savings for 100 100 matrices!
Naive multiplication requires 100
3
= 10
6
= 1, 000, 000 multiplications. Strassens algorithm
requires 7
log
2
100
= 7
7
= 823, 543. | indicates the smallest integer larger than.
The input-output map of the system here accepts as inputs the values of the elements of the
matrices A, B, and produces as output the values of the elements of the product matrix C = AB.
This map is multilinear (in the elements of A, B). To reduce multiplications, we must introduce
intermediate computations and proceed recursively. Strassens algorithm computes 7 intermediate
variables, using 7 multiplications. Applications of state realization theory to the evaluation of
multilinear functions have been considered by researchers. In this particular problem, we do not
have a complete minimality theory; only theories producing lower and upper bounds on the number
of multiplications are needed.
8