MATRIX STRUCTURES FOR IMAGE

APPLICATIONS: SOME EXAMPLES AND OPEN

PROBLEMS
Fabio Di Benedetto
Dipartimento di Matematica
Università di Genova
dibenede@dima.unige.it

Summary

We present two different examples of image applications where exploiting struc-

tured matrices seems attractive, either for reducing the high complexity of the
reconstruction problem or because a shift invariance naturally occurs in the
mathematical description. We discuss how structures can be incorporated in the
models and we point out some new open problems arising in these contexts.

1 Introduction

In the last years the insight about structured matrices has received a strong
impulse by several research groups in numerical linear algebra. Nearly ultimate
results have been recently achieved in two main directions.
1. The “horizontal” one, that is the development of effective mathematical
tools to describe and investigate new types of structures (e. g., the concept
of displacement operator, see [8]).
2. The “vertical” one, that is the construction of more and more efficient
algorithms for solving linear systems and least squares problems: in this
field the most attractive approach is the Preconditioned Conjugate Gradi-
ent Method (see [5] for a wide presentation of circulant preconditioning),
for which unifying convergence theories have been proposed very recently
[12, 6].
On the other hand, a wide number of applications take a great advantage from
the description in terms of matrix structures. Image processing is one of them
(perhaps the most popular): many papers of the group of Plemmons (see, e. g.,
[11]) are concerned with the reconstruction of atmospheric images, where the
blurring process can be well described with the help of Toeplitz matrices.
 Besides, there are still several image applications for which structured matri-
ces have not been fully exploited. Here we present two different examples, by
discussing how structures can be incorporated in the mathematical models and
by pointing out some new open problems derived from these contexts.

2 A first application:
reconstruction of tomographical images

The basic problem consists of inverting an integral operator

Z
g(s,θ) = K(P ; s,θ)f(P ) dP, (2.1)
Ω

where f : Ω −→ IR is typically the (unknown) distribution of some physical

quantity over a 2-dimensional body Ω, while g(s,θ) is the measured value of de-
tected radiations, corresponding to the position s on a detector which is rotated
of the angle θ around the body.
For simplicity, we can assume that the center of rotation is the origin and
that Ω is a disc with the same center (the value of f is often negligible outside a
suitable “region of interest”: then it suffices to choose Ω entirely containing this
region).
The noisy data set of the problem contains the projections {g(sj ,θk )}: {θk }
represents the sequence of angular positions where the detecting device is moved
during the data acquisition, while the whole surface of the detector is discretized
into a grid whose generic element (bin) is centered at sj .

2.1 Natural pixels discretization

A numerical approach to the continuous model (2.1) must take into account the
discretization of the data set; leaving the pixel P treated as a continuous variable
leads to the so-called natural pixels operator [4]

R: f 7→ {g(sj ,θk ); j = 1, . . ., q, k = 0, . . . , p − 1}

acting from L2 (Ω) to IRpq . We assume, as is the standard in practice, that both
the θk ’s and the sj ’s are equally spaced, θk = k∆θ and sj = j∆s.
It is evident that R has a nontrivial null space, so that the equation Rf = g
potentially has infinitely many solutions. Among them, the most significant are
the following.

• The generalized or pseudo-solution f † , that is the minimal L2 -norm solution

of Rf = g. It is given by the formula f † = R∗ (RR∗ )−1 g, R∗ being the
adjoint operator of R.

2
 • The regularized solution fµ = R∗ (RR∗ + µI)−1 g, where µ is a suitable
parameter that avoids ill-conditioning effects on f † .
• The positive solution
f+ = arg min {kfkL2 : Rf = g, f ≥ 0} , (2.2)
which takes into account the physical meaning of the function f. This case
will be the subject of future work.
The first two choices lead to a linear system of pq equations in pq unknowns.
In fact, the adjoint R∗ maps a vector h = (hjk ) ∈ IRpq to the L2 function
q p−1
X X
(R∗ h)(P ) = hjkK(P ; sj ,θk ); (2.3)
j=1 k=0

hence, a solution f † or fµ belongs to the linear subspace of L2 spanned by the

pq instances of the integral kernel K appearing in (2.1). The coefficient vector h
of this linear combination satisfies the normal equations
(A + µI)h = g, A = RR∗ ∈ IRpq×pq . (2.4)
The symmetric, positive semidefinite matrix A can be partitioned in p2 blocks
of size q × q: its (i,k) block has the expression
q
Ai,k = (hK(P ; sj ,θi ),K(P ; sl ,θk )i)j,l=1 , (2.5)

h·,·i being the scalar product in L2 (Ω).

This large size matrix has to be constructed and stored only at once; whenever
a new data set is acquired, solving (2.4) gives the coefficients of the solution with
respect to the integral kernel basis {K(P ; sj ,θk )}j,k. Thus the unknown f can be
evaluated only at the points P of interest, by computing the linear combination
(2.3).

2.2 Exploiting the structures

In some typical applications, the dimensions of A are set to p = 120 angular

views and q = 128 bins, so that the number of elements to be generated should
be 153602.
In absence of structure, the computational cost of resolution for the linear sys-
tem (2.4) would become prohibitive. Moreover, our 2-d model must be extended
along a third spatial direction in order to match the real problem, so that the
2-d reconstruction step has to be repeated for each vertical discrete level we are
considering [2].
Hence, finding and exploiting structures in the matrix A plays a fundamental
role in order to reduce the complexity. An intuitive observation reveals that
the tomography problem is invariant under rotation: this must be quantified in
specific applications.

3
 The following result [9] states that, if rotational invariance occurs, the natural
pixels discretization allows us to find a strong structure in the matrix A.

Theorem 2.1 Let Uθ : IR2 → IR2 be the operator which rotates every point P
by the angle θ. Assume that the kernel K(P ; s,θ) of the integral operator (2.1)
satisfies the invariance property

K(P ; sj ,θk ) = K(Uθk P ; sj ,0). (2.6)

Then the matrix A has the following structure:

1. if θk = k∆θ with ∆θ < 2π/p, then A is block Toeplitz, that is Ai,k = αi−k;
2. if θk = 2πk/p, then A is block circulant, that is Ai,k = α(i−k)modp .

Case 1 is called the tomographic limited angle problem in literature. In most

applications the detecting device is designed to perform a complete turn around
the body, so that case 2 is the matter.
The presence of a circulant structure makes it possible to pull down the com-
plexity with respect to p.
• As the construction of A is concerned, it suffices to compute only the
first block column. Taking into account the symmetry of A, the necessary
computations are further reduced by a factor of two. More precisely, the
L2 scalar products appearing in (2.5) with the choice of subscripts
jpk p
k = 0 ; i = 0, . . . , ; j ≥ l if i = 0 or i =
2 2
allow us to retrieve all the elements of A.
• Since A + µI is block diagonalized by the FFT, the normal equations can
be solved in at most O(q p log p) + O(q3p) operations performed on the
data set, plus an extra O(q2 p log p) preprocessing step on A + µI (see [9]
for details).
According to the acquisition model, we can take full advantage from the struc-
ture of the problem just by checking the invariance condition (2.6). In [9] this
check is quite simple in the case of x-ray transmission.
We investigated the more difficult problem of SPECT (Single Photon Emission
Computed Tomography), where each point of the body emits γ-rays in all the
directions, so that the system must select those radiations that fall “almost
orthogonal” to the detector surface. This is done by “filtering” the rays through
a lattice of holes called collimator (Figure 1).
Thus the integral kernel K(P ; sj ,θk ) measures the fluence of photons from the
point source P passing through some hole and striking the detector (placed at
the angle θk ) into the interval [sj − ∆s/2, sj + ∆s/2] representing the j-th bin.
It can be expressed in terms of characteristic functions and turns out to satisfy
the following formula.

4
  
Xk
Theorem 2.2 [3] Given the point P and the indices l and k, define =
Yk
Uθk P and the parameters
L+B L+B
ak = 1 − , bl,k = sl − Xk ,
R0 + L + B + Yk R0 + L + B + Yk
B B
ck = 1 − , dl,k = sl − Xk ,
R0 + L + B + Yk R0 + L + B + Yk
where the constants R0, L, B are the distances explained in Figure 1. Then
q   
1 X ∆s bl,k + W/2 dl,k + W/2
K(P ; sj ,θk ) = min sj + , ,
∆s 2 ak ck
l=1
 
∆s bl,k − W/2 dl,k − W/2
− max sj − , , .
2 ak ck +

Here W is the hole diameter and {·}+ stands for max(·,0).

An immediate check shows that condition (2.6) is satisfied. Hence, the re-
construction of the body activity from the SPECT data can be accelerated by
exploiting the circulant structure of the normal equations. The efficient compu-
tation of the scalar products appearing in the entries of A is under investigation.

2.3 Open problems

First of all there is a lack in literature about the fast computation of the positive
solution f+ given by (2.2), if we want to exploit the structure of RR∗ .
However, in our opinion the most interesting challenge consists of reducing the
complexity with respect to the number of bins, q: is there any structure “hidden”
in the variable s which makes it possible?
We believe that such a structure exists, even though it is not apparent. Our
“clue” is the success obtained in some tomographic problems by multigrid tech-
niques involving the variable s [7].
This class of methods is based on a sequence of increasing levels of discretiza-
tion, at which the continuous problem can be projected. Our normal equations
represent the finest level; the solution is computed by an iterative method where
each step is the composition of a smoothing technique (e. g., Gauss-Seidel it-
eration) and a recursive call to the same problem defined at the coarser level
(whose dimension is halved).
For the SPECT model, it seems natural to associate each level to the param-
eter ∆s that is doubled when we make the transition to the coarser level [10];
thus, the simplest equation to be solved in the recursive calls corresponds to as-
suming a single bin of width 128∆s (the total length of the detector) collecting
all the emissions passing through the different holes. Remark that the circulant
structure with respect to the angles is retained at all the levels.
We don’t know at yet if a multigrid approach can succeed also for this problem.

5
 R0
O P

detector

sj

∆s
collimator

j - th bin

θk

Figure 1 SPECT collimator-detector system and geometrical constants

3 A second application: infrared astronomy

In the observation of astronomic objects by means of infrared techniques, the

thermal emission of the atmosphere and of the telescope optics produces a uni-
formly large noise affecting source images. In order to cut off the noise effects, one
usually applies a differential technique called chopping and nodding, by forming
the image g depending on the unknown object f as follows:

6
 g(x,y) = −f(x,y − ∆) + 2f(x,y) − f(x,y + ∆) (3.7)
where ∆ is the chopping amplitude and (x,y) is a generic pixel: this is done
by pointing the telescope to two sky areas close to the region of interest, corre-
sponding to the shifts ±∆.
Here the shift invariance is apparent, resulting in a band Toeplitz matrix A
discretizing (3.7), with many inner diagonals equal to zero: it is given by
(A)m,n = −δm,n + 2δm+K,n − δm+2K,n , m = 1, . . ., N, n = 1, . . . , N + 2K,
where δm,n is the Kronecker symbol; K is related to the ratio between ∆ and
the sampling distance.
A theoretical inspection of the associated system of equations then gives a lot
of information about the expected quality of reconstructions [1].
In particular, the condition number of A only depends on the quotient, q,
between N and K and is asymptotical to q2 (thus it is decreasing with respect
to K); the pseudo-inverse A† has piecewise constant diagonals, which produce
discontinuities in the reconstructions; the generalized solution f † is orthogonal
to the vector of all 1’s and therefore its components sum to zero.
In the light of the last property, for this problem it is not convenient to apply
standard fast methods for the solution of Toeplitz systems, since it is required
the computation of a constrained solution. More precisely, the problem

min kAf − gk
(3.8)
f ≥0
has a unique nonnegative solution of minimal norm, having much more physical
significance than the generalized solution.
Despite the complete theoretical understanding, at present it is not clear how
the structure of A can be exploited to obtain efficient methods for solving (3.8).
The best results are actually provided by the application of the so-called Pro-
jected Landweber method [1], that produces a sequence of nonnegative approx-
imations slowly converging to a solution of (3.8).

4 Conclusions

We presented two examples of image applications where taking into account the
structure of the matrices involved could help in order to reduce the computa-
tional cost of reconstructions.
The first example is concerned with tomography (more precisely SPECT imag-
ing): the problem is computationally very difficult, a matrix structure can be
found with respect to one of the two variables, at present no insight is available
concerning the other variable.
In the second example (infrared astronomy) the theoretical knowledge is com-
plete and the structures are apparent, but very few is known about reconstruction
algorithms exploiting them.

7
 We point out that all the numerical methods used in these applications should
take into account that the structured matrices discretize an inverse problem and
therefore some regularization approach is needed.

References

[1] M. Bertero, P. Boccacci, F. Di Benedetto and M. Robberto, “Restoration of

chopped and nodded images in infrared astronomy”, Inverse Problems, 15 (1999),
in press.
[2] P. Bonetto, P. Calvini, A. R. Formiconi and A. Passeri, “Accurate 3D modeling
in SPECT reconstruction and optimized computer implementation”. In Abstract
SIMAI ’98.
[3] P. Brianzi and F. Di Benedetto, “Exploiting matrix structures in SPECT models”,
In Abstract SIMAI ’98, Vol. 2, pp. 166–167, 1998.
[4] M. H. Buonocore, W. R. Brody and A. Macovski, “A natural pixel decomposi-
tion for two dimensional image reconstruction”. IEEE Trans. Biomed. Engrg. 28
(1981), pp. 69–78.
[5] R. Chan and M. Ng, “Conjugate gradient methods for Toeplitz systems”, SIAM
Rev., 38 (1996), pp. 427–482.
[6] F. Di Benedetto and S. Serra Capizzano, “A unifying approach to abstract matrix
algebra preconditioning”, Numer. Math., 82 (1999), pp. 57–90.
[7] V. E. Henson, M. A. Limber, S. F. Mc Cormick and B. T. Robinson, “Multilevel
image reconstruction with natural pixels”, SIAM J. Sci. Comput., 17 (1996),
pp. 193–216.
[8] T. Kailath and A. H. Sayed, “Displacement structure: theory and applications”,
Siam Rev., 37/3, (1995), pp. 297–386.
[9] H. Kruse and F. Natterer, “A direct algebraic algorithm in computerized tomog-
raphy”, In Large Scale Scientific Computing, Progress in Scientific Computing 7
(1987), pp. 245–256.
[10] D. Martinez, Metodo multilivello applicato a un problema SPECT , Graduate the-
sis in Mathematics, University of Genova, 1998.
[11] J. G. Nagy, V. P. Pauca, R. J. Plemmons and T. C. Torgersen, “Degradation re-
duction in optics imagery using Toeplitz structure”, Calcolo, 33, (1996), pp. 269–
288.
[12] E. Tyrtyshnikov, “A unifying approach to some old and new theorems on distri-
bution and clustering”, Linear Algebra Appl., 232 (1996), pp. 1–43.