4.4.4 Wavelets: Concepts and examples What is a wavelet?

A wavelet is a function (t ) of finite energy (L2) whose average is zero:

(t ) dt = 0

(4.47)

Example 4.9 For the reasons that will become apparent a while later, a useful wavelet should be a small wave, meaning that it has a compact support, or at least it approaches zero very quickly as t approaches infinity. However, a wavelet can be either smooth or not so smooth, with or without compact support. See, for example, the wavelets in Figure 4.21.

(a) Haar wavelet: nonsmooth, compact support 1 0 1 0.5 1 0.5 0 0.5 5 0 0.5 1 (b) Mexican hat: smooth, noncompact support 1.5

2 1 0 1 2 3 (c) Daubechies wavelet D4: Continuous, compact support

1 0.5 0 0.5 1 1 0.5 0 0.5 1 1.5 2

Figure 4.21

=== MATLAB code for Example 4.9 and Fig. 4.21 ===
2

% Inputs: sig -- parameter sigma for the mexican hat wavelet. % N -- length of the Daubechies wavelet (must be even). % Writen by W.-S. Lu, University of Victoria. % Last modified: Dec. 23, 2002. % Example: w = ex4_9(1,4) function w = ex4_9(sig,N) figure(1) subplot(311) t = -0.5:2/1023:1.5; p1 = [zeros(1,256) ones(1,256) -ones(1,256) zeros(1,256)]; plot(t,p1,t,zeros(1,1024)) axis([-0.5 1.5 -1.5 1.5]) grid title('(a) Haar wavelet: non-smooth, compact support') subplot(312) t = -5:10/1023:5; c = 2/((pi^0.25)*sqrt(3*sig)); p2 = c*(1 - t.^2/(sig^2)).*exp(-t.^2/(2*sig^2)); plot(t,p2,t,zeros(1,1024)) grid title('(b) Mexican hat: smooth, non-compact support') figure(2) [h,g,rh,rg]= daub(N); [f,w]=wavelet(rh,rg,9,0); sz = length(w)-1; t = (1-0.5*N):(N-1)/sz:(0.5*N);
3

figure(1) subplot(313) plot(t,w) axis([1-0.5*N 0.5*N 1.2*min(w) 1.2*max(w)]) grid title('(c) Daubechies wavelet D4: Continuous, compact support') ===============

For a wavelet to be useful in DSP and other fields of engineering and applied science, additional features such as those listed below turn out to be desirable: (a) Generate (through dilation and translation) orthogonal or biorthogonal basis functions in space L2; (b) Local in both time and frequency domains; (c) Vanishing moments
k t (t ) dt = 0

for k = 0,1,

, K 1

(4.48)

Fourier transform (FT) versus wavelet transform (WT) For a signal f (t ) L1 L2 , its Fourier transform is defined as

( ) = f

f (t ) e jt dt
4

(4.49)

which measures the strength of signal oscillations at each frequency . A weak point of the FT is that it does not say when these oscillations actually occur in the time profile of signal f(t). The reason for this is that the basis function e jt has an infinite region of support hence the poorest (zero) resolution in the time domain, although it has the finest resolution in the frequency domain. Example 4. 10 Consider two continuous-time signals

1 (sin 2 t + sin10 t + sin 20 t ) for 0 t 15 x1 (t ) = 3 0 elsewhere

and

sin 2 t sin10 t x2 (t ) = sin 20 t 0

for 0 t 5 for 5 < t 10 for 10 < t 15 elsewhere

Both signals x1(t) x2(t) contain frequencies 1Hz, 5Hz, and 10 Hz. For x1(t) these frequencies are everywhere for t [0, 15], but for x2(t) at any given time instant in [0, 15] it contains only a single frequency (1Hz for t [0, 5], 5 Hz for t (5,
5

10], and 10 Hz for t (10, 15]). To get the spectrum of the signals, we sample them for t [0, 15] with rate fs = 40 Hz, yielding 601 samples for each of the signals denoted by x1[n] and x2[n] respectively, and perform DFT of the discrete-time signals obtained. The signals x1(t) and x2(t) as well as their spectra are shown in Fig. 4.22. It is observed that although the time-domain profiles of the signals are quite different, their spectra are similar to each other. In other words, the spectrum provides no immediate and explicit information as which part(s) of the signals time-domain profile have contributed to the signals spectrum strength at a specified frequency.

(a) Signal x1(t) on time interval [0, 15s] 1

1 150 100 50 0 1

5 10 (b) Spectrum of x1(t) on frequency interval [0, 20Hz]

15

6 (c) Signal 8 x2(t) on10 12 [0,15s]14 time interval

16

18

20

1 150 100 50 0

5 10 (d) Spectrum of x2(t) on frequency interval [0, 20Hz]

15

10

12

14

16

18

20

Figure 4.22

=== MATLAB code for Example 4. 10 ===

Ts = 1/40;
7

t1 = 0:Ts:5; t2 = (5+Ts):Ts:10; t3 = (10+Ts):Ts:15; t = [t1 t2 t3]; x1 = (sin(2*pi*t)+sin(10*pi*t)+sin(20*pi*t))/3; x21 = sin(2*pi*t1); x22 = sin(10*pi*t2); x23 = sin(20*pi*t3); x2 = [x21 x22 x23]; xf1=fft(x1); xf2=fft(x2); fs = 1/Ts; L = (length(x1)-1)/2; f=0:(fs/2)/L:(fs/2); figure(1) subplot(411) plot(t,x1) grid title('(a) Signal x1(t) on time interval [0, 15s]') subplot(412) plot(f,abs(xf1(1:L+1))) grid title('(b) Spectrum of x1(t) on frequency interval [0, 20Hz] ') subplot(413) plot(t,x2) grid title('(c) Signal x2(t) on time interval [0,15s]') subplot(414)
8

plot(f,abs(xf2(1:L+1))) grid title('(d) Spectrum of x2(t) on frequency interval [0, 20Hz]') ==========

Remark: At another extreme of the spectrum, we note that Diracs delta function, which has also been popular in signal analysis, has the finest resolution in the time domain but the poorest resolution in the frequency domain. Now suppose we have a wavelet function (t ) L2 which satisfies (4.47) and (t ) = 1. The wavelet transform of f(t) at time u and scale s is defined as

f (u , s ) =

f (t )

1 t u ( ) dt s s

(4.50)

From (4.50) we see that a WT of signal f(t) defines a family of functions rather than a single function: For each fixed scale (resolution) s, the WT in (4.50) can be viewed as a linear filtering process with a filter whose impulse response is given by

s (t ) =

1 t ( ) s s
9

(4.51)

resulting in a time function to describe the details or changes in signal f(t) at scale s. From (4.51) we can compute the transfer function of the filter as

( ) = s ( ) s ( ) is the FT of the wavelet function (t ). where

Since for each s, we have (0) = (t ) dt = 0

(0) = s (0) = 0 s

This implies that the wavelet filter is a bandpass filter.

Example 4.11
1. Mexican-Hat wavelet (t ) =

1/ 4

t 1 e 2 3
2

t2 2 2

whose Fourier transform is given by

10

8 5 / 2 1/ 4 2 2 2 / 2 ( ) = e 3 The plots of the Mexican hat wavelet (with = 1) and its FT are shown in Fig. 4.23. We observe that (t ) is smooth. It has an infinite support but most of its energy is concentrated in a rather small region in the time domain. ( ) is bandpass with its largest gain at 1.42 rad./s. ( ) has an infinite support as well, but its energy is concentrated in a small region in the frequency domain.

11

(a) Mexicanhat wavelet in the timedomain 1

0.5

0.5 5

0 Time

(b) Mexicanhat wavelet in the frequency domain 2

1.5

0.5

0 5

1 0 1 Frequency in rad./s

Figure 4.23

12

2. Haar wavelet

1 (t ) = 1 0
`Note that

for t [0, 1/ 2) for t [1/ 2, 0) elsewhere

(4.52)

(t ) is a wavelet with a compact support on interval

[0, 1], and it is normalized to satisfy

2 (t ) dt = 1.

The FT of the Haar wavelet is given by

( ) =

4 je j / 2 sin 2 ( / 4)

Fig. 4.24 depicts the amplitude response of the Haar wavelet filter. We see that it has an infinite support and a rather poor energy concentration in the frequency domain.
13

(a) Haar wavelet in the time domain 1.5 1 0.5 0 0.5 1 1.5 1 0.5 0 0.5 Time 1 1.5 2

(b) Haar wavelet in the frequency domain 1 0.8 0.6 0.4 0.2 0 50

40

30

20

10 0 10 Frequency in rad./s

20

30

40

50

Figure 4.24

3. Daubechies orthogonal wavelet of length 4 Daubechies compactly-supported orthogonal wavelets can be easily generated using the Daubechies orthogonal filter bank discussed earlier through an iterative algorithm called the cascade algorithm that will be discussed later. The wavelet
14

generated by the Daubechies orthogonal filter bank of length 4 is shown in Fig. 4.25a (see also Fig. 4.21c), and its spectrum is depicted in Fig. 4.25b.
(a) Daubechies wavelet (D4) in the timedomain 2

2 1

0.5

0.5 Time

1.5

(b) Spectrum of Daubechies wavelet 1000 800 600 400 200 0 50

40

30

20

10

0 10 Frequency in Hz

20

30

40

50

Figure 4.25

We observe that
15

 Although not very smooth, the Daubechies wavelet is continuous, and has a finite region of support. ( ) is bandpass. ( ) has an infinite support, but its energy is concentrated in a small region relative to that of the Haar wavelet.

Scaling functions
Wavelet function acts like a bandpass filter that picks the changes (i.e., details) of the signal. Obviously, one needs another type of functions (filters) to pick the main content of a signal. This is what a scaling function, which is essentially a lowpass filter, will do. Formally, the scaling function, often denoted by (t ) , is defined by the associated wavelet function as

( ) 2 2 ds ( s ) ( ) = = d s 1

(4.53)

16

We see that the scaling function is essentially an aggregation of wavelets at scales 2 larger than 1. Once ( ) is computed using (4.53), (t ) can be found by the inverse Fourier transform of ( ) .
Example 4.12
1. The scaling function for the Mexican-hat wavelet is defined by

2 3/ 2 1/ 4 1 2 2 / 2 2 ( ) = + 2 e 3
See Figs. 4.26a and b for (t ) and ( ) , respectively. Note the lowpass characteristics of ( ) . 2. The Haar scaling function can be conveniently defined in the time domain as

(t ) =

1 0

for t [0, 1) elsewhere

(4.54)

Plots of (t ) and ( ) are shown in Fig. 4.27.

17

(a) Mexicanhat scaling functionin timedomain

0.8 0.6 0.4 0.2 0 0.2 5 4 3 2 1 0 Time 1 2 3 4 5

(b) Mexicanhat scaling function in frequency domain 2 1.5 1 0.5 0 5 4 3 2 1 0 1 Frequency in rad./s 2 3 4 5

Figure 4.26

18

(a) Haar scaling function in time domain 1.5

0.5

0.5 1

0.5

0.5 Time

1.5

(b) Haar scaling function in frequency domain

1 0.8 0.6 0.4 0.2 0 50 40 30 20 10 0 10 Frequency in rad./s 20 30 40 50

Figure 4.27

We note that for the Haar scaling function and wavelet:

Both (t ) and (t ) are very simple and have support on interval [0, 1]. Both (t ) and (t ) have discontinuities.
19

 Both ( ) and ( ) are continuous and somewhat local. ( ) exhibits a lowpass characteristic.
1. The Daubechies scaling function associated with Daubechies orthogonal filter bank of length N can also be generated using the cascade algorithm. The scaling function has a compact (i.e. finite) region of support on the time interval [0, N 1]. Figs 4.28a and b depict the scaling function with N = 4 and its spectrum, respectively.

20

(a) Daubechies scaling function (N = 4) in timedomain 2

0.5

1.5 Time

2.5

(b) Spectrum of Daubechies scaling funtion (N = 4) 1200 1000 800 600 400 200 0 50 40 30 20 10 0 10 Frequency in Hz 20 30 40 50

Figure 4.28
21

Compared to the Haar scaling function, it is observed that the Daubechies scaling function is continuous in the time domain with improved energy concentration in the frequency domain.

Dyadic wavelet transform

In the continuous wavelet function

s ,u (t ) =

1 t u ( ) s s

if one chooses s = 2 j and u = 2 j k with j , k integers , then the wavelet function becomes

j ,k (t ) = 2 j / 2 (2 j t k ) for j , k Z

(4.55)

Note that here the integer variable j indicates the scale or resolution, with larger j corresponding to higher resolution; and integer variable k represents the location in time axis.

22

So if the wavelet is quite local, then we can estimate how signal f(t) is changing itself in a neighborhood of time instant t* at a given resolution level j by projecting the signal onto one of the basis functions in (4.55) with k = 2 j t .

Example 4.13 With (t ) being the Haar wavelet, Fig. 4.29 shows nine basis functions j ,k (t ) for j = 1, 0, 1 and k = 1, 0, 1. We see that
For each fixed resolution which is determined by the value of index j, the basis functions j ,k (t ) have the same width of support and can move along the t-axis to anywhere by using an appropriate value of index k.
The resolution can be increased by using an increased value of index j or decreased by using a decreased value of j.

23

(a): j = 1, k = 1 2 1 0 1 2 5 2 1 0 1 2 5 2 1 0 1 2 5 0 5 0 (g): j = 1, k = 1 5 0 (d): j = 0, k = 1 5 2 1 0 1 2 5 2 1 0 1 2 5 2 1 0 1 2 5

(b): j = 1, k = 0 2 1 0 1 0 (e): j = 0, k = 0 5 2 5 2 1 0 1 0 (h): j = 1, k = 0 5 2 5 2 1 0 1 0 5 2 5

(c): j = 1, k = 1

0 (f): j = 0, k = 1

0 (i): j = 1, k = 1

Figure 4.29

24

 Dyadic scaling functions

Similarly, the family of the dyadic scaling functions can be generated by a mother scaling function (t ) as

j ,k (t ) = 2 j / 2 (2 j t k ) for j , k Z

(4.56)

Suppose the mother scaling function (i.e., function (t ) ) has a small compact support, then for fixed indices j and k, the projection of signal f(t) onto j ,k (t ) provides information of signal f(t) at time t near by the time instant 2 j k , and the information you get is accurate up to a resolution of 2 j (often referred to as resolution level j ).

Signal approximation by wavelets

Given a signal f(t) in space L2, one can approximate it in many ways. Essentially, each method of approximation is associated with a system of basis functions in L2, say { bk (t ) }. For simplicity let us assume the system { bk (t ) } is orthonormal and complete in L2, then the signal f(t) can be represented by the system as

25

f (t ) = f (t ), bk (t ) bk (t )
k

(4.57)

A truncation of the first K terms of (4.57) can be used as an approximation of the original signal f(t), i.e.,

f (t ) f K (t ) =

k =1

f (t ), bk (t ) bk (t )

(4.58)

Now suppose we have another orthonormal and complete system of basis functions { ck (t ) }, hence the signal can also be approximated by a similar truncation:
K f (t ) f K (t ) = f (t ), ck (t ) ck (t ) k =1

(4.59)

The question is, which approximation, (4.58) or (4.59), is better? To address this question, we need a quantitative measure for the performance of an approximation. A general consensus is that an orthonormal and complete system is considered efficient if a small (relative to other systems) number of coefficients in the orthogonal expansion are found large in magnitude.
26

The wavelet theory has shown that basis functions generated from good wavelets often hold the promise of being efficient systems for signal representation (compression). As explained below, wavelet and scaling functions come naturally in signal representations.

Example 4.14 Given a signal f(t) in L2, first we try to represent it by a staircase approximation:

f (t ) f (k ) (t k )
k =

(4.60)

where (t ) is the Haar scaling function, and f(k) is the sampled value of f(t) at t = k. Fig. 4.30 depicts what happens on interval [1, 6].

27

Haar approximation of signal f(t) at resolution level j = 0 4

3.5

2.5

1.5

0.5

0 1

2 Time

Figure 4.30

If the above representation is found not accurate enough, we double the sampling rate to obtain a more accurate representation:

k f (t ) f ( ) (2t k ) k = 2
28

(4.61)

Note that the Haar scaling functions used in (4.61) are at resolution level j = 1, while the Haar scaling functions in (4.60) are at resolution level j = 0. Fig. 4.31 shows an improved approximation of f(t) on the same interval.
Haar approximation of signal f(t) at resolution level j = 1 4

3.5

2.5

1.5

0.5

0 1

2 Time

Figure 4.31
29

By examining the above figure carefully, we see that the improved (i.e., higher resolution) approximation can be achieved using the preceding (lower resolution) approximation plus a properly weighted Haar wavelet function of same (i.e.,lower) resolution on each interval [k, k+1). For example, it can readily verified that on the interval t [0, 1) , we can write

f (t ) f (0) (2t ) + f (0.5) (2t 1) 1 1 = ( f (0) + f (0.5)) (t ) + ( f (0) f (0.5)) (t ) 2 2

In general, it can be shown that the improved approximation of f(t) as described in (4.61) can be written as

1 f (k ) + f k + f (k ) 2 f (t ) (t k ) + k = k = 2 f 0 (t ) + d 0 (t )

1 f k + 2 (t k ) 2 (4.62)

30

where function f0(t) denotes the first sum in (4.62) which is an approximation of f(t) at level j = 0, and d0(t) denotes the second sum in (4.62) which indicates the changes (details) in signal f(t) at resolution level j = 0. The significance of Eq. (4.62): It says that if we combine an approximation and together with the details, which was lost in such an approximation, in an appropriate manner, a more accurate signal representation can be achieved. It indicates that the reverse also holds: A high-resolution signal representation can be split into a lower resolution approximation plus a lower resolution detail signal. In the above example, if we denote the improved approximation (at resolution level j = 1) as f1(t), then (4.62) becomes

f1 (t ) = f 0 (t ) + d 0 (t )
As we can see from the second sum in (4.62), the detail signal d0(t) contains many small-magnitude coefficients that are likely negligible. But what about the term f0(t) ?
31

We can apply the same idea of splitting a signal into lower resolution approximation and detail components to f0(t) to obtain
f 0 (t ) = f 1 (t ) + d 1 (t )

where d-1(t) denotes the details at resolution level j = 1 , which likely contains many small-coefficient terms, and f-1(t) denote the signals approximation at resolution j = 1. It is important to stress that in practice signals in many DSP applications are of finite length, often representing the sampled values of continuous-time signals in certain finite time duration. In such cases, as the decomposition proceeds the total number of terms in lower resolution approximation components decrease at rate 21 . In general, let the signal of interest be given at a certain resolution level, say, j = j*, as fj*(t). Then an efficient representation of f(t) can be obtained as

32

f j* (t ) = f j*1 (t ) + d j*1 (t ) = f j*2 (t ) + d j*2 (t ) + d j*1 (t ) = = f j* K (t ) + d j* K (t ) + + d j*2 (t ) + d j*1 (t ) (4.63)

4.4.5 Connecting Wavelets to Filter Banks: Mallats Multiresolution Analysis (MRA)

In his celebrated work S. Mallat, A theory of multiresolution signal decomposition: the wavelet representation, IEEE Trans. Pattern Recognition and Machine Intelligence, vol. 11, pp. 674-693, 1989. Mallat proposed a rigorous mathematical framework for wavelet representation of signals. In what follows we briefly describe two key components of his theory, namely, the multiresolution analysis (MRA) and fast discrete wavelet decomposition algorithm.

33

 Mallats Multiresolution Analysis A sequence {V j , j Z} of closed subspaces of L2 is said to be a multiresolution approximation if the following properties hold: (1) For every ( j , k ) Z 2 , f (t ) V j f (t 2 j k ) V j (2) For every j Z, V j V j +1 (3) For every j Z, f (t ) V j f (2t ) V j +1 (4) lim V j = L2
j

(5) lim V j = {0} (6) There exists (t ) such that { (t k ), k Z} is a Riesz basis of subspace V0. The rationale of the MRA can be easily understood by using the Haar scaling function as (t ) in property (6). One can think of subspace Vj as a place that collects all the signals with resolution level j. If we have to deal with a signal with resolution higher than j, then by property (4) we know that it is contained in some subspace Vk with k > j. Property (2) implies that {V j , j Z} has a nested structure shown in Fig. 4.32.
34
j

V1

V0 V-1

Figure 4.32

There are several immediate conclusions that one can draw from the MRA: (a) Since { (t k ), k Z} is a basis for V0, {2 j / 2 (2 j t k ), k Z} is a basis for Vj. Moreover, if { (t k ), k Z} is an orthonormal basis, i.e.,

(t i ), (t k ) = (i k )
then so is {2 j / 2 (2 j t k ), k Z}.
35

(4.64)

(b) Since (t ) V0 V1 , (t ) cab be expressed in V1 as

(t ) = 2 hk (2t k )
k =

(4.65)

Eq. (4.65) is called the dilation equation, which is one of the key equations in wavelet theory. The relation between two consecutive subspaces, i.e., V j V j +1, makes one wonder what is in the gap between the two subspaces? Actually, this gap is where the wavelet function (t ) comes to play: space Wj is defined as the orthogonal gap between Vj and Vj+1:

V j +1 = V j W j

(4.66)

where denotes the orthogonal direct sum. As such, Wj carries the details necessary to increase the resolution from level j to level j + 1. Fig. 4.33 illustrates (4.66) as well as the nested relation in {V j , j Z} .

36

V1 W0 V0

Figure 4.33

Based on (4.66) and MRA properties (4) and (5), we can write

V j +1 = V j W j = V j 1 W j 1 W j = = kj = Wk L2 = j = W j (4.67)

Eq. (4.67) gives an orthogonal decomposition of space L2. It implies that for a finite-energy signal f(t) in L2, we have
37

f (t ) = P W j f (t )
j =

(4.68)

where Pwj denote the orthogonal projection operator onto space Wj. Since spaces Wj collect details of the signal, many terms in (4.68) are likely small and can therefore be neglected to obtain an accurate yet efficient representation of f(t). Now suppose that subspace W0 admits a basis { (t k ), k Z } which is generated by a wavelet function (t ) , then we (4.66) implies that

(t ) W0 V1
which means that (t ) can be expressed as an element in V1 as

(t ) =

k =

g k (2t k )

(4.69)

This important equation is called the wavelet equation. It follows that once the scaling function (t ) has been determined, (4.69) can be used to compute the wavelet function (t ) , provided that the coefficients {g k , k Z} are available.
38

 Orthogonal Wavelets and Orthogonal Filter Banks Suppose that { (t k ), k Z} is an orthonormal basis for V0, and { (t k ), k Z } is an orthonormal basis for W0. Then the dilation equation (DE) and wavelet equation (WE), i.e.,

(t ) = 2
and

k =

hk (2t k )

(DE) (WE)

(t ) = 2

k =

g k (2t k )

can be used to determine their coefficients:

hk =

1 t ( ), (t k ) 2 2

and

gk =

1 t ( ), (t k ) 2 2

Conversely, if we know the coefficients in the dilation and wavelet equations, then the scaling and wavelet functions can be numerically determined using these equations through the so-called cascade algorithm (to be discussed shortly).
39

The number of terms in the DE and WE will be reduced to N for compactly supported wavelet functions such as Haar and Daubechies orthogonal wavelets, where N is the length of the associated filters H0(z), etc. For example, it is easy to examine that for the Haar scaling and wavelet functions, the DE and WE simply become

(2 t ) (2 t 1) (t ) = 2 + 2 2

(4.70)

and

(2 t ) (2 t 1) (t ) = 2 2 2

(4.71)

We see that the coefficients in (4.70) and (4.71) are the impulse responses of the lowpass and highpass synthesis filters in the orthogonal Haar filter bank, respectively. This connection holds for any compactly supported orthogonal scaling and wavelet functions. In particular, the Daubechies scaling function is defined by

40

 (t ) = 2 hk (2t k )
k =0

N 1

(4.72)

where {hk, k = 0, , N 1} are the coefficients of the lowpass synthesis filter F0(z). The following iterative algorithm, known as the cascade algorithm, can be used to determine the numerical values of the scaling function (t ) on its support interval [0, N 1]:

( i +1)

(t ) = 2

N 1 k =0

hk (i ) (2t k )

(4.73)

for i = 0, 1, , with (0) (t ) being the Haar scaling function. Convergence of the algorithm is deemed when

( i +1) (t ) ( i ) (t ) 2
for a prescribed tolerance . Fig. 4.34 shows the 2nd, 4th and 12th iteration results of the cascade algorithm as applied to the Daubechies filter of length 4 (D4).
41

2nd iteration

1 0.5 0 0 0.5 1 1.5 4th iteration 2 2.5 3

1 0.5 0 0 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 3 0.5 1 1.5 12th iteration 2 2.5 3

Figure 4.34
42

=== MATLAB code of the cascade algorithm ===

% This function computes the compactly supported % scaling and wavelet functions based on the % impulse response of a lowpass wavelet fileter h^(z) % using the cascade algorithm. % Inputs: h -- coefficents of the lowpass wavelet filter % g -- coefficients of the highpass wavelet filter % epsi -- tolerance (for convergence) % Outputs: phi -- scaling function on [0, N-1] % psi -- wavelet function on [-N/2+1, N/2] % Example: [h,g,rh,rg] = daub(4); [phi,psi] = cascade(rh,rg,1e-3); % Written by W.-S. Lu, University of Victoria % Last modified: Feb. 18, 1999. function [phi,psi] = cascade(h,g,epsi) % Initialize iteration using Haar scaling function N = length(h); phw = [ones(1,1024) zeros(1,(N-2)*1024)]; e = 1; % Compute scaling function using the cascade iteration algorithm while e > epsi, phw20 = [phw(1:2:(N-1)*1024-1) zeros(1,(N-1)*512)];
43

phw2 = h(1)*phw20; for k = 1:N-1, w = h(k+1)*[zeros(1,k*512) phw20(1:(N-1)*1024-k*512)]; phw2 = phw2 + w; end phw2 = sqrt(2)*phw2; e = norm(phw2-phw)/N; phw = phw2; end phi = phw; % Compute wavelet function using the wavelet equation psw20 = [phw(1:2:(N-1)*1024-1) zeros(1,(N-1)*512)]; psw2 = g(1)*psw20; for k = 1:N-1, w = g(k+1)*[zeros(1,k*512) phw20(1:(N-1)*1024-k*512)]; psw2 = psw2 + w; end psi = sqrt(2)*psw2; % Plot the scaling and wavelet functions computed t = 0:(N-1)/((N-1)*1024-1):N-1; t1 = -N/2+1:(N-1)/((N-1)*1024-1):N/2; figure(1) subplot(121) plot(t,phi)
44

axis([0 N-1 1.1*min(phi) 1.1*max(phi)]) axis('square') grid title('scaling function') xlabel('time') subplot(122) plot(t1,psi) axis([-N/2+1 N/2 1.1*min(psi) 1.1*max(psi)]) axis('square') grid title('wavelet function') xlabel('time')

Once (t ) is found, the wavelet function can be obtained using the wavelet equation

(t ) = 2

k = ( N 2)

g k (2t k )

(4.74)

where {gk, k = - (N 2) , , 0, 1} are the coefficients of the highpass synthesis filter F1(z) with their indices shifted backwards by (N 2). Note that (t ) has a support on [1- N/2, N/2]. The orthogonal Daubechies wavelets of length N has p = N/2 vanishing moments. Fig. 4.35 shows the Daubechies scaling and wavelet functions with p = 2, 3, and 4
45

vanishing moments.

Mallats Fast Wavelet Transform (FWT)

Let a finite-energy signal f(t) be in space Vj+1 so that it can be expressed as

f (t ) = 2

( j +1) / 2

k =

( j +1) k

(2 j +1 t k )

(4.75)

Since V j +1 = V j W j , f(t) can be decomposed as

f (t ) = P V j f (t ) + P W j f (t ) where
( j) j/2 j P f = 2 a (2 t k) k Vj j/2 P d k( j ) (2 j t k ) Wj f = 2 k k

with (t ) and (t ) being a pair of orthogonal scaling and wavelet functions.

46

scaling function N = 4, p = 2 1.5 1 0.5 0 0 1 2 3 scaling function N = 6, p = 3 1 0.5

wavelet function N = 4, p = 2

1 0 1 1 0 1 2 wavelet function N = 6, p = 3

1 0

0 1 0 5 scaling function N = 8, p = 4 1 0.5 0 0 5 2 0 2 wavelet function N = 8, p = 4 1 0.5 0 0.5 1 2 0 2 4

Figure 4.35
47

The fast orthogonal wavelet transform proposed by Mallat addresses two basic questions:
( j +1) ( j) , k Z}, how to compute {ak , k Z} and Given {ak {d k( j ) , k Z}? (Signal Decomposition) ( j) , k Z} and {d k( j ) , k Z}, how to construct Given {ak ( j +1) {ak , k Z}? (Signal Reconstruction)

For the decomposition,

( j) k

n =

n2 k

( j +1) an = (a ( j +1) * h(0) )(2k )

(4.76) d
( j) k

n =

n2 k

( j +1) an = (a ( j +1) * h(1) )(2k )

where h(0) and h(1) are the impulse responses of H 0 ( z ) and H1 ( z ). Notice that the operation (2k) in (4.76) corresponds to the downsampling-by-two operation in a subband system.
48

For the reconstruction,

( j +1) k

k =

hk 2 n a

( j) n

k =

( j) g k 2 n d n

= (a ( j ) * h)(k ) + (d ( j ) * g )(k ) where

( j) a if k = 2 p ak( j ) = p 0 if k = 2 p + 1

(4.77)

and

( j) d if k = 2 p d k( j ) = p 0 if k = 2 p + 1

Notice that the operation required to generate signals a ( j ) and d ( j ) from a ( j ) and d ( j ) in (4.77) corresponds to the upsampling-by-two operation in a subband system. Therefore, together (4.76) and (4.77) admit a 2-channel filterbank interpretation with a(j+1) as its input signal. Based on (4.76) and (4.77), a K-level fast signal decomposition and reconstruction can be carried out in an MRA framework as follows, where signal f(t) corresponds to a(J) : It is quite obvious that the above decomposition and reconstruction can be implemented using an orthogonal filter bank with a K-level tree structure, where
49

the coefficients of the synthesis lowpass and highpass filters are obtained from DE and WE, respectively.
References 1. I. Daubechies, Orthogonal bases of compactly supported wavelets, Comm. on Pure and Appl. Math., vol. 41, pp. 909-996, Nov. 1988. 2. I. Daubechies, Ten Lectures on Wavelets, SIAM, Philadelphia, PA, 1992. 3. S. Mallat, A theory for multiresolution signal decomposition: the wavelet representation, IEEE Trans. PRMI, vol. 11, pp. 674-693, July, 1989. 4. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed., Academic Press, New York, 1999.

7.1 Wavelet De-Noising 7.1.1 The de-noising problem

Suppose we have a noise-corrupted discrete-time signal x[n], which is modeled as

x[n] = s[n] + w[n]

noisy signal signal noise

n = 0,1,..., N 1

(7.1)

where s[n] is the noise-free signal, and w[n] is noise with n = 0, 1, , N 1. The de-noising problem is to estimate s[n] using the noise-corrupted observations
50

x[n]. In what follows we present a de-noising method based on discrete wavelet transform (or equivalently subband decomposition) of the noisy signal x[n]. The method was proposed by David Donoho and Ian Johnstone in 1993. The basic assumptions under which the method works are: (1) The number of samples, N, is large. (2) The noise w[n] has a normal distribution with zero mean and variance 2 , namely,

1 N 1 w[n] 0 N n =0

and

1 N 1 2 w [ n] 2 N n =0

(7.2)

7.2.2 Methodology and implementation

As illustrated in Fig. 7.1, the de-noising of a discrete-time signal x[n] starts by applying a discrete wavelet transform (DWT) to it, resulting in a subband signal y[n].
Noisy signal x[n] DWT y[n] Thresholding
^[n] y

Inverse DWT

De-noised ^[n] signal x

Figure 7.1
51

From Chapter 4, we know that the DWT of x[n] can be performed by repeatedly using a two-channel analysis filter bank in a treelike structure. See Fig. 7.2 or Fig. 4.11 for a 3-level case, where the subband signal y[n] is formed as

y = [a3 d 3 d 2 d1 ]
H0(z) H0(z) x(n) H0(z) H1(z) 2 2
Figure 7.2

2 2

a3 d3 d2 d1

2 2

H1(z)

H1(z)

Next, soft thresholding (or an alternative such as hyperbolic thresholding), that will be explained shortly, is applied to the components of y[n] (which are sometimes called wavelet coefficients). This shrinks these wavelet coefficients according to a prescribed threshold that is determined chiefly by the noise [ n] . variance. The modified subband signal is denoted by y
52

[n] to construct Finally, the inverse DWT is applied the modified subband signal y [n] which hopefully would offer a less noisy profile of the a discrete-time signal x original (clear) signal s[n]. It follows from Chapter 4 that the inverse DWT can be implemented using a tree type filter-bank structure which is mirror-image symmetrical to that in Fig. 7.2, where the analysis filters H0 and H1 are replaced with synthesis filters F0 and F1, respectively, and the downsamplers are replaced by upsamplers. See Fig. 7.3 for a 3-level inverse DWT.
a3 d3 d2 d1
Figure 7.3

2 2

F0(z) F1(z) 2 2 F0(z) F1(z) 2 2 F0(z) F1(z) ^(n) x

Thresholding (shrinkage) policies

There are several ways in which thresholding of a given signal may be performed for a given threshold :
53

(1) Hard thresholding

y[n] yhard [n] = 0

(2) Soft thresholding

if if

y[n] > y[n]

(7.3a)

sgn( y[ n])( y[n] ) ysoft [n] = 0

if if

y[n] > y[n]

(7.3b)

(3) Hyperbolic thresholding (proposed by B. Vidakovic, also known as almost hard thresholding)

sgn( y[n]) y 2 [n] 2 yhyper [n] = 0

if y[n] > if y[n]

(7.3c)

Example 7.1 For y[n] = 0:1:9 and = 1, evaluate yhard, ysoft, and yhyper. The evaluation results are given in Table 7.1.

54

Table 7.1: y = 0 : 1 : 9, = 1
y[n] yhard ysoft yhyper 0 0 0 0 1 0 0 0 2 2 1 1.7321 3 3 2 2.8284 4 4 3 3.8730 5 5 4 4.8990 6 6 5 5.9161 7 7 6 6.9282 8 8 7 7.9373 9 9 8 8.9443

From the above table, it is quite clear as why the hyperbolic shrinkage is regarded as almost hard. Determination of threshold An important issue in the wavelet de-noising is to choose an appropriate value for threshold . The universal threshold proposed by Donoho and Johnstone in 1993 has been proven effective: it determines as

= 2log10 ( N )

(7.4)

where N is the length of the signal, and 2 is an estimate of the noise variance.

55

A convenient way to estimate noise variance 2 is to use the detail subsignal at the highest resolution, i.e., d1. Let d1 = { di(1) , i = 1, , N/2}, then an estimate of 2 can be obtained as

2 N / 2 (1) (di d ) 2 = ( N 2) i =1
2

2 N / 2 (1) where d = mean(d1 ) = di N i =1

(7.5)

Example 7.2 We now apply the above de-noising method with various wavelets and shrinkage policies to several noise-corrupted signals.
1. Piecewise constant signal (1024 samples, see Fig. 7.4). Noise: N(0, 2 ) that is a random noise sequence of normal distribution with zero mean and variance 2 , where 2 = 0.0107. Wavelet: Haar Shrinkage: Hyperbolic Threshold: Universal Results: The signal-to-noise ratio (SNR) before and after de-noising are denoted by SNRbefore and SNRafter, respectively, where

56

SNR before = 20log10

s[n] 2 = 27.1056dB x[n] s[n] 2 s[n] 2 = 41.0253dB x [n] s[n] 2

SNR after = 20log10

The clear, noise-corrupted, and de-noised signals are depicted in Fig. 7.4. For comparison purposes, the soft shrinkage defined by (7.3b) was also applied, which with the Haar wavelet yields SNRafter = 31.4887 dB. Application of different wavelets (with hyperbolic shrinkage) were applied and found that the Daubechies filters D4 yields SNRafter = 30.2252 dB and the SNRafter offered by higher-order Daubechies filters were even worse. 2. Doppler signal (1024 samples, see Fig. 7.5) Noise: N(0, 2 ) with 2 = 0.0107. Wavelet: Daubechies D8 Shrinkage: Hyperbolic Threshold: Universal Results: The SNR before and after de-noising are evaluated as
57

SNR before = 20log10

s[n] 2 = 14.0750dB x[n] s[n] 2 s[n] 2 = 19.6312dB x [n] s[n] 2

SNR after = 20log10

The clear, noise-corrupted, and de-noised signals are depicted in Fig. 7.5.

58

original signal 0.4 5 4 3 2 1 0 1 2 0 0.5 1 0.2 0.1 0 0.1 0.2 0.3 0 0.3

the noise

0.5

noisecontaminated signal 5 4 3 2 1 0 1 2 0 0.5 1 5 4 3 2 1 0 1 2 0

denoised signal

0.5

Figure 7.4
59

The Daubechies filters D8 employed in above example can be obtained using the wavelet toolbox UVi_Wave: the coefficients of Daubechies filters H0, H1, F0, and F1, which are denoted by h0, h1, f0, and f1 respectively, can be obtained as
[h0,h1,f0,f1] = daub(length);

where length is the length of the D-filters, which must be a positive even integer. Also note that, when length = 2, the above command returns with the coefficients of the Haar filters. For comparison purposes, different wavelets (with hyperbolic shrinkage) were applied and found that the Haar wavelet yielded SNRafter = 15.6099 dB and the Daubechies D4 offered an SNRafter = 18.0301 dB.

60

original signal 0.4 1 0.5 0.1 0 0.5 1 0 0.5 1 0.3 0 0 0.1 0.2 0.3 0.2

the noise

0.5

noisecontaminated signal

denoised signal

1 0.5 0 0.5 1 0 0.5 1

1 0.5 0 0.5 1 0 0.5 1

Figure 7.5
61

MATLAB Code for wavelet de-noising of 1-D signals

% This function de-noises an input of length N = 2^n using the wavelet % shrinkage method in which the "universal" threshold is used. % Note: The function requires the use of UVi_Wave wavelet toolbox. % Inputs: x0 -- Original signal. % r -- Noise signal. % h0 -- Impulse response of the lowpass analysis filter. % h1 -- Impulse response of the highpass analysis filter. % f0 -- Impulse response of the lowpass synthesis filter. % f1 -- Impulse response of the highpass synthesis filter. % choice: 0 -- Hyperbolic (almost hard) shrinkage. % 1 -- Donoho's soft shrinkage. % Output: y -- Denoised signal. % Written by W.-S. Lu, University of Victoria. % Last modified March 1, 2003. function y = deno_uv(x0,r,h0,h1,f0,f1,choice) x0 = x0(:); r = r(:); x = x0 + r; Nx = length(x); t = 0:1/(Nx-1):1; % Handle the not-power-of-2 case p = ceil(log10(Nx)/log10(2));
62

N = 2^p; if N > Nx, xa = [x;zeros(N-Nx,1)]; else xa = x; end % Wavelet decomposition xd = wt(xa,h0,h1,p); % Estimate noise variance d1 = xd(N/2+1:N); db = mean(d1); s2 = sum((d1-db).^2)/(N/2-1); % Compute Donoho's universal threshold delta = sqrt(2*log10(N)*s2) % Soft (or hyperbolic) shrinkage of wavelet coefficients xd if choice == 1, xs = sign(xd).*max(abs(xd)-delta,zeros(N,1)); else xs = sign(xd).*sqrt(max(xd.^2-delta^2,zeros(N,1))); end
63

% Reconstruct the denoised signal y1 = iwt(xs,f0,f1,p); y = y1(1:Nx); y = y(:); % Performance evaluation nf = norm(x0); disp('signal-to-noise ratio before denoising:') SNR_before = 20*log10(nf/norm(r)) disp('signal-to-noise ratio after denoising:') SNR_after = 20*log10(nf/norm(y-x0)) % Plot the results mi = min([min(x0) min(x) min(y)]); mi = (1 - 0.1*sign(mi))*mi; ma = max([max(x0) max(x) max(y)]); ma = (1 + 0.1*sign(ma))*ma; figure(1) subplot(221) plot(t,x0) title('original signal') axis([0 1 mi ma]) axis('square') grid subplot(222) plot(t,r)
64

title('the noise') axis('square') grid subplot(223) plot(t,x) title('noise-contaminated signal') axis([0 1 mi ma]) axis('square') grid subplot(224) plot(t,y) title('denoised signal') axis([0 1 mi ma]) axis('square') grid

7.1.3 Wavelet de-noising of still images

Extension of the wavelet de-noising method described in Sec. 7.1.2 to still images is straightforward. The noisy image is modeled by

x[i, j ] = s[i, j ] + w[i, j ]

noisy image image noise

i, j = 0,1,..., N 1
65

(7.6)

As explained in Fig. 7.6, the wavelet de-noising method in this case begins by applying a two-dimensional (2-D) DWT to the noisy image x[i, j] that yields a 2D subband signal y[i, j]; it then applies thresholding (shrinkage) to the subband signal y[i, j] pointwisely using one of the shrinkage policies described in (7.3a) to [i, j ] ; finally, the 2-D inverse DWT is applied to the modified (7.3c) to yield y [i, j ] to construct image x [i, j ] which would hopefully be a less subband signal y noisy profile of the clear digital image s[i, j].
Noise-corrupted image x[i, j]

2-D DWT y[i, j]

Shrinkage of Wavelet Coefficients

^[i, j] y

2-D Inverse DWT

De-noised ^[i, j] image x

Figure 7.6

2-D discrete wavelet transform (2-D subband decomposition) The concept of subband decomposition of discrete-time signals can be extended to two-dimensional signals such as still images in a straightforward manner. To this end, we need a subband decomposition-reconstruction framework similar to what has been developed for 1-D signals, with straightforward modifications in that a 2-D filter-bank building block replaces the one-dimensional filter-bank in
66

both the analysis filter-bank tree and the synthesis filter-bank tree. Fig. 7.7 depicts a 2-level 2-D analysis filter bank tree that performs a 2-level subband decomposition for a still image.
N N
N 2

N 2

N 2

N 4

N 2

N 4

N 4

a(2) H0 H0 a(1) H0 2 H0 H0 {x(i, j)} 2 H1 2 H1 H1 2 2 2 2 H1 2

(2) d10

2
(2) d01

(2) d11 (1) d01

H0 H1 2 H1 Level 1 Input Vertical Horizontal

(1) d10

2 Level 2 Vertical Horizontal

(1) d11

Figure 7.7
67

To illustrate how it works, let input signal {x[i, j ],0 i N 1,0 j N 1} represent an image of size N N, where the value of x[i, j] is the gray scale of the image pixel at position [i, j]. So it is of convenience to treat a digitized image as a matrix, just like a digitized 1-D signal is treated as a vector. At each decomposition level, we see a basic 2-D analysis filter bank that consists of 3 twochannel 1-D filter banks, where the first 2-channel filter-bank filters and downsamples the input image column by column. As a result, it outputs an approximate image of size N/2 N and a detail image of the same size. Then each of these images is filtered by a 2-channel filter bank row by row, yielding 4 (1) (1) (1) , d10 , and d11 . This subimages of size N/2 N/2 that are denoted as a (1) , d 01 completes the first-level 2-D subband decomposition. If necessary, the approximation subimage a(1) can be further decomposed using the same filter (2) (2) (2) of size bank module described above to obtain subimages a (2) , d 01 , d10 , and d11 N/4 N/4. Thus a two-level 2-D subband decomposition yields a total of 7 subimages. These subimages can be put together as shown in Fig. 7.8 to form a two-level 2-D subband decomposition of the input image. For an image of size N N with N = 2K, a K-level 2-D subband decomposition (2-D DWT) cab be performed with this tree-structured analysis filter banks.

68

a(2)
(2) d10

(2) d01 (2) d11

(1) d01

(1) d10

(1) d11

Figure 7.8

Also similar to the 1-D case, the 2-D inverse DWT can be performed using a treestructure 2-D synthesis filter bank that is mirror-image symmetrical to the analysis filter bank in Fig. 7.7, where the filters H0 and H1 are replaced with F0 and F1, respectively, and the downsamplers are replaced by upsamplers. See Fig. 7.9 for a two-level 2-D inverse DWT. Determination of threshold Let the input image be of size N N, Donohos universal threshold is then given by
69

 = 2 log10 N

(7.7)

2 is an estimate of the noise variance that can be obtained using the where (1) as highest resolution detail subimage d11
N
N 4 N 4 N 4 N 2 N 2 N 2 N 2

a(2) 2
(2) d01

F0 2 F0

F1 2 F0

(2) d10

F0 2 F1 2 F0 {^ x(i, j)} F1 2 F1

(2) d11 (1) d01

(1) d10

2
(1) d11

F0 2 F1 F1 Level 1

2 Level 2 Horizontal Vertical

Horizontal

Vertical

Output

Figure 7.9
70

4 N / 2 N / 2 (1) 2 = 2 d i j d [ ( , ) ] 11 N 4 i =1 j =1
2

(7.8)

4 d= 2 N

d
i =1 j =1

N /2 N /2

(1) 11

(i, j )

Example 7.3 We now apply the above de-noising method with hyperbolic shrinkage and several wavelets to a 8-bit gray-scale image fruits of size 256 256. Noise: N(0, 2 ) with = 25.5953. Wavelet: MIT 9/7 Shrinkage: Hyperbolic Threshold: Universal
where the MIT 9/7 wavelet is characterized by a linear-phase lowpass analysis filter H0 of length 9 and a linear-phase lowpass synthesis filter F0 of length 7 whose coefficients are given by

71

h0 =

2 [1 0 8 16 46 16 8 0 1] 64 2 [ 1 0 9 16 9 0 1] 32

f0 =

For completeness, the highpass analysis and synthesis filters are characterized by
h1 = 2 [ 1 0 9 16 9 0 1] 32 2 [1 0 8 16 46 16 8 0 1] 64

f1 =

Results: The signal-to-noise ratio (SNR) before and after de-noising are given by

72

s[i, j ] F SNR before = 20log10 = 15.2327 dB x[i, j ] s[i, j ] F s[i, j ] F SNR after = 20log10 = 22.2289 dB [i, j ] s[i, j ] F x
where x[i, j ]
F

is the Frobenius norm of matrix {x[i, j]} defined by

x[i , j ]

x [i, j ]
2 i =1 j =1

The original, noise-corrupted, and de-noised images are depicted in Fig. 7.10.

73

origimal image

noisecontaminated image

denoised image

Figure 7.10

References: 1. D. Donoho, I. Johnstone, G. Kerkyacharian, and D. Picard, Wavelet shrinkage: Asymptopia? J. Roy. Stat. Soc., vol. 57, pp. 301-369, 1995. 2. D. Donoho, De-noising by soft thresholding, IEEE Trans. Information Theory, vol. 41, pp. 613-627, 1995.
74