Classification and coding of contour information

Diogo Cortez, Manuel Menezes de Sequeira, Fernando Pereira

INSTITUTO SUPERIOR TÉCNICO - Secção de Telecomunicações
Av. Rovisco Pais, 1096 Lisboa Codex - Portugal, email: ddcf@tele1.ist.utl.pt

Abstract

In order to improve the performance of the classical pixel-based image coding techniques, new tech-
nical approaches are deemed necessary. These approaches aim at a better understanding of the image
content, in terms of regions, textures, etc., thus “healing the blindness” of the old schemes.
The new image coding approaches require the identification of regions or “objects”, i.e., image
segmentation. This process results in an image partition matrix, which includes contour information,
and a set of parameters characterising each region according to a given set of image features.
The contour information represents an extremely important and sensitive information for the human
visual system, being thus also of great importance in the image coding context.
This paper proposes a classification of partition and contour models according to a tree structure,
in order to clearly identify the contour models to which contour coding techniques can be applied. It
also proposes a lossless contour coding technique based on Freeman’s chain codes [5].

Keywords: partition matrix, contour matrix, contour coding, chain coding, crossing.

1 Introduction

Since the first appearance of digitised images in the beginning of the twenties [6], the main challenge that
image processing researchers have faced is the enormous amount of information to process. The evolution
of digital technology which took place during the last three decades, however, allowed the development and
implementation of image processing techniques with increasing complexity. Thus, the use of digital image
started to spread rapidly.

The large amount of information associated with images and video lead to the need to develop efficient coding
algorithms, able to drastically reduce the size of stored or transmitted information, while maintaining good
subjective quality. Most of the image coding techniques developed until now have been essentially pixel-
based (e.g., hybrid-coding, transform coding, and subband coding). These techniques seem to have now
reached a performance plateau.

In order to improve the performance of the classical pixel-based coding techniques, new approaches were
deemed necessary. These approaches are based on a better understanding of the image content, in terms of
regions, textures, “objects”, etc., thus evolving from the “blindness” of the old schemes. Simultaneously, new
user needs have sparkled the research in new services and functionalities, mostly related to the image/video
content manipulation capabilities. These new perspectives led to the establishment of ISO MPEG-4 (Moving
Picture Expert Group - 4), whose main objective is to define a standard supporting universal communication,
access, and manipulation of digital audio-visual data. Some of the most important new functionalities being
aimed at by MPEG-4 require access to the audio-visual content [?]. The new standard is due in 1998.

The new image coding approaches usually require the identification of regions which are more or less
homogeneous in terms of a given set of image features (e.g., luminance, texture and/or movement), thus
aiming at a deeper image understanding. This process, usually known as segmentation, results in an image
partition matrix and a set of parameters characterising each region in the partition according to the selected
features.

The position and shape of the contours in a partition matrix represent an extremely important and sensitive
information both for the human visual system and for the coding performance [8]. Hence, it is fundamental
to study and develop contour coding techniques that allow an efficient storage or transmission of contour
information.

1
Although contour coding performance is strongly dependent on the coding technique used, it is also closely
related to the properties of the partition matrices whose contour information is to be encoded. This paper
proposes a classification of partition and contour models according to a tree structure, which clearly identifies
the contour models to which contour coding techniques can be applied.

This paper also proposes a lossless contour coding algorithm based on Freeman’s chain codes [5]. The algo-
rithm is applied to one of the contour models defined. The results are evaluated according to a methodology
which is based on the estimated number of bits spent encoding the contours.

2 Contour information classification

2.1 Basic definitions

A partition matrix contains information about the different regions of the image, which have been identified
through a segmentation process. Regions are identified by assigning a discriminating label to their pixels:

Definition 2.1 – Region: is the set of all pixels in a partition matrix having a given label. 2

Depending on the segmentation process, regions can be connected or disconnected. Thus, it is useful to
introduce an intermediate concept between region and pixel:

Definition 2.2 – Zone: each of the largest sets of connected pixels having the same label (a zone is
therefore equivalent to what is usually called a connected component). A connected region always coincides
with its unique zone. 2

Definition 2.3 – Connectivity: a set of pixels is n-connected if there is at least one n-path in the set
between any pair of pixels in the set. 2

Definition 2.4 – Path [6]: a n-path from pixel p with coordinates (xp , yp ) to a pixel q with coordinates
(xq , yq ) is a sequence of distinct pixels with coordinates (x0 , y0 ), · · · , (xL−1 , yL−1 ) where (x0 , y0 ) = (xp , yp ),
(xL−1 , yL−1 ) = (xq , yq ), and where (xi , yi ) is always a n-neighbour of (xi−1 , yi−1 ) (i = 1, · · · , L − 1). L is
the path length. 2

The definition of n-neighbourhoods, for several types of lattices, can be found in [10].

It is now possible to define partition and contour matrices:

Definition 2.5 – Partition matrix: is a lattice of M × N elements (pixels) with the same structure as
that of the originating image. Each element has a label value between 1 and R. R is the maximum number
of regions in the partition. If R = 2, the partition matrix is binary. If R > 2, the partition matrix is mosaic.
2

Definition 2.6 – Contour matrix: a contour matrix is obtained from the contour information in a
partition matrix. Its lattice is a superset of the partition matrix lattice, with edges and vertices aside
from the pixels (edges, vertices, and pixels are known as contour matrix elements). If a rectangular lattice
is used in a partition matrix with M × N pixels, see figure 1a), the corresponding contour matrix has
(2M + 1) × (2N + 1) elements, see figure 1b). Contour matrix vertices and edges can be labelled with only
two values: 0 or 1. If a contour matrix vertex or edge has two neighbour pixels with different labels in
the partition matrix, it is labelled 1. All other contour matrix vertices and edges are labelled 0. Contour
matrix pixels are marked with the same label as the corresponding pixel in the partition matrix (between 1
and R) if any of its neighbours belongs to a different region. All other contour matrix pixels are labelled 0.
Thus, contour elements labelled 0 do not belong to any contour. Contour matrix elements (vertices, edges,
and pixels) at the border of the contour matrix lattice are always considered as part of a contour, and thus
are never labelled 0. All contour matrix elements labelled l 6= 0 are named contour elements (also contour
vertices, contour edges, and contour pixels). 2

2
 a) b)

Figure 1: Rectangular lattice: a) Partition matrix; b) Contour matrix. 2 → pixel (white is label 0); — or
| → edge, and ◦ → vertex (thin is label 0, thick is label 1).

Definition 2.7 – Contour: a set of connected contour edges and vertices, all with label 1, is a contour
defined on the vertices and edges. A set of connected contour pixels, all with the same label l 6= 0, is a
contour defined on the pixels. 2

Notice that, by the above definition, a contour can enclose several zones from several regions. Also, there
can be several different contours in a contour matrix, eventually enclosing different zones from the same
region (if it is a disconnected region, of course).

Contours may be uniquely specified only by using contour edges. Contour vertices will be used here only
for the definition of crossings:

Definition 2.8 – Crossing: is a contour vertex with at least three neighbouring contour edges (labelled
1). The number of such neighbours minus one is the order of the crossing (2 or 3 in a rectangular lattice, 2
in a hexagonal lattice). 2

Definition 2.9 – Pixel crossing: is a contour pixel with at least three contour pixel neighbours with
the same label. The possible pixel crossing orders depend on the neighbourhood system considered: for
4-neighbourhoods, second and third orders are possible; for 8-neighbourhoods crossings, from the second to
the sixth order may occur. 2

2.2 Contour matrices models

Contour coding performance strongly depends on the technique used. This performance depends also on
the properties of the partition matrices whose contour information is to be encoded. Consequently, it
is necessary to clearly classify the partition matrices models according to their properties and to define
corresponding contour matrices models that will be used by the contour coding techniques.

This paper proposes a classification of partition and contour matrices models according to a tree structure,
shown in figure 2 for rectangular lattices. This classification has two main levels:

• Partition properties – related to the partition matrix.

• Contour representation and properties – related to the contour definition.

In the first level, four main properties are considered:

1. Partition matrix lattice type – e.g., rectangular or hexagonal.

2. Partition matrix type – binary or mosaic.
3. Partition matrix zone connectivity – type of connectivity used (e.g., 4-, 6- or 8-connected).

3
 Rectangular lattice (R)

Partition Binary matrix (B) Mosaic matrix (M)

matrix
properties

4-connected 8-connected 4-connected 8-connected

zone (4) zone (8) zone (4) zone (8)

Connected Disconnected Connected Disconnected Connected Disconnected Connected Disconnected

regions regions (D) regions regions (D) regions regions (D) regions regions (D)
(C) (C) (C) (C)

Contours: Contours: over Contours: Contours: over Contours: Contours: over Contours: Contours: over
Contour matrix over the the edges and over the the edges and over the the edges and over the the edges and
representation pixels the vertices pixels the vertices pixels the vertices pixels the vertices
and properties (P) (E) (P) (E) (P) (E) (P) (E)

4-n (4) 8-n (8) 6-n 4-n (4) 8-n (8) 6-n 4-n (4) 8-n (8) 6-n 4-n (4) 8-n (8) 6-n
(6) (6) (6) (6)

Figure 2: Rectangular lattice. Contour classification according to a tree structure.

4. Partition matrix regions connectivity – connected or disconnected regions.

The level of contour representation and properties enables a distinction between the contour types in the
contour matrix. Thus, contours can be classified according to two properties:

5. Contour elements definition – over the pixels or over the edges and the vertices of the contour matrix.
6. Contour elements neighbourhood (pixels or edges) – e.g., 4-, 6- or 8-neighbourhood.

When the partition properties and the corresponding contour representation and properties are established,
a contour matrix model is completely defined.

3 Contour coding techniques

During the last three decades, considerable attention was devoted to the problem of edge detection and
to the problem of representing (and manipulating) efficiently curves defined on a discrete grid. However,
few works were dedicated to contour coding. It is thus very important to identify which of the techniques
developed can be useful for contour coding.

Contour coding techniques can be distinguished according to the preservation of information – lossy or
lossless – and to the contour representation used:

• Space domain, such as chain codes [5] or polynomial approximation [9].

• Frequency domain, such as Fourier descriptors [11].

As referred in [3], the discrete Fourier transform used to generate the Fourier descriptors from a parametric
curve description of a contour, can be very expensive in terms of bitrate since the coefficients (i.e., the
Fourier descriptors) must be encoded with a high accuracy.

Freeman’s chain codes [5] and the skeleton representation and coding [7] are lossless techniques, while poly-
nomial approximations are in general lossy techniques. As happens with transform coding, the polynomial
approximation can be used for contour shape simplification, as suggested by [7].

While skeleton representation and coding, being a lossless technique, enables the exact contour reconstruc-
tion, small variations in contour shape can drastically increase the skeleton’s complexity.

In the following section the fundamentals of Freeman’s chain codes are reviewed, and a contour coding
technique based on this well known method is proposed.

4
3.1 Freeman’s chain codes

Chain codes, proposed by Herbert Freeman in 1961 [5], allow lossless encoding of arbitrary geometric
configurations.

Let a contour be approximated by a polygon. The faithfulness of the approximation increases with the
number of polygon segments. For the limit situation, when the number of segments is equal to the number
of contour elements, the approximation is exact. In this context, Freeman’s chain codes are a particular case
of polygonal approximation. Considering Freeman’s chain codes and the contour matrix models proposed
in this paper, two new concepts must be defined: bond and chain.

Definition 3.1 – Bond: is a vector that connects two neighbour contour elements. The possible vector
directions depend on the lattice and on the contour matrix elements used. 2

Each bond is associated with a symbol, which represents its direction. A sequence of these symbols defines
a chain.

Definition 3.2 – Chain: is a sequence of bond symbols, and other special symbols that convey information
about the contour absolute position and the occurrence of crossings. 2

In the context presented, the construction of chain codes can be summarised as follows:

1. Find the next initial contour element, e0 (of a not already processed contour). If it does not exist, the
current contour is complete.
2. Encode the initial element e0 coordinates.
3. Find the next contour element ei , by searching in the directions allowed by the contour matrix model
used. If it does not exist, go to 1.
4. Encode the bond between ei−1 and ei , that is, store its direction in the chain code.
5. Go to 3.

Initial contour elements are the only ones for which coordinates are encoded. In fact, these initial coordinates
contain information about the absolute position of the contours, while all the other symbols in the chain
code contain shape information.

Freeman’s chain codes do not use the concept of crossings, since they were developed mainly for “the
encoding of arbitrary geometric configurations so as to facilitate their analysis and manipulation by means
of a digital computer” [5].

A simple modification to this algorithm, consisting on encoding the change of direction between two consec-
utive bonds, reduces both the number of possible symbols and the entropy of the total alphabet of symbols,
thus improving the coding efficiency [2].

3.2 Contour coding algorithm

This section presents a lossless contour coding algorithm, based on Freeman’s chain codes, that considers
the contour matrices models described in section 2.2. In particular, the model RM4CE6 will be con-
sidered (according to figure 2 terminology, rectangular partition matrix lattice, mosaic partition matrix,
4-connected zones, connected regions, contours defined on edges and vertices, and 6-neighbourhoods for
contour elements).

3.2.1 Introduction

Encoding contours in their entirety (see definition 2.7) reduces the need for encoding absolute position
information, since each contour in a partition matrix will require a single initial symbol. However, this

5
 1 2
8 3

7 4
10 9 6 5
11 16

12 15
13 14

Figure 3: Model RM4CE6: a pseudo-crossing (•).

makes it necessary to deal and encode properly crossing information (which can be seen as a clever way of
encoding initial symbols in an implicit and relative, rather than absolute, way).

When a contour contains crossings, there can be, usually, several different ways of encoding it, some better
than others. The best ways of tracing a contour often correspond to what might be called the “drawing
without lifting your pencil and without repetitions” game. The less one needs to lift the pencil, the less
crossing symbols need to be encoded. Hence, the more effective the encoding will be, since crossing symbols
usually are quite “expensive” to encode (though less than initial symbols). The optimisation of contour
tracing is a hard problem, actually, it is the so-called generalised “Königsberg bridges” problem [1]1 . Hence,
a simple heuristics is proposed here, which tends to avoid encoding pseudo-crossings such as the one shown
in figure 3: when the encoding algorithm finds a crossing, it attempts first to go ahead, without turning.
The example in the figure illustrates this:

Example 3.1 The contour considered in figure 3 can be chain encoded in many different ways. Two of
them are here considered:

• 1, 2, 3, 4, 5, 6, Crossing, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16.

• 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 16, 7, 8.

The second option leads to less coded information, due to the presence of a triple crossing that does not
need to be encoded (it is a pseudo-crossing). 2

3.2.2 Encoding strategies

Contours in matrix limits: When coding contour elements in the contour matrix boundary, attention
should be paid to the fact that these elements cannot establish as many bonds as for contour elements
inside the matrix. Thus, special symbols should be used in this situation, since the total number of possible
symbols is reduced.

If contour elements in contour matrix boundary are not encoded, contours defined on edges and vertices
can have “dead-ends”. However, this causes no problems, since the decoder can immediately detect such
situations and proceed accordingly. In fact, the encoding of contour elements in the contour matrix boundary
seems to be a rather wasteful decision.

Contours position information: When contour elements in the contour matrix boundary are not en-
coded, the selection of initial elements should always be attempted first on the image boundaries. The
reason for this is that selecting any other contour element would lead to either an increased number of
crossings or to contour splitting. Also, the border initial elements’ position can be easily differentially en-
coded, eventually using run-length codes, thus reducing substantially the bit overhead for initial symbols.
This is true, because contour matrices often have a lot of boundary “dead-ends”. This, however, requires
the use of extra information (1 bit) for discriminating boundary initial elements from the regular ones.
1 Whose complexity is comparable to the one of the “travelling salesman” problem.

6
 MAVT Carphone Flower Garden
Initial Boundary Initial Boundary
Regions symbols initial Regions symbols initial
symbols symbols
1582 18 4 2843 54 13
242 11 5 843 38 9
40 14 5 190 58 16
a) b)

Table 1: Comparison between the number of regions in the partition matrix and the number of generated
initial elements: a) MAVT Carphone; b) Flower Garden.

Crossing information: During the tracing process, when a crossing is found, the information associated
to its order, and the direction of its entry and exit bond must be stored in a FIFO memory. Sometimes this
information does not need to be encoded:

1. For contours defined on the pixels, if a contour pixel is the exit of several crossing pixels, it is sufficient
to encode only one of these crossings.
2. For contours defined on edges and vertices, encoding of pseudo-crossings is not necessary.

3.2.3 Algorithm outline

In this section the proposed contour coding algorithm is briefly outlined for contours defined on edges and
vertices and considering that the contour elements in the matrix boundary are not encoded.

1. Find the initial contour element. The search should start on the contour matrix boundary, as explained
before (section 3.2.2).
2. Trace the contour, from the initial element on. In case of crossings, try to exit first ahead, then right
and finally left.
3. When a crossing is found, information regarding its order and entry and exit directions should be
stored in the FIFO memory.
4. When the tracing reaches the initial element, a matrix boundary, or a previous crossing, extract a
crossing from the FIFO memory (this is a true, not pseudo, crossing). Decide whether the new exit
direction needs to be encoded. If it does, encode it. If the FIFO is exhausted, the current contour
ended. Otherwise continue to trace the contour.

4 Results

The contour coding technique outlined in the previous section has been applied to the contour matrices
model RM4CE6, as defined before.

The partition matrices used in this section were obtained applying the segmentation algorithm proposed
in [4] to two different types of sequences (frame 0): mobile videotelephony, viz., MAVT Carphone, and
TV-like, viz., Flower Garden, allowing the evaluation of the encoding technique in two different situations
(refer to the paper for the values of the segmentation algorithm parameters). Results were taken from the
segmentation algorithm at the end of the merge step, after small regions elimination, and at the suggested
final number of regions for these sequences (40 regions for MAVT Carphone and 190 regions for Flower
Garden). Figures 4 and 5 show the obtained partition and (RM4CE6) contour matrices.

The savings in bits spent encoding the initial contour elements’ positions, when crossings are used, can be
seen in table 1: the number of encoded initial elements is much smaller than the total number of partition
regions.

The encoding results were evaluated according to the estimation of the total number of bits spent encoding
the contour information. The number of bits spent encoding initial elements was added to the entropy of

7
 a) b) c)

Figure 4: MAVT Carphone. Partition and contour matrices with: a) 1582 regions; b) 242 regions; c) 40
regions.

a) b) c)

Figure 5: Flower Garden. Partition and contour matrices with: a) 2843 regions; b) 843 regions; c) 190
regions.

8
 F lower Garden

1.7
2843 regions
1.6 843 regions
190 regions
1.5
[bit\symbol]

1.4
Entropy

1.3

1.2

1.1

1
0 1 2 3 4 5 6 7

a) b)
Markov’s source order

Figure 6: Symbol stream entropy for: a) MAVT Carphone; b) Flower Garden.

the symbol stream, for bonds and crossings, multiplied by the total number of bond and crossing symbols.
Table 2 shows the obtained entropy and total information values, when a memoryless source is considered
for entropy estimation. This table also shows that the bits spent encoding initial elements and crossing
information accounts for a small percentage of the total. Hence, most of the encoding efforts from this
point on, should concentrate of how to efficiently encode the bond symbols, using the Freeman’s definition
of contour segments [2].

MAVT Carphone
Number of Number of Entropy Bond Crossing Initial Total
Regions bond crossing [bit/symbol] information information elements information
symbols symbols (bonds and [bit] [bit] information [bit]
crossings) [bit]
1582 17125 2053 1.5774 27013.3 3238.4 314 30565.7
242 13522 389 1.3091 17701.7 509.2 174 18384.9
40 5731 40 1.2785 7327.1 51.1 231 7609.2
a)
Flower Garden
Number of Number of Entropy Bond Crossing Initial Total
Regions bond crossing [bit/symbol] information information elements information
symbols symbols (bonds and [bit] [bit] information [bit]
crossings) [bit]
2843 29116 4064 1.6286 47418.3 6618.6 935 54971.9
843 22037 1401 1.4654 32293.0 2053.0 659 35005.0
190 11263 215 1.3281 14958.4 285.5 990 16233.9
b)

Table 2: Estimation of entropy and total information for: a)MAVT Carphone; b)Flower Garden.

If a source generates a stream of statistically dependent symbols, i.e., the source has memory, it is possible
to model it using a class of random processes known as discrete stationary Markov processes [12]. Figure 6
shows the evolution of the estimated entropies when the bond and crossing symbols in the stream are
considered to depend on an increasing number of preceding bond and crossing symbols. These results show
the high statistical dependency between symbols, and hence the advantage of using predictive encoding
schemes depending on an increasing number of previous symbols.

5 Conclusions and future work

This paper proposed a partition and contour model classification according to a tree structure. The tree
is divided in two main levels: the first level defines partition matrix models and the second defines the

9
corresponding contour matrix models.

A contour coding algorithm is also proposed. Results have been obtained for two different types of video
sequences: mobile videotelephony and TV-like. These results show the advantage of using the concept of
crossings and also of using predictive techniques to explore the chain codes’ memory.

The following topics still require further research:

• Contour chain codes based on Freeman’s definition of contour segments [2], which make use of the
symbol stream memory to further reduce the entropy.
• Contour coding using simplification of contour information (lossy algorithms or pre-processing tech-
niques).
• Extension of the contour coding technique so as to encode efficiently the evolution of partition matrices
along a sequence of images.

References
[1] Richard Bellman and K. L. Cooke. The Königsberg bridges problem generalized. Journal of Mathe-
matical Analysis and Applications, 25:1–7, 1969.
[2] Michael James Biggar. Source Coding of Segmented Digital Image and Video Signals. PhD thesis, De-
partment of Electrical Engineering, Imperial College of Science and Technology, University of London,
October 1987.
[3] Patrick Brigger, Antoni Gasull, Chuang Gu, Ferran Marqués, Fernand Meyer, and Christophe Oddou.
Contour coding. CEC Deliverable R2053/UPC/GPS/DS/R/006/b1, EPFL, UPC, CMM, LEP, De-
cember 1993.
[4] Diogo Cortez, Paulo Nunes, Manuel Menezes de Sequeira, and Fernando Pereira. Image segmentation
towards new image representation methods. Signal Processing: Image Communication, 6(6):485–498,
February 1995.
[5] Herbert Freeman. On the encoding of arbitrary geometric configurations. IRE Transactions on Elec-
tronic Computers, 10:260–268, June 1961.
[6] Rafael C. Gonzalez and Paul Wintz. Digital Image Processing. Addison-Wesley Publishing Company,
Inc., Reading, Massachusetts, 1987.
[7] Petros A. Maragos and Ronald W. Schafer. Morphological skeleton representation and coding of binary
images. IEEE Transactions on Acoustics, Speech and Signal Processing, 34(5):1228–1244, October 1986.
[8] Ferran Marqués, Josep Sauleda, and Antoni Gasull. Shape and location coding for contour images.
In Proceedings of the Picture Coding Symposium (PCS’93), page 18.6, Lausanne, March 1993. Swiss
Federal Institute of Technology.
[9] Frederic I. Parke. Parameterized models for facial animation. IEEE Computer Graphics & Applications,
2:61–68, November 1982.
[10] Theodosios Pavlidis. Structural Pattern Recognition. Springer-Verlag, Berlin, 1977.
[11] Eric Persoon and King-Sun Fu. Shape discrimination using fourier descriptors. IEEE Transactions on
Systems, Man and Cybernetics, 7(3):170–179, March 1977.
[12] K. Sam Shanmugam. Digital and Analog Communication Systems. John Wiley & Sons, Inc., 1985.

10