Multimedia Image Compression

1. Objectives
The objective of this program is to develop first-of-a-kind MPEG-4 architectures, optimised for high performance
and low power. The main output of the program are system-on-chip MPEG-4 architectures, system-level
implementation methodologies and tools supporting the design flow of this family of applications.

2. Background
Advanced multimedia image compression applications are characterised by media diversity and dynamic
resource availability, making the use of first generation image compression techniques not appropriate. To
overcome these inefficiencies, new image compression algorithms have been developed, and are being
standardised within the ISO/IEC MPEG-4 context.

MPEG-4 can indeed be considered as the first true multimedia standard, because it includes concepts such as
objects, scalability, scene composition, etc. As such, it will be the select standard for a variety of applications
ranging from mobile multimedia terminals to game applications and set-top boxes.

A MPEG-4 system no longer compresses images as a whole, but allows for 3-D scene composition of multiple
heterogeneous objects, e.g. video sequences of arbitrary shapes, face and body animated mannequins, wavelet-
based still images and wavelet-based textures. Mechanisms are provided to adapt to changing communication
bandwidth and bit error rate as well as to different (possibly changing in time) terminal resources. These
mechanisms include advanced error resilience, spatial-, SNR-, and temporal-based scalability and graceful
degradation operation modes. Additionally, MPEG-4 provides the necessary hooks for user interaction with the
3-D scene.

The challenges for designing a MPEG-4 architecture are being able to 1) cope with the high system complexity, 2)
to efficiently design the new functionality (e.g. wavelets), and 3) combine the flexibility requirements resulting
from the heterogeneous nature of MPEG-4 and from the user interaction, with the efficiency of application
specific architectures. Current research proves that the main bottlenecks for a MPEG-4 system are with the
memory requirements (memory size and bandwidth).

IMEC is consequently currently focusing on the development of methodologies and CAD tools for supporting
the implementation of MPEG-4 like systems, on the implementation of MPEG-4 scalability functionality, and on
the design of specific memory architectures for MPEG-4 systems.

IMEC has investigated for several years the use of microelectronics for complex digital signal processing
systems. In the field of multimedia image compression, the DESICS division has several realisations including the
design of a hardware motion estimator for a H.261 video-conferencing encoder, the development of a low power
architecture for a H.263 video-conferencing decoder and a colour transformation chip for region-oriented coding,
the development of a low-power architecture for a MPEG-4 video encoding, and the world first silicon for MPEG-
4 specific functionality (Ozone).

IMEC’s strategy includes an active participation in the MPEG-4 standardisation committee.

3. Target Customers
Target customers for this program are system houses and multimedia component providers that want to extend
their existing expertise with MPEG-4 specific implementation knowledge.

4. Program Outline
The program consists out of 6 work packages:

WP1: Application definition and analysis.

WP2: System-level optimisation.
WP3: Quality of Service and Graceful Degradation.
WP4: Design of MPEG-4 IP Blocks.
WP5: Advanced topics.
WP6: Demonstration

In this work program, it is assumed that the partner will define a limited set of MPEG-4 application scenarios at
the start of the project. Quantitative characteristics of the data streams, data sources and user interaction in this
application will be needed.

Every six months there will be a meeting to review this work plan and update it, if necessary.

WP1: Application definition and analysis

At the start of this work package, high level models and implementations of MPEG-4 functionality are assumed to
be available in the IIAP program. In this work package, an initial assessment of the memory, computation and
communication requirements for the different application scenario's is performed. For this purpose, ATOMIUM
advanced analysis tools are used.

WP2: System level optimisation

Based on the quantitative outcome of WP1, an initial architecture for the defined application(s) will first be
established. This architecture will be focussing on a selected scenario, but effort will be made to investigate
whether generic optimised architectural elements can be designed.

The main cost factors under consideration are performance and power consumption.

The ATOMIUM system level design methods and optimisation tools which are developed at IMEC will be used
to dimension and optimise the memory architecture and interconnections. Together with the definition of the
architecture, also the mapping of the functionality of MPEG-4 onto the architecture has to be performed.

WP3: Quality of Service and Graceful Degradation

One way to eliminate some of the bottlenecks that are defined in WP2 is the use of graceful degradation
techniques. When the architecture has insufficient processing, communication or memory resources, the quality
of the decoding (e.g. frames per second, resolution, SNR) can be traded for decoding complexity. The main
challenge in implementing the graceful degradation techniques is the treatment of the dynamic nature of the
decoding process, caused by the variable characteristics of the incoming data stream and the user interactions.

In this work package, we will first define complexity metrics and complexity prediction. Consequently, graceful
degradation strategies will be worked out. To this end, graceful degradation will be considered both at the
system level and the object decoder level. The developed techniques will mainly focus on 3D rendering, where
the expected impact is the greatest. These techniques shall be based on the information found in the MPEG-4
normative bitstream and on optional MPEG-4 meta-information.

WP4: Design of MPEG-4 IP Blocks

In WP 2, implementation bottlenecks are identified. Consequently, optimised implementations are necessary (in
hardware or software). It is the intention to process the hardware (if any) blocks as chips, as long as they are in
line with the research pipeline of the multimedia IIAP program.

Page 2 Multimedia Image Compression March 00

The HW IP blocks will preferably be developed by means of the OCAPI design flow, which support the
development of hardware from C++. The main advantage of OCAPI is the possibility to rapidly generate derived
versions (re-use) of the IP blocks.

WP5: Advanced topics

The goal of this work package is to monitor the functionality that has a high probability to be included in future
MPEG-4 versions. Given the partner’s interests and preliminary information on novel MPEG-4 versions, in
particular, early detection of implementation bottlenecks is aimed for.

WP6: Demonstration
In this last work package, a demonstration will be built for the results of this project. A possible implementation
would consist of a PC with a plug in card for the hardware accelerators, connected via a cable to a handheld
personal organiser on which the received information is displayed.

The demo will show the relevance of the architectural refinements (e.g. memory optimisations, graceful
degradation, etc.).

5. Deliverables
Participating in this program offers the following results to the industrial partner:
• assessment of specific memory architectures for MPEG-4,
• transfer of IMEC’s system-level implementation methodologies,
• access to ATOMIUM CAD for the design of MPEG-4 systems,
• insight in MPEG-4 graceful degradation techniques,
• assessment of low power techniques,
• insight in issues concerning integration of MPEG-4 systems with other state-of-the-art developments where
IMEC has an important research activity (e.g. broadband modems and wireless communications systems),
• up-to-date information on the MPEG-4 standardisation,
• access to implementation know-how for state-of-the-art wavelet compression,
• evaluation of the performances and feasibility of next-generation multimedia algorithms (e.g. wavelet-based
coding of video sequences).

Page 3 Multimedia Image Compression March 00

6. Program Timing
The following is an overview of the program planning:

Year 1 Year 2 Year 3

The major results of the IIAP are:

System-level implementation methodologies

The development of system-level implementation and optimisation methodologies bootstrap on DESICS'
yearlong tradition in the field of silicon compilation. With respect to multi-media, the main focus is on the
reduction of the data transfer and storage cost, which is the predominant cost factor, both with respect to
performance as well as power consumption. An overview of the developed ATOMIUM methodology can be
found at http://www.imec.be/vsdm/projects/atomium/ (over 50 scientific publications).

The main results of applying ATOMIUM on designs are:

1. The reduction of the (worst-case) power consumption of a H.263 videophone decoder with a factor 9
2. The reduction of the storage size with a factor 10 for a MPEG-4 visual texture encoder
3. The reduction of the (worst case) power consumption with a factor 8 for MPEG-4 motion estimation
4. The reduction of the number of memory transfers with a factor 9/8/4 (averages for low complexity/medium
complexity/high complexity video) for the MPEG-4 simple profile decoder (compared to MPEG-4 FDIS C
code)

Robust CAD tools formulti-media system exploration

Tool support for the above-mentioned methodologies is now actively being developed. The objective is to
implement robust tools, supporting the application of the system-level methodologies, focusing on the
requirements of multi-media applications and supporting the most error-prone and critical tasks concerning
design exploration and optimisation. The tools operate on functionality described in the C language.

Currently, following tools are being used for MPEG-4 design:

Page 4 Multimedia Image Compression March 00

• Dynamic pruning of unwanted MPEG-4 functionality: given the MPEG-4 reference code and a given MPEG-4
profile, this tool supports the removal of redundant code. For an MPEG-4 videophone encoder application,
the MPEG-4 reference code was reduced to only one fifth of its original size using dynamic pruning.
• Simulation-based instrumentation of the memory requirements (data transfer and storage) of MPEG-4
• Formal memory cost analysis of C code
• C-in, C-out memory compaction tools that support:
− Fully automatic intra-data optimisation and window calculation
− Fully automatic inter-data optimisation (placement)
− Advanced lifetime analysis
− The (possibility for) manual assignment of data to memory
− Designer imposed constraints (e.g., fix offset)
− Automatic transformation of the C code
− Address optimisations
This functionality minimises (the size of) memories and the number of capacity misses in caches

• OCAPI-1-1, to translate C descriptions of MPEG-4 functionality into VHDL

The following functionality is currently under development:

• Interactive code modification
• Automated support to optimise the physical memory management. Given a cycle budget, this functionality
minimises the cost (power consumption and silicon area) of the on-chip memories for custom architectures
by distributing the available cycle budget to reduce the required memory bandwidth and assigning the data
to the allocated memories in a cost effective way.

Implementation of optimised MPEG-4 functionality

Following implementations of MPEG-4 functionality have been designed:
• The Ozone, a chip for scalable encoding of wavelet images consisting of a zero-tree encoder and a high-
speed adaptive binary arithmetic encoder. The Ozone allows for real-time compression (30 fps) of CIF
images (352x288 pixels) and is the first chip specifically being designed worldwide for novel MPEG-4
functionality. Higher resolutions and frame rates are achievable by using parallel Ozone chips. The Ozone
has been demonstrated on a PC plug-in card.
• Optimised VHDL and C for the zero-tree encoder of the Ozone
• Optimised VHDL and C for the adaptive binary arithmetic encoder of the Ozone
• Optimised software implementing some selected profiles of the MPEG-4 standard (C language), more
specifically:
• Platform independent C code for the MPEG-4 simple profile decoder
• Platform independent C code for the MPEG-4 simple profile encoder
• PC Platform C code for the MPEG-4 simple profile decoder (achieving CIF 30 fps on a 350 MHz PC)
• PC Platform C code for the MPEG-4 simple profile encoder (achieving CIF 15 fps on a 350 MHz PC)
• Optimised software of the integer-to-integer lossless forward and inverse wavelet filtering (including
Pentium II MMX, SHARC multiprocessor, TI C6x and TriMedia TM1)
• Optimised Local Wavelet Transform C code, allowing for a block-based computation of wavelet filter
coefficients
• Optimised C and Java for the compression of artificial objects.

For more Information

Program Manager: Jan Bormans, IMEC, 75 Kapeldreef, B-3001 Leuven, Belgium, Tel.: +32 16 28 15 72,
Fax: +32 16 28 15 15, E-mail: Jan.Bormans@imec.be
IMEC web site: http://www.imec.be/

Page 5 Multimedia Image Compression March 00