Second Workshop on Digital Media and its Application in Museum & Heritage
Virtual Presentation and Animation of Qingming Festival by The Riverside
Zhigeng Pan, Gengdai Liu and Zhi Li
State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China
Ningbo Institute of Technology, Zhejiang University, Ningbo, China
{zgpan,liugengdai}@cad.zju.edu.cn, iback1121@yahoo.com.cn
Abstract
In this paper, we present a novel method to display
a famous ancient Chinese painting for the purpose of
cultural heritage preservation. The framework of
multi-projector display wall based digital storytelling
system is introduced. The aim of this system is to
exhibit the ancient painting on a large display wall
dynamically. Before being projected, the painting has
to be repainted with very high resolution and
segmented into many smaller tiles, so that it can be
displayed on the large flat display wall. Additionally,
in order to make this still painting attractive and
living, the technique of digital storytelling is used. So a
script called water transportation story is drafted to
tell a story about the prosperous city. We extract the
movable objects from the original painting and redraw
them with three dimensional figures, and then the
computer animation technology is used to animate
them.
1. Introduction
In this paper, a multi-projector display wall based
digital storytelling system is presented. This system
involves displaying a famous ancient Chinese painting
on a large display wall dynamically [1]. This famous
ancient painting, Qingming Festival by The Riverside
that was painted in Song dynasty about 1000 years ago
is a rare realistic artwork and thought to be a world-
class cultural relic. It is about 500 centimeters long and
25 centimeters wide. There are more than 500 human
figures, 60 animals, 100 trees, 20 wooden boats, 30
buildings and pavilions and about 20 vehicles in this
painting. It gives audience a bird’s eye view of the
flourishing scene of Bianliang city, the capital of Song
dynasty. It was completed from a panoramic view and
detailed scenes along a river that stretched for dozens
of miles were covered. At present, the original version
of this painting is kept in the National Palace Museum
in Beijing, China.
0-7695-3065-6/07 $25.00 © 2007 IEEE 102
DOI 10.1109/DMAMH.2007.75
There are some examples of physical reconstruction
of this painting in China, such as Qingming Gardens in
Kaifeng and Movie Town in Hengdian. They are both
large parks made after the fashion of the scene in the
painting.
But we use a distinct method to display this painting.
Firstly, a multi-projector display wall system is created
for displaying it. And then the painting, Qingming
Festival by The Riverside is repainted by artists and
then is projected on a large display wall. The music
and commentary is played synchronously as the video
goes on so that he flourishing scene of the Song
dynasty’s capital appears before audience vividly.
It is necessary to make the objects on this still
painting moving, so as to make this famous painting
more lively. Therefore, we use IBR (Image Based
Rendering) and computer animation technology to
create a 2D picture with moving 3D moving objects on
it and display the animation on a large screen. In
addition, we introduce the concept of digital
storytelling into our system. A script is composed to
tell a story about the city, which consists of 11
scenarios.
In the next section, the related works about digital
cultural heritage which is similar to us are introduced.
And in the third section, we described our method for
virtual presentation and animation of the large painting.
In the third part of this paper, we give some final
results of our method. Finally, the conclusion and
future work is discussed.
2. Related Works
The technology of digital museum is an effective
method to preserve those natural and cultural heritages.
More and more specialists in heritage preservation and
computer science are engaged in this field. Hu [2]
made a summary of the application of virtual reality
and digital media to digital heritage preservation.
As for the current research situation of digital
cultural heritage preservation, there are four main
aspects focused by researchers all over the world. The
first one is the digitization, storage and display of
cultural relics. The second aspect involves virtual
museums on the Internet. Kwon [3] presented the key
techniques of digital museum on the internet, and a
general scheme of the modeling, management, storage
and transmission of large quantity of data of digital
museum. The third aspect is concerning how to create
the avatar-based multi-user immersive virtual museum.
Usaka [4] presented the prototype system of an
immersive virtual museum. The terminal users can
enter the virtual museum as avatars via Internet to visit
all the exhibits in it and can communicate with other
users. The forth aspect is the application of large
display projection in the real museums. Ding [5]
presented a large screen projection system using 3
projectors to display an ancient street in Hangzhou,
China. Our work can fall into this category. But in our
work, the most important distinction is that the display
wall is much larger and much more projectors are
employed.
IBR (Image Based Rendering) is a kind of new
technique to render 3D virtual scenes from 2D images.
It has been applied to many projects. Hirose [6]
introduce many IBR applications to digital museum.
Youichi [7] present a novel method to tour into the
picture by making animation from a single picture
using spidery mesh. In our system, we use Youichi’s
method to create animation by moving the viewpoint.
3. Methods for Virtual Presentation and
Animation
3.1. Outline
Figure 1 shows a part of the original painting, which
is very long. There are many interesting story hiding
behind the scene of the painting. Therefore, we can
compose a story consists of many scenarios from this
single painting.
Figure 1. Qingming Festerval by The Riverside
Figure 2 shows the procedure of our method. In the
first step, we invited some professional artists to
redraw the ancient painting above with very high
resolution. The new version of the painting is more
realistic, but keeping the style and fashion of the
original painting. In the second step, we selected 11
different parts of this painting to be the main places
103
where the story began. As a result, a long story with 11
scenarios was plotted. After the story was done, we
could create the animations. First of all, the TIP
technique [7] was used to enable the viewpoint to
move, so the audience could feel like walking into the
painting and touring all over in it. The three steps
discussed above are implemented offline.
After the offline work was done, we got a video
containing the story. When the video is playing, the
every frame is segmented and projected on the large
display wall.
Figure 2. Procedure of our method
3.2. Projection system
The display wall in the museum is 6 meters high
and 12 meters wide. We have to design a multi-
projection screens based projection system in terms of
the size of the wall. Therefore a PC-based multi-
channel VR projection system is employed in order to
project a high-resolution storytelling video on so large
a display wall. Considering the area of the wall, the
distance between the wall and the projectors and
brightness those projectors can give off, we decided to
use 15 projectors, 16 high-performance PCs and a
clustering multi-channel switch.
Those 15 projectors compose a 5*3 projectors
matrix, and accordingly the whole screen is split into
15 rectangular tiles, each of which is covered by the
image from one projector. C/S architecture can be used
in this projection system. The video is played and
every frame is segmented into 15 tiles in the PC server.
Then each segment is transmitted to the corresponding
client PC through switch synchronously. Each client
connected with a projector is responsible for rendering
one tile and projecting this tile to the proper position
on the large wall. The architecture of projection system
used in our system is shown in Figure 3.

Figure 3. The framework of projection system
In order to guarantee that the video on the large wall
is consistent and these 15 tiles are seamlessly stitched,
geometric calibration must be used. But the manual
calibration is boring, especially for so many projectors,
because the position and orientation of every projector
should be adjusted. So we decided to use software
geometric calibration to stitch all the tiles seamlessly
and automatically. Multiple mapping matrices are used
for tiled display during software geometric calibration
as Zhou described [8]. Multiple matrices are computed
for each projector to map each pixel on the projected
screen to the frame buffer. The method goes as follow:
firstly, feature points for each projector are displayed
alternatively. Secondly, feature points for each
projector are taken photos. Then, the calibration
algorithm starts to recognize the feature points and
calculate the mapping matrices and the effective
projection regions. Multiple mapping matrices method
means that one matrix on each calibration point must
be calculated and then the corresponding point in the
frame buffer can be found. Then all other pixels will be
mapped to the frame buffer like texture mapping. This
method guarantees that there’s no overlap region and
gaps between two adjacent projectors. The
configuration of this system has to be high, especially
the memories and network cards of PC server and
throughput and bandwidth of the multi-channel switch
due to the large quantity of data.
In our system, every frame of the video must be
segmented into 15 tiles and all the data of these tiles
have to be transmitted through the switch, which put
heavy burden on the PC server and the multi-channel
switch. So we have to reduce the resolution of the
video. However, because the distance between the wall
and the audiences is far enough, the question caused by
lower resolution can be ignored and the final result is
quite acceptable.
104
3.3. Creation of Animation
To make the objects in the painting movable, we
extracted them from the 2D background and animate
them. If the objects were solid ones, we just translated
and rotated them in terms of requirement. If the objects
are characters, including humans and animals, we use
skeleton-based character animation to drive them.
Figure 4 shows the skeleton (green) and skinned mesh
(purple) of a working people.
Figure 4. Skeleton-based character animation
To move the viewpoint when the animation goes on,
we use TIP techniques to create spidery meshes firstly.
Because the painting was repainted with panoramic
view and the perspective relationship is kept, the
spidery meshes can be generated easily.
Additionally, in order to give audience the feeling
of historical grandeur, the hue of the image is dim and
yellowish. And in order to combine 3D objects with
2D background harmoniously, textures, lights and
shadows are created and edited elaborately.
3.4. Storytelling Design
In order to make this static painting attractive and
charming, the technology of digital storytelling is used.
So a script called Qingming Story is composed
elaborately by us to show the magnificent scene along
the river. This story is composed of many scenarios.
For example, in the first scenario called The Dock,
many porters carried goods onto the boats by the
riverside, and then boatmen ferried the goods along the
river. There are some other locomotive objects in the
river or by the riverside, such as carriages, mill wheels,
pedestrians and other workers as shown in Figure 5.

Figure 5. Turning wheel and people working with it
Many other scenarios then follow, such as Listening
to Music in The Pavilion and The Shadow of Pagoda,
etc. Each scenario tells a historic story happened in the
city along the river.
4. Implementation Results
The final result of one of the scenarios is shown in
Figure 6. In this figure, a snapshot of the large display
wall is captured.
Figure 6. Final result
5. Conclusions
In this paper, we introduce a multi-projector large
wall based digital storytelling system applied to
cultural heritage exhibition. In this system, 15
projectors, 16 high-performance PCs and a multi-
channel switch are used for projecting a famous
ancient Chinese painting on a very large display wall.
We use software calibration method for all the
projectors in stead of manual calibration because the
former can lead better result. We repaint the original
painting and put some locomotive 3D objects on it to
make the scene more lively and magnificent. To
discover the story hidden behind the famous painting, a
script composed of 11 scenarios is plotted.
The system will continue to be implemented in the
future. At present, our system can only play animation
video and there is no interactive capability. In the real
implementation, we intend to make this system
interactive. Another point could be improved lies in the
105
architecture of the projection system. We can change it
to meet the problems caused by simple C/S
architecture.
References
[1] Z.Li, G.D.Liu, Z.G.Pan. Story of Ancient Water
Transportation: Design of A Virtual Presentation System.
The 4th Workshop on Digital Museum and Digital Heritage,
Kaifeng, 2006.
[2] W.X. Hu, Z.G. Pan, etc. Digitized World Heritage
Protection Methods. Journal of System Simulation. Vol.15.
No.3.2003, pp.315-318.
[3] Y.M.Kwon, I.J.Kim, etc. Virtual Heritage System:
Modeling, Database & Presentation. In Proc. of VSMM’01,
IEEE Press, 2001.
[4] T.Usaka, K.Sakamura. A Design and Evaluation of the
Multi-User Virtual Environment Server System for the
Digital Museum. In Proceedings of the 13th TRON Project
International Symposium, IEEE Press, 1996, pp60-69.
[5] J.H.Ding, Y.G.Wang, Z.G.Pan. Hefang Street
Walkthrough Based on Large - Scale Projector System.
Journal of Hangzhou Dianzi University, Vol.25.No.1.2005,
pp.56-59.
[6] M.Hirose. Image-based virtual world generation. IEEE
Multimedia. IEEE Press, 1997,4 (1) , pp.27-33.
[7] H.Youichi, A.Kenichi, A.Kiyoshi. Tour into the picture:
Using a spidery mesh interface to make animation from a
single image, Proceedings of the 24th Annual Conference on
Computer Graphics and Interactive Techniques, New York,
1997,pp.225-232.
[8] R. Zhou, H. Lin, J.Y. Shi. Geometric Calibration on Tiled
Display Using Multiple Matrices. In Proceeding of Pacific
Graphics ,2005, pp.97-99.