Tele-Immersion will enable users at

geographically distributed sites to collaborate in

real time in a shared, simulated environment as if
they were in the same physical room.
 The idea of Tele-immersion was proposed by Jaron
Lanier, the head of Advanced Networks and
Services in 1997. National Tele-Immersion Initiative
(NTII) was formed to implement this.
 In May 2000, researchers at University of North
Carolina (UNC), the University of Pennsylvania and
Advanced Networks and Services developed this
technology.
 A user sitting in office at UNC was able to see
lifelike 3D images of his colleagues hundreds of
mile away.
 Bank of digital cameras
capture participants from
a variety of angles.
 There are about 7-15
camera clusters each
contains 4 cameras out of
which 3 are black and
white and one is color.
 B&W cameras are for
stereo construction.
 The color camera is for
extracting the appearance.
Three Camera Views
Each camera is synchronized to other cameras in
the same bus.
Each camera unit is connected to a workstation.
Machines in the workstation are dual boot with
Linux and Windows 2000.
The scene is illuminated by imperceptibly
structured light.
4 microphones and 4 speakers are used for
sound.
Each set of images taken at a given instant is
sorted into subsets of overlapping trios of
images.
From each trio of images a disparity map is
calculated, reflecting the degree of variation
among the images at all points in the visual field.
 Disparities are analyzed to yield depth.
 All depth maps are combined into a single view
point sculptural model of the scene at a
particular moment
 This Reconstruction uses Trinocular Stereo
Algorithm.
 After constructing the disparity map, apply
median filtering to remove a few outliers in the
disparity map.
 Then the median filtered disparity map is
combined with a registered color texture to
reconstruct a 3D point cloud.
 Each pixel have 5 bytes out of which 2 bytes for
depth information and 3 for color of the pixel.
Why?...
If the background is stationary, no need to
reconstruct at each time frame.
Reconstruct the background scene in advance of
the session and transmit it once to the remote
sites.
System, which generates 640×480 pixel depth
color maps from 10 camera clusters in a rate of
15 frames per second (fps), requires a 220mbps
bandwidth.
Around 1.2gbps is necessary for high quality
effects!!
 3D point cloud is skeletonized
 Skeletonized data and the residuals are send to
the destination.
 Compression and decompression are done using
Modern Driven compression algorithm.
(a) (b) (c) (d) (e)

Reconstructions from the compressed data and their differences with the
uncompressed data. (a) Uncompressed data. (b) Compressed with 50%
residuals. (c) Difference between (a) and (b). (d) Compressed with 100%
residuals. (e) Difference between (a) and (d).
 Color Reduction scheme
3 byte color information is reduced to 1 byte.
A color map is send along with the data.
1 byte is for representing color index.
 JPEG scheme
Change video into JPEG form.
Motion JPEG uses JPEG still image compression on each
frame separately.
Sacrifice quality
 Visual quality before  Visual quality after
color compression color compression
Telecubicle :
Consists of two stereo immersive wall
surfaces and a stereo immersive desk surface.
These three display surfaces join to form a corner
desk unit. Walls appear as windows to other
users’ environment and desk forms a virtual
conference table.
CAVE
10 foot-cubed room that is projected with stereoscopic
images creating illusion that objects appear to co-exist with
the user in the room
ImmersaDesk
Small, drafting table like system capable of projecting
stereoscopic images.
 The software for implementing Trinocular Stereo
Algorithm and Modern Driven Compression
algorithm is CAVERNsoft.
 CAVERNsoft is a C++ toolkit.
 Work across SGI, Linux, Solaris and Win32
platforms.
 Next Generation Internet initiative to
dramatically advance the speed and quality
of the Internet’s infrastructure and thus the
ability to evolve new applications.
 Provide high bandwidth.
 A mark of around 8.8 gigabits per seconds
was set on April 26, 2006 .
 Remote medical diagnosis and therapy.
 Distributed decision making.
 Remote and distributed education.
 Distributed artistic performance and interaction.
 Virtual travel and tourism.
 Automobile Engineering.
 Application continues…
No need to travel long distance in order to
conduct business meetings.
A geographically distanced surgeon could be tele-
immersed into an operation theater.
This could be lifesaving!
Scene from original surgery Captured scene rendered from
reconstructed 3D geometry
with annotations and trainee's
hands mimicing the surgeon
Automobile designers from different continents
could design new generation vehicles.
 Dancers from different countries could train
together, dance together.
 Dance with yourself!
Quality of transmitting multimedia and Tele-
Immersion data streams over the Internet is
affected by high packet loss rates.
Solution: FEC/UDP.
Expensive.
High bandwidth requirement.
Tele-Immersion provides natural interaction to
the remote world.
In the tele-immersive environment, computers
recognize the presence and movements of
individuals and objects and then permit them to
be projected in realistic, multiple, geographically
distributed immersive environments where
individuals can interact with each other and with
computer generated models.