S

I FOR REAL
Much work is needed in
this new field, but that is

John A. Adam Senior Associate Editor

22

Go for it! we shouted. A visitor at the

University of North Carolinas G lab was
teetering at the edge of make-believe stairs
descending into a dark basement. The
problem was he could not see his own legs.
Uh-uh. No way. No way! He took off his
helmet and returned to the real world.
Later he said hed felt he was being sucked
into a bottomless well.
When my turn came, I enclosed my head
in the helmet and entered that fearsome
ersatz world-a simulated household
kitchen. Beacons on the labs ceiling allowed
my motions to be tracked by helmet sensors
and then computers calculated and (usually)
rendered images at a rate of about 20
frames a second, fast enough to create and
sustain the illusion.
There were no preplanned routes. I
moved within this fake space at whim.
Mischievously, I picked up a weightless
black frying pan with my disembodied pink
hand icon, then dropped it onto a blue tiled
floor. Then I dragged some chairs through
wood-grain cabinets. The experience, while
impressive, hardly seemed real-until I
reached the stairs. They looked steep.
I paused. Fear short-circuited reason. I
knew, of course, that physically I was on a
level floor in the lab. Yet my eyes were
telling me otherwise. My leg trembled uncontrollably. I finally went forward, my
hands groping reflexivety for the stair walls,
preparing for a tumble that never came.
That was my most gut-wretching experience with VR. I also rode into canyons on
a pterodactyl swooped off a skyscraper with
a hang glider, and stared down the tailpipes
of an F-15 fighter. These experiences were
fun and impressive but (done on commercial
systems and not on the universitys special
renderer) did not approach realism.
The current technology is a t t h e
boundary of usability, said James Helman
of the advanced graphics division of Silicon
Graphics Inc., Mountain View, CA, a leader
in this field, but a long way from reality.
Virtual environments have been around
for decades, mainly in expensive flight simulators and military displays. Whats new
about
l
reality is that the necessary
computing power and displays are more affordable and new peripherals such as data
gloves have become available, generating
001&9235/93/$3.00W393IEEE

still further dimensions of immersion and

interactivity.
Before, you wouldnt spend $6 million
for a truck simulation because using real
truck prototypes was much cheaper, said
Ron Toupal, president of Paradigm
Simulation Inc., Dallas, TX. Now, with the
price of some very capable VR systems less
than US $100 OOO, all that is changing.
In prosaic terms, VR endeavors to create
a very natural interface between humans
and computers, where communication can
take the form of moving 3-D imagery,
spatial sounds, and even physical forcesfrom motion to crude touch. (Smell is also
being examined.) The better systems can be
so responsive and render imagery so fast
that the user can feel immersed in the synthetic environment.
This has all sorts of entertainment possibilities, like immersive video games and
films, and many practical ones, too. For instance, the ability to do softdesigns of cars
or space stations that engineers can literally
enter (rather than view from the window of
a computer) might result in better products
at lower cost.
There are many mutations of virtual environments. One type, such as the North
Carolina kitchen [seepicture, opposite], emphasizes immersion. The computer-generates all that is seen by the user. Another
approach, called augmented reality, enhances the real world by computer-generated overlays. For instance, a worker
might ask the computer to display the
correct electrical connection while he is
looking at the actual aircraft wiring with
see-through goggles. And some virtual
presence work goes beyond videoconferencing to sense and manipulate objects
from afar. Such applications include monitoring a robot on Mars or diagnosing casualties near hazardous front lines.
In the mid-l980s, researchers a t the
National Aeronautics and Space
Administration (NASA) Ames Research
Center had to use stick graphics to get
2 F 3 0 frames per second in architectural
modeling. Now much richer scenes are
available and it gets better every year. A
small VR industry has sprung up, offering a
variety of mainly first-generation methods
of creating synthetic environments.
IEEE SPECTRUM OCTOBER 1993

head-coupled, 3-D glasses, and multiscreen

projections-which are priced from several
thousand dollars to roughly a d o n dollars
[see Toolsof the trade, belowl.

factors. Nausea, dizziness, or other ailments

may result if a user is immersed for more
than several minutes. But the potential is so
great, and improvements are coming so

rapidly, that industries from manufacturing

to health care to defense are already investigating VR applications.
One of the most enthusiastic industrial

CA, and the Cave, created by the University of Illinois, Chicago [see p. 301.
Liqui6crystal shutter glasses, such as CrystalEyes by Stereographies, must be
worn. The Cave can cost as muchas US $1 OOOOOO, but several are being built
in industry and government.
High-end graphics workstations or specialized computers usually gene&?
the virtual scene from a 3-D database that is often derived from some unique
modeling packagelike computer-aiM design software andlor a finiteelement analysis program. The rendering component transforms the abstract
numerical representationsinto easily interpreted images, usually composed of
colored polygons. The final result is hwo stereo images that are transmitted to
ew of this system preventsa sense of immersion but, on the display.
s k r frwne rates and h m sharper imagery. Also, the
Most of today3 tracking systems, which may sit atop workstatiis or be suscompared to more immersive displays. Leading com- pended from ceilings, use either migrteticfields or ultrasound to measurethe
glasses and other equipment include Stereographics position and orientation of objects like HMDs o
,and TWwix Inc., Beaverton, OR.
transmitter emits a continous signal that hits the
ense of immersion is provided by the head-inomted display translated into six-degreedfreedom meeso
ks out the real world and wen allows the user to assume a for the computer. Leaders in the hadring area in
such as that of a fish. There are also see-through HMOs that and PolhemusInc., Colchester, VT.
ination or the virtual and real worlds, known as augmented
Of the various input d e v i i used in VR, the wired glove is often the most
new HMO with 1-inch CRT screens was recently introduced by useful. Its user can touch or grasp both virtual and real ob@s without undue
of the unit, diffi&y. Informationfrom sxnsors W measure the W
n
ig of finger and harKl
are still u n c o m f o ~ ~ l e ~thei t bulkiness
h
though, must wait for &ses with built-in displays.
joints is related to the position of the glove, which is tracked separately. In imA heabcoupled dwky (HCD)is like a huge pair ot binoculars supported rnersiveVR, the wired glove is represented in the vittuai scene, typically as a
by a a b l e robot-like arm. The image sources and optical components are disembodted, floating hand.
The computer processes the trackiig information and OWinputs from&
endof anarm,which also has a buitt-in W n g sy&m with no lag.
solutionand a wider field of view beGause weight
HMDs. They also offer immersionwith the added
t, which is why they are a favorite at trade shows
ion is the Boam (binocular ~ n i mm~ i ~ i ~
o Park, CA. A disadvantage in some applications
manipulate the display and audio is not standard.
use multiple large-screen projection displays,
ulf groups of viewers .Examples are the M1
4AA.
icrosystems Computer Corp., Mountain View,

a virtual reality (VR) system consists of a display, a tracking

a computer image generator, a three-dimensional
n soflware. The several types of displays available are
suited to & i t tasks.
The most basic and economical, costing several thousand dollars and not
incbding the computer, is the computer monitor approach, with liquid-crystal
&otter glass and trackr. The monitor dispkay changes accordingto the users
head movements as tracked by a device a m the monitor. Glassesenable 3-0
vievling of imagery as in the w of the Hubble Space Telescopes corrective

Adam-Vi

23

reality is for real

--

users of VR so far is Caterpillar Inc., Peoria,

IL. The $10 billion company is the worlds
largest manufacturer of earth-moving and
construction equipment. Its operators do
much more than drive-controlling huge
hydraulic shovels, for instance-so visibility
from all sorts of angles is critical to efficiency and safety. Assessing interior visibility
on new prototypes is the purpose of
Caterpillars first endeavor in the VR field,
something beyond its regular computeraided design work.
The perspective that you get in [existing]
3-D visualization packages isnt really true,
said design engineer Dave Stevenson of the
Wheel Loader and Excavator Business Unit.
Although you can rotate the image and even
go inside it, you cant do what you d do in
the vehicle, he said, such as lean a few centimeters to the side to get a better view of
what is happening, as well as a better
dynamic sense of the design model.
FAKE TRACTORS. With VR, that is a cinch. To
test a design, Stevenson sits in a l i s t
physical mock-up of a cab. On his right are
two levers that lift and tilt a huge virtual
bucket. In front are the steering wheel and
gas and brake pedals. On his left is the gear
shifter. These are actual physical controls,
but they are also rendered and registered in
the VR world so that, when a user reaches
for the gear shifter in the fake world,
contact is made with the actual shift in the
real world.
When Stevenson dons the head-mounted
display, the lab disappears and he finds
himself seated inside a futuristic wheel
loader at a construction site. He works the
controls, and the bucket scoops up dirt. He
leans forward and gazes down at the action.
Then he backs up, makes a tight turn, and
goes toward a dump truck to empty his load.
All the while, he cranes his neck, checking
visibility.
Except for the digging, nothing is canned.
Stevenson can maneuver wherever he
wants. Even while the vehicle is moving, he
r

24

can look out any window, although for

complex scenes the frame rate dips below
the 20 frames per second needed to create
the illusion of fluid movement.
The imagery is deliberately kept simple
enough to be rendered in close to real time,
but also sufficient for the design engineers
examining visibility. It takes just 2 to 3
minutes in VR immersion to evaluate a
design change, Stevenson told me, so
nausea and other human factors issues have
not been a big problem.
The first VR-influenced designs will be
seen in back-hoe loaders (contraptions with
a bucket up front and in back a ditch digger
articulated arm) to be introduced around
1996. John Ekttner, a Caterpillar engineer
in the Building Construction Products
Division, told Spectrum they had three
distinct designs in concept, all ostensibly
equally feasible.
But in the VR world, Ekttner said, they
found that with two of the designs, when
you operate the back-hoe linkage, you can
get into a position that occurs dynamically,
where you cant see what youre doing.
Without VR, he said, this would not have
been determined until the first prototype
was built. The result: Caterpillar chose a
design nine months earlier than would otherwise have been possible and saved the
tremendous cost of prototyping the two bad
designs in iron.
The company produces hundreds of
products, many of them, such as heaving
mining equipment and log loaders, in small
production runs. Consequently, if a profit is
to be made on a line, money cannot be
wasted on generations of iron prototypes,
some of which may cost millions.
Virtual reality could change Caterpillars
whole design approach. Weve been able to
look at new ideas and either scrap them or
develop them, Stevenson said. Its expanded by orders of magnitude what we can
try.In a few days we can probably crank out
a dozen wssibilities.
Cate&illars VR work is beine done at the
National Center for Supercomputing
Applications, a government-private sector
consortium at the University of Illinois,
UrbanaChampaign. Stevenson credited the
consortium, as well as work begun three
years ago with the University of Iowa,
Ames, for the companys success with VR.
The technology was too unproved and risky,
he said, for Caterpillar to pursue on its own.
All the equipment used was commercially
available: Silicon Graphics Skywnter for the
graphic rendering, a Hewlett-Packard 735
for calculating the dynamics of new scenes,
and Sense8s WorldToolKit software. In addition, an IBM PC drives the hardware
inputs from the platform, like the gearbox
controls, and an Apple Macintosh generates
sounds. For displays, there is a choice of a
head-mounted display or 3-D glasses and a
1.8-meter screen. Recently, Caterpillar
began working with a Cave display with
multiple projection screens.

The toughest part of Caterpillars VR

work probably was converting the parts
fdes. Now that some automatic translators
have been written, it takes about three days
to convert a parts file-a ProEngineer
computer-aided design (CAD) file of a cab, a
tire, a frame, a sheet metal part, and so
on-for VR use, according to Stevenson.
(One reason for the delay is that the CAD
data, already in three dimensions, has a resolution that is too high for todays virtual environments.)
Right now, Stevensonwould never show a
virtual prototype to a potential customer.
He wouldnt buy it, he said. It would be
the VR flaws that hed notice first.For instance, mirrors are absent from the rendering because the computational complexity they require would lower frame
rates. An eventual goal, of course, is to incorporate customer input into VR prototypes that make use of sound and motion
as well as nice imagery.
AUTO MAKERS. Detroit has dabbled in VR,
too. The Big Three among US. automobile
makers have found the current VR tools not
yet worth widespread adoption, but of such
great potential that they are joining with a
US. Army vehicle center, United
Technologies automotive division, the
University of Michigan, Ann Arbor, and a
dozen other smaller companies to form a
consortium for industrial VR.
VR has the potential of revolutionizing
design and manufacturing, the industrial
VR group stated in a July proposal to the
Pentagons Advanced Research Projects
Agency. It went on to predict savings in
time and money, better market response,
and better products.
Hyperbole or not, other companies corroborated many of the proposals claims,
specifically:
Virtual prototyping will reduce or eliminate the need for costly physical mock-ups.
Engineering analysis will become more efficient through the integration of simulation
results with virtual prototypes. For example,
air-conditioning flows or even crash tests
can be visualized in 3-D. Eventually, it will be
possible to alter designs and see the immediate effects.
Operational simulations will permit the
direct involvement of human beings in performance and ergonomic studies. For instance, passengers of various sizes will be
able to comment on the convenience and
look of a virtual cars interior. Immediate
design modifications will get immediate
feedback. More iterations will be feasible
than with physical prototypes.
Virtual simulation of assembly,production,
and maintenance tasks will reveal possible
problems at an early stage of the design
process.
Support of assembly and maintenance
through augmented reality will reduce job
training and increase workforce flexibility.
For instance, pages of a maintenance
manual or diagram might be projected while
IEEE SPECTRUM OCTOBER 1993

Adan-Vi

reahty IS for real

-_

BA
available, is the point of entry for simulating
early prototype designs, Socks said.
k i n g ' s new airliner, the 777, to be rolled
out next year, is its first to be designed
without the use of a full-scale physical mockup. The aircraft design is being specified digitally, using Catia software on IBM mainframes and CAD workstations.
It is a tribute to what can be done in soft
design without VR. Current VR systems are
orders of magnitude away from being able
to render the complexity and detail necessary for such a design at acceptable frame
rates. Representations capable of being displayed in VR of the part geometries for the
whole aircraft would consist of between 5
billion and 10 billion polygons, estimated
David Mizell, virtual systems manager at
W i g Computer Services, Seattle, WA.
But the allure is great. In certain situ26

ations where an engineer is trying to

develop a mental image of a very complex
collection of parts, sticking his head into the
design and looking around should be more
effective, in Mizell's opinion, "than merely
looking at it 'through the window' of a workstation."
The Boeing team has written software
that can import Catia geometry into their
Silicon Graphics-based VR system. They
are experimenting with a variety of algorithmic techniques aimed at rendering
virtual worlds with the complexity and detail
of CAD models at the frame rates required
for VR. Currently, they can view subsets of
the CAD design, such as a passenger compartment interior or some of the hydraulics.
But the frame rates are not yet high enough
for production use.
'When you're in the virtual interior and

you look over to the side at the windows,

the frame rate is really nice," Mizell told
Spectrum. "But when you look straight
down the aisle, [it1 drops quite noticeably."
This is because most of the aircraft then
comes within the user's field of view. The
graphics hardware has to process several
million polygons, representing all the objects
in front of the viewer, whether or not they
get displayed on the screen.
Mizell does not foresee each of k i n g ' s
thousands of engineers getting VR capability at all soon. He believes that conventional CAD workstations are generally adequate for designing individual parts. VR
would be desirable to visualize a part in its
installed context, among many other parts,
and determine, say,whether it could be maneuvered around the other parts in order to
be installed or removed for maintenance.
E E E SPECTRUM OCTOBER 199

Mizell sees modification of the CAD design

while inside the virtual environment as
farther off than visualization only. Were
making improvements, but weve got a ways
to go, particularly with the rendering performance, before we can make the decision
to use VR in the design of the next plane,
he said.
Another E k i n g project, on augmented
reality, begun in 1990, might come to fruition
first-at least, it is less computationallydemanding. The goal, said Mizell, is to give
more information to aircraft factory
workers, many still doing hand labor. For instance, they might be able to consult a
manual without having to stop work, or assemble p a r t s marked with a virtual
template.
One possible application would be hand
layup of composite cloth. Aircraft manufact u r e r s a r e building some parts out of
graphite composites instead of aluminum.
Parts with complicated, compound curvature are often laid up by hand. Currently,
expensive templates or other marking
systems are employed to mark on the
previous layer where the next layer of cloth
is to be placed.
A worker using augmented reality (AR)
could simply look down at the layup tool and
see a computer-generated rectangle
seemingly drawn on the previous layer to
mark where the next layer was to be placed.
They would perceive the graphic as painted
on the workpiece,Mizell said, even as they
moved around. After laying this cloth, the
user could push a button or say Next into
a simple speech recognition system, and the
AR system would then display the outline of
the subsequent layer of composite cloth.
Another potential application the Boeing
team is experimenting with is wire bundling
assembly. An aircrafts electrical wiring is
assembled into bundles on formboards,
from one to eight 0.9-by-2.5-meter (3-by-8foot) particle board sheets that have a
circuit diagram of the wire bundle glued to
them. This makes each formboard unique
to a wire bundle, incurring storage and construction costs. With augmented reality,
Mizell said, a worker could assemble a wire
bundle on a blank formboard, with one or
two wires at a time being shown in the
workers AR display, appearing to be drawn
on the board.
All this could be done in a self-contained
unit-a see-through display on the workers
head and a fanny pack with memory and
batteries around the waist. Were just
drawing a couple of lines out there, not rendering everything the user sees, said
Mizell, so it is generally easier to do, with
one exception.
Tracking the users head position and orientation presents a challenge because of the
need for high accuracies @or example, 0.1
inch or better positional and 0.1 degree
angular) over relatively long distances.
Moreover, most commercially available
head trackers are inadequate for factory

floor usage. Magnetometers lose accuracy ,

around radio frequency noise and ferrous 1
metal. Ultrasonic trackers are affected by
ultrasonic noise-from drills, rivet guns,
and so on. Consequently, over the next few
months, Boeing will be experimenting with
optical trackers.
Once the tracking has been addressed,
the human factors issues will be tested, det e d g , say, whether a worker can use
the gear for two hours at a stretch without
getting a headache. Were not even at a
point where we can address that, because
weve got crude laboratory equipment,
Mizell said.
I
MILITARY DESIGN. Defense work and space
exploration, both of which often take place 1
in hostile environments, have had much to
do with the creation of VR. The Pentagons
Advanced Research Projects Agency (ARPA) sponsored Ivan E. Sutherlands pioneering head-mounted display research in
the 1960s. Shrinking government budgets
are now forcing both defense and space
agencies to think more about virtual capabilities for design.
Aircraft carriers, submarines, and space
stations are becoming unaffordable, said
Gary W. Jones, simulation program
manager in ARPAs maritime systems technology office. Their complexity is growing.
The new Seawolf submarine has three
times as many drawings as its predecessors.
An aircraft carrier now has 30 million components, he told Spectrum, not counting
nuts and bolts.
How can humans keep up with all of
that? he asked. ARPA is betting on a combination of virtual environments, physicsbased modeling, and distributed simulation.
Last March it awarded 18-month contracts
to two teams led by Lockheed Missiles &
Space Co., Palo Alto, CA, and General
Dynamics Corp.s Electric Boat Division,
Groton, CT, to assess virtual ship and sub
marine design. The initial contracts are to
experiment with pieces of the system, such
as a torpedo room [see picture opposite,
top]. The goal, several years off, will be live
fire tests of virtual prototypes on a simulated sea. This will permit a much more
realistic assessment of a candidate design
early in the process, Jones said, when
changes can be made much more cheaply
than in engineering production.
While many users are now content with
10 OOO polygons per frame, Joness project
requires jumps to 1 000 OOO polygons per
frame at 20-30 Hz.Users must also be able
to interact on various scales with a large
complex geometry on the order of 30
million objects. More than lo00 designs with
hundreds of parameters are to be generated
and evaluated and modifications (which may
affect over 50 percent of detailed design
drawings) must be accommodated.
Dynamic rapid access to more than 200 gigabytes of memory must be used.
Jones expects the project to drive such
critical technologies as interface standards,
~

I
1
1
,

database management, physics-based engineering analysis, real-time interference

(such as collisions) analysis, and distributed
high-performance computing among a large
number of vendors.
HEALTHY
Virtual reality applications in
medicine, still years from general clinical
use, benefit from two trends in health care.
The first is the extensive use of imaging
tools, such as ultrasound and magnetic resonance imaging (MRI), to peer into the body.
The second is a U y invasive surgery,
such as endoscopic procedures, in which the
doctor looks not at the patient but at a video
screen to guide an optical-fiber light probe
and tiny camera. Instruments are inserted
through puncture holes and manipulated
from outside the body.
Researchers at the University of North
Carolina (UNC) at Chapel Hill are working
on 3-D ultrasonic imaging of a fetus superimposed on a pregnant womans abdomen
[see picture, p. 25, bottom right]. Better
imagery would be possible with computer
tomography and MRI scans, said UNC
computer scientist Gary Bishop. But the ultrasonic scans are inexpensive and easily accessible for tackling basic problems of
timely rendering and correlation of the superimposed image and the actual fetus.
ComDanies like Medical Media Svstems.
Hanover, NH, hope to use VR in training
surgeons in new techniques, much as pilots
are trained for new planes on simulators.
Medical Media, in cooperation with
Dartmouth Universitys medical and engineering schools, recently demonstrated a
simulation of endoscopic knee surgery. High
Techplanations Inc., Rockville, MD, expects
t o complete a surgical simulator for
laproscopy. Georgia Tech in Atlanta is
trying to simulate surgery on the human
eye, complete with force feedback.
V i u a l renderings might also be used to
help plan surgery and in remote assessments of patients by teams of specialists. One possibility, being examined by
Georgia Tech and others, is to couple tactile
gloves with visual and audio links for t e l e
diagnosis in rural areas where there are few
doctors. ARPA is also preparing a medical
initiative that will include VR, one possible
military application of which would be its
use by doctors to quickly diagnose front-line
casualties remotely.
Patients are expected to begin therapy
for acrophobia this fall using a 50-story
virtual glass elevator, designed at Georgia
Tech. In physical therapy, playing with
virtual objects can be more appealing than
playing with real ones. Moreover, forces,
such as the resistance a patient encounters
in squeezing a ball, can be scaled and his or
her recuperation can be better measured
and tracked.
RESEARCH AND EDUCATION. There are numerous scientific VR visualizations, from
atoms to galaxies. Users working in three dimensions can try to dock a drug in a
complex molecular structure, scaled up, to

m.

27

Adan-Virtual reality is for real

feel the forces from different energy levels.

Manipulators, at sites like the University of
North Carolina, can vary from tweezers to a
robot arm.
Some telepresence at the atomic scale
was recently achieved when researchers
from UNC and the University of California
at Los Angeles (UCLA) announced a new
way to view and control a scanning tunneling microscope with a VR nanomanipulator [see picture, p. 26, left]. Examining
graphite, UCLA chemistry professor R.
Stanley Williams said the technique enabled
him to discover that what had previously
been thought of as noise was actually pieces
of dislodged carbon. VR permits the viewer
to rotate the sample and change the light
source for different perspectives.
Something intriguing can be investigated
further, such as the space between atoms
in a crystal lattice.
Research will reach new dimensions in
November, when a Cave VR system is to be
hooked to a Connection Machine 5 supercomputer in Portland, OR. The result will
be simulations in real time or close to it.
The arrangement will enable engineers and
scientists to modify the situation as it
happens, said Carolina Cm-Neira, a Ph.D.
candidate at the University of Illinois,
Chicago. For instance, designs might be
altered to assess aerodynamics.
Initial applications for the supercomputer
VR include hurricane studies (complete with

28

3-D sound), galaxy configurations and black

hole collisions, and representations of mathematical worlds such as hyperbolic space.
At NASAs Johnson Space Center,
Houston, TX, R. Bowen Loftin, who explores VR training for astronauts, has developed a virtual physics lab for high school
students. Once inside the virtual world, the
students can change its properties on a
control panel by adjusting gravity, friction,
or time. For instance, they can stop time or
slow it down in order to perceive fastmoving experiments. Among the tools are
two balls with alterable (and traceable)
bounce and a variable-length pendulum.
AMUSEMENTS. Entertainment uses for VR
naturally have received the most media attention, and experts agree that this large
market will be a driving force in VR technology development. Already, consumers
with PCs can create 3-D racing-car games
or air combat simulationswith software such
as Sense8s WorldToolkit. Combined with
Windows-based networking, the game can
be played by several people together. Sega
expects to introduce a low-end headmounted system this fall. Silicon Graphics
and Nintendo recently announced plans for
a 64-bit 3-D game machine intended to go on
sale by late 1995 for about $250.
VR arcade machines are springing up,
many of them from W Industries Ltd.,
Leicester, UK, which sells some that can be
networked so that operators can work in

teams. VR parks began in Chicago a few

years ago with Battletech, which now has
been expanded to Japan, too. Instead of
going bowling, some teams regularly pay to
wage war on each other in a futuristic
setting.
Voyage to Mars is a park that is to open
this year in Walnut Creek, CA, using simulators from Hughes Aircraft Co., Los
Angeles. The companys training division
sells the Mirage, a multiseat networkable
unit with a wraparound display for views of
6 meters to infinity. Another park, Magic
Edge, Mountain View, CA, is building space
capsule simulators with Silicon Graphics
Reality Engines and hydraulics to simulate
an F-14 fighter scrambling from an aircraft
carrier to intercept hostile flyers.
Perhaps most eagerly awaited is the
outcome of a Star Trek collaboration
between Paramount Pictures Corp., Los
Angeles, and Spectrum Holobyte, Alameda,
CA (a computer-game maker). Reportedly,
users are to be able to wander about the
Enterprise and interact with the characters.
Plans call for 50 Star Base centers, the first
to open late next year.
Over the last year, there has been lots of
activity in VR by Hollywood studios and
casinos, according to Silicon Graphics marketing head, Daniel Vivoli. Most deals apparently remain confidential.
DESIRE FOR IMPROVEMENT.If VR has piqued
commercial interest, the movement beyond

IEEE SPECTRUM OCTOBER 1993

the research lab has also highlighted deficiencies-and spurred an urgent desire for
improvement. The current performance is
marginal. It will stay marginal for quite a
while, Silicon GraphicsHelman told several
hundred designers at Siggraph 93 in August.
For a truly entrancing experience, a
iumble of things must happen simultaneously and seamlessly. Perhaps most important is keeping up an acceptable frame
rate for the user, usually about 20-30 Hz
for each eye, while juggling tasks like mathematical simulation, visual rendering, audio,
force feedback, and collision detection. Input
data from trackers, motion platforms, and
so on must be taken into account, too.
Currently, all these tasks are too taxing
for most workstation arrangements so d e
signers must make tradeoffs. To orchestrate so many pressing demands, two basic
approaches are employed synchronous and
asynchronous.
With the synchronous method, the user
always gets the most recent result, just in
time. But it is limited by the speed of the
slowest task. In the unpredictable VR world,
noted Helman, it is hard to have all tasks
run predictably in their allotted time.
Consequently, frame rates may slow down,
but the frame will have complete data.
With the asynchronous method, the frame
rate is kept up by sacrificing completeness.
For instance, if collision detection takes a
tenth of a second and visual rendering takes
a sixtieth of a second, the scene can be
rendered six times while the collision is calculated. This scheduling is obviously more
complex and latencies may occur. The car
may have already gone off the racetrack,
Helman observed.
A compromise between the two approaches, with the tasks being divided
among different machines with different priorities, is often necessary to keep frame
rates consistently high and latencies low.
Another common compromise is just to
drop features like as collision detection.
Many applications also neglect sound and
force feedback. Most attention is currently
focused on the tug of war between image
quality and frame rate. Frame rates around
10 Hz are very choppy. The trouble is,
people like complex images, so we tend to
bring it to m i n i frame rates, said
Daniel Sandin of the University of Illinois,
Chicago.
If a graphics computer can render a
million polygons per second, then getting 30
frames per second would allow (simplistically) about 33 000 polygons to depict a
scene. Nonetheless, a molecule can look
much better when depicted with 100 000
polygons, so a user may put up with a frame
rate of 10 Hz.
Compensatory techniques such as the
blurring from anti-aliasing can make W b y 512-pixel displays seem almost as good as
1280-by-1024screens (which have L3 million
pixels that need refreshing), said Silicon
Graphics Helman. Texturing can save on
Adam-Virtual

reality is for real

With a commercial
DataGlove, a user can
crush a virtual soda
can [left1 programmed
with compliance and
tensile-linearity parameters. Small pneumatic microcylinders,
attached to the glove
from the palm to the
Fngertips with Velcro,
produce force feedback,
so the user feels a near
natural tactile sensation,said researchers at
the Human-Machine
Interface hboratory,
Rutgers
University,
Piscataway, NJ

the order of millions of polygons in lieu of

Tracking also needs improvement to
the best shading. But all this visual trickery better synchronize real-world motions with
virtual-world depictions. When a user stops
is less impressive in stereo, he admitted.
Choosing rural scenes (canyons are a fa- moving a data glove, for instance, and his
vorite) rather than more difficult cityscapes hand keeps moving in the virtual world, the
is also a way of dealing with current short- illusion suffers. The noise and latencies are
comings. Precomputing scenarios and particularly bad now, so systems cannot be
limiting paths to certain branches are two predictive, said Daniel Sandin of t h e
University of Illinois, Chicago, a factor that
other means.
As imagery from computers improves, so could aid in the computing of complex
must the display capability. Current head- scenes. Latency-the change in time from
mounted displays, usually the best for im- input to result-can also make a user
mersion, are often 320 by 240 pixels, giving unaware that an action has been received,
the user the vision of the legally blind, so he repeats it, compounding the problem.
except with a much narrower field of view. Latencies may also be nauseating.
Other problems, noted recently by VR
The best displays, just starting to appear
commercially, have 1280 by 960 pixels. pioneer Frederick P. Brooks Jr. at the
These rely on cathode-ray tubes, which are University of North Carolina in Chapel Hill,
heavier than other devices. Consequently, are MITOW field of view (usually much less
there is a desire to create lightweight head- than peripheral vision affords), poor regismounted displays that are not much more tration with the real world (so a user might
bump into real walls while walking through
obtrusive than glasses.
Dick Urban, a project manager in MAS a virtual room), tethered ranging (which
electronic systems technology office, is limits how far a user can wander), and difsponsoring development by 1994 of a 1280- ficult wayfinding in virtual worlds (where it
by-1024-pixel monochrome electrolumi- is much easier to get lost than it is in the
nescent display to fit in a standard ll-ounce real world).
So far, much emphasis has been placed
(0.3-kg) Army goggle. A color liquid-crystal
display of about 1square inch (6.5 cm2)with on the visual. But sound and force radically
that acuity is the aim of another project, enhance the experience,Brooks observed.
which recently demonstrated pixels of 24 The more senses that are bombarded with
km. Urban told Spectrum that these are redundant information, the more real the
probably the worlds smallest pixels (they experience appears to be, noted Scott S.
also have great potential for highdefinition Fisher, managing director of Telepresence
television, offering 1OOO lines per inch, or 40 Research, a Portolla Valley, CA, company
per millimeter). Even if these pixels can be that develops VR applications .
In addition, building synthetic envimanufactured, they will still provide the
equivalent of only 20140 vision and a 40- ronments is usually lots of work. At present,
20 OOO to 40 0o0 polygons might be used to
degree field of view, Urban said.
But 12-km liquid-crystal display pixels render a house. A single room in a better
are conceivable-just enough light would model takes thousands of polygons and is
equivalent to writing about 20 OOO lines of
still pass through the pixel, he believesand the result could be displays of 2048 by code. As for how to make authoring more
2048 pixels, which approaches the quality of efficient, Brooks said, Weve not begun to
even address this problem.
20/20 vision.

+
29