From Simulation to Virtual Reality
A Robotic Application
M. Gerke R. Dicken
Chair of Process Control
FemUniversitiit Hagen
D-58084 Hagen, Germany
Abetract
Bosed on a simulation rgsicm for factory workcell
opiimizaiion and ofline progmmming of kincmaiic sy-
stems a softwan interface for natiimc coupling i o pro-
cesses is pwscnied in ihis paper.
Roboiic wmponents a n available in the mal work-
cell whose wnirollers o8er nigh speed interfaces for
naliimc paih modificaiion. Boih subsjrsicms can be
connecied via a communicaiion managing computer i o
improve from simulation to telepnscnce and ieleopem-
iion. firthermon comfortable man-machine- inierfa-
ces a n iniegmicd i o provide roboiic conirol in 'Virtual
Realiiy'. Our concept depends on industrial standard
components which gives easy access i o pmciiiioners.
Keyworde
Simulation of mechanical systems, Industrial Ro-
bots, Virtual Reality
1 Introduction
Based upon powerful simulation systems todays
man-machineinterfaces open the door to the world
of 'Virtual Reality'. The most challenging aspect of
'Virtual Reality' b not to leave the operator alone in
any irreal surrounding but to allow telepresence and
teleoperation in regions of the real world where ac-
cess is restricted in reality (e.g. microcoemoe, nuclear
areas).
This leads to realtime interproeese communication
between simulation and real world processes. In re
botica the idea of 'Virtual Reality' has been improved
to become a mighty concept for telepresence and t e
leoperation as well a8 a tool for online-programming
351
0-8186-5292-6/94 $3.00@ 1994 IEEE
H. Hoyer
during the laet decade. (Think of robotic manoeuvera
in space or in nuclear power plants far away from any
operator but atill under human guidance and control).
Despite of long term reeearch and development stra-
tegies, there are major components of the concept
available at present. Integration into existing simula-
tion and offlineprogramming eysteme as well as real-
time coupling to standard robotic controllers used in
todays industrial environment is possible. This inclu-
des online visualization of robot motions according to
the occurences in the workcell and comfortable control
tools for the robot kinematics.
With regard to immediate availability for researcher
and practitioners our Chair of Procese Control has de-
veloped an environment of 'Virtual Reality' for r o b
tic applicationsbased on industrial standard computer
hardware and simulation software. To give the system
a practical relevance in todays factory environment it
ie build up just to a degree of 'Virtual Reality' which
b feasible for industrial operators.
The following demands are made on the environment
[GER911, [HOY931:
- 'Design Tool' for optimal factory workcells
- 'Programming Tool' for o f f l i p a t h planning
with downloading option towards the robot con-
troller
- 'Visualization Tool' (graiical terminal, eyephone)
via online data link with the robot controller
- 'Control Tool' for telepresence, teleoperation and
onlineprogramming supporting a man-machine-
interface (teach ball, data gloves)
2 Stepping into Virtual Reality
Improving a simulation system for kinematic com-
ponents by modern man-machine-interftxes and estab-
lihing a realtime communication to the technical pn>-
cess leads to a VRsyetem of industrial relevance.
In robotica the efficiency of a VRsystem depends
on the availsbity of communication interfaces bet-
ween different hardware and software components. To
build up a VRsystem today means you still have to
chooee standard system componenb such aa the robot
controller hardware and simulation software carefully
whereas future components are expected to be opened
for user purpoeee.
The immediate availability of our VR-system for ex-
ample is guaranteed by choosing the following hard-
ware configuration:
UNIMATION PUMA 562 industrial robots with
MARK 111-controller (Operation System VAL 11) are
built up in a practical workcell. T h e standard robots
offer a serial ALTER-interface for onlinemotion con-
trol by external computers. The robots are integrated
into Ethernet via their serial SUPERVISOR-interfacea
and a network front-end computer.
The simulation software runs on a high perfor-
mance grafical workstation HP-WS 30-625 under DO-
MAIN/OS 10.3. The workstation is included into
Ethernet and offers serial data links.
Serial communication management is done on an in-
dustrial Personal Computer 80386 (under MSDOS)
with four data interfaces.
To provide comfortable manipulator motion control,
a robot command teach ball serves as man-machine-
interface. The ‘Virtual Reality‘-system hardware plat-
form consists exclusively of commercial equipement
without any hardware modification. The configura-
tion is presented in figure 1. No changes are necessary
with regard to the robot controller. The simulation
software S.M.S.version 3.0 (SIEMENS) provides gra-
fical visualization and motion animation of mechanical
systems. P L O 911.
This ’VR’-system is a comfortable tool for high level
control of robotic applications. A typical example is
given in figure 2.
3 Managing communication
The communication manager (PC 80386) uses se-
rial interfaces (RS232, Rs 422) to administer and su-
pervise the easential communications. The communi-
cation link to PUMA 562 MARK 111-controllersis per-
352
manently established via its eerial ALTERinterfacee.
Due to the fact that online motion control is highly cri-
tical in time (controller cycle time: 28 me), a simple
communication protocol with fault detection is used.
MARK 111-Controllers offer the actual poeition data
set of the robote Tool Center Pointe (TCP) and are
ready to receive correctional data from the communi-
cation manager. Another serial communication link
interconnects the manager and the grafical worksta-
tion.
If the system is working (YI ’Visualisation Tool’ the
simulation system orders the data seta of the actual
positions of the robob from the communication ma-
nager by hardware interrupt.
In ’Control Mode’for teleoperation a robot command
teach ball is connected to the communication mana-
ger as system sensor input. The teach ball transforms
human force/momentum data to digital data thus nx+
ving the robots T.C.Pa directly in i b Cartesian m r -
dinate frame.
Beyond all theee communication links easential to re%
l i e a ‘Virtual Reality‘ environment, there is an*
ther serial data link necessary between MARK III-
controller and any network computer supporting
DDCMP-profocoll (e.g. DEC-VAX) via the control-
lers SUPERVISOR-Interfaceto include P r o g r v m h g
Tool features to the system. Path planning has to be.
done carefully offline with the simulation system and
afterwards motion instructions (programme, locations,
variables) are downloaded fiom the grafical worksta-
tion via network towards the front-cnd computer and
via serial data link to the robot motion controllers.
A communication software is developed for the Com-
munication Manager to assure that data transport and
administration for MARK 111-controller, robot com-
mand teach ball and grafical worlretation is correct
according to the subsystems specifications (protocol,
data format). Special action has to be performed on
hardwareinterrupt to provide position data on request
from the simulation software [DI 931. Due to the time
critical data interchange between communication ma-
nager and robot controller, position data sets and -in
return- correctional data have to be transmitted and
computed within 15 me; otherwise a ‘communication
error‘ is stated by the robot controller. In addition
to all the communication tasks the Communication
Manager has to administer data which includes data
transformation and allocation.
The simulation system S.M.S.3.0 is particially modi-
fied to our needs; beyond its standard volume S I E
MENS opened a eoftware-interface for us which is ex-
tensively wed for our ‘VR‘-reslition. Communi*
A
PUMA 682
MARK 111

I IV -
Teach Bell
DIMENSION 6

Figure 1: System data link configuration

m
-
ch Ball
DIMENSION 6

\\ Industrial PC

~~~~~~

Figure 2: Telepresence in critical environment

353
tion software had to be integrated to handle interrupt
request and data interchange with the Communica-
tion Manager. Some subroutines neeeosary in robotic
applications such as backtransformation of 'homogene
coordinates' [PAUL 811 had to be implemented.
The VRsystem will be improved within short terms
by integration of an image processing system to reco-
g n k error conditions occuring in the factory workcell
and to handle dangerous motion commands from the
operator. With regard to industrial standards a VME-
computer under OS9 is to be integrated aa communi-
cation manager.
4 Examples
To demonstrate the functionality of our 'VR'-
concept preaented above two typical tasks of teleope-
ration have been carried out:
The simulation system is initialized by loading a simu-
lation environment according to the actual workcell. A
PUMA 562 in our real workcell moves into a predefined
start position and initiates ALTERMode. The simu-
lation environment is referenced via data communiccl-
tion presenting a PUMA 562 in start position. Now
manual input from sensor teachball leads to the robots
Cartesian motion directing the robots T.C.P. to any
goal position. Within each controller cycle (28 ma),
the robot sends positional data to the Communication
Manager to permanently hold available the actual p e
sition data set for any request from the simulation
environment. So visualisation of the robot motion is
kept online. The robots corresponding motion trajec-
tory can be protocolled grafically in a TRACE-mode.
Two screen copies of teleoperation tasks are preaented
in figure 3 and 4.
5 Software Availability
Source code software and documentation for the
Communication Manager is written in C (MS-DOS)
and useful for PC 80x86's Serial Duart 8250; im-
plementation on 680x0's Serial Duart 68681 is cur-
rently under developement in C (OS-9). This
software is available for free; send email to
'michael.gerke@fernuni-hagen.de'.
S.M.S.is a product of SIEMENS A.G.(Dep. ZPL-PA)
in Erlangen, Germany; currently it is available for HP-
APOLLO workstationsunder DOMAIN-OS and it has
to be purchased. Software menus are in German lan-
guage.
This product can be replaced by any other robotic
simulation system if a user programmable software in-
terface is provided by the developer. The following
functionality is necessary: Receiving positional data
on q u e s t via eerial data link, data transformation
for visualization. A data base for standard industrial
robots should be included.
Modern controllers for industrial robots (e.g. UNI-
MATION PUMA, ADEPT) offer hardware interfaces
for realtime path control; so no changea in robot con-
troller hardware or software are necessary.
6 Acknowledgement
S.M.S.3.0 has been provided by SIEMENS AG Er-
langen; special thanks Dipl.Ing. P. Kolbenschlag of
SIEMENS, who adopted a softwareinterface acwr-
ding to our common definition.
References
[GER 911 Gerke M.,Borgolte U., Hoyer H., Roboter-
simulation ala kombiniertea Entwurfa- und Pro-
grammierwerkseug, Simulationstechnik, ?. Sym-
posium, Hagen 1991 (in German)
[HOY 931 Hoyer H., Gerke M.,Virtual Reality fuer
Roboter im industriellen Einsate, Simulations-
technik, 8. S y m p i u m , Berlin 1993 (in German)
[WLO 911 Wloka D.W. (Hrsg.), Robotersimulation,
Springer-Verlag 1991 (in German)
[DI 931 Dicken R., Implementierung eines Virtual
Reality Konzeptes, FernUniversitat Hagen, Di-
ploma Thesis 1993 (in German)
[PAUL 811 Paul R.P., Kinematic Control Equations
for Simple Manipulators, IEEESMC-11, No.6

354
 Figure 3: Teleoperation: Dominant t r d a t i o n a i components of motion

II - I

Figure 4: Teleoperation: Translational components of motion

355