Evolutionarily relevant methods for testing the generalizability of virtual reality.

Buck, E. E., Davis, R. N., Felton, W. M., Fulgueira, M. N., Lundholm K. R., McCormick, L. M.,
Onofre-Murillo, J., Pugliese, B. J., Ringer, K., Salinas, O. F. D., & Jackson R. E.

ABSTRACT METHOD RESULTS

Overview Data indicate that the virtual methods employed here approximated real-world behavior by roughly 92%
Virtual reality is a rapidly expanding method of studying real-world behavior, in We determined from Evolved Navigation Theory a set of distance estimates that are related to evolved (Table 1).
part because it provides researchers unrivaled experimental control. However, navigation risks. Participants made these distance estimates in real and virtual environments and then we All but one of the six vertical estimates correlated significantly between real-world and virtual-reality
measured the similarity across estimates as a means to quantify presence in virtual reality. We also tested
we have little ability to measure the extent to which virtual reality studies four key predictions from Evolved Navigation Theory that had not been tested previously in virtual reality.
estimates (the smallest of which r (20) = .66, p = .001). Estimates of the medium vertical surface from its
top correlated positively, but not significantly, r (20) = .33, p = .135. There was no significant correlation
generalize to real behavior. One limitation in current virtual reality research is Environment between any horizontal estimate in virtual-reality and the real-world, the average of which r (20) = .09, p =
the absence of evolutionarily relevant methods. The present study describes Twenty-two college student participants estimated distances across horizontal and vertical surfaces .690.
outdoors. Participants made the vertical estimates while standing at both the top and bottom of the These data clearly supported Evolved Navigation Theory Predictions 1, 3, and 4, with partial support for
an evolutionary method derived from Evolved Navigation Theory for testing the vertical surface (Figure 1). Prediction 2. Participants overestimated environmentally vertical surfaces (Prediction 1). Participants
generalizability of virtual reality findings. Twenty-two participants estimated Participants gave 18 total estimates that varied in three dimensions within-subject: modality, distance, overestimated vertical surfaces more while standing at the top than bottom (Prediction 3). Participants
and surface. Modality included either the real or virtual environments. Distance included three lengths of
distances in functionally identical real-world and virtual environments using an 1.32, 2.21 and 4.75 m. Surface included a horizontal surface or a vertical surface that participants
estimated horizontal surfaces roughly accurately (Prediction 4). The magnitude of vertical overestimates
increased with the length of the surface, but only on vertical distances estimated from the top (Prediction 2).
evolutionarily relevant distance matching procedure. The results of the study estimated while standing at its top or bottom. We partially randomized participants’ estimates across these
indicate that this method performs superior to nonevolutionary methods and three dimensions.
Participants completed the virtual portion of their estimates in an indoor room with a single-user head
provides greater precision in calculating the extent to which virtual findings mounted display. We constructed the virtual reality environment to be as similar as possible to the real-
generalize to real-world behavior. We recommend that future virtual reality world environment in actual distances of surfaces, texture, and major structures.
Estimation Method
experiments include a concise version of this task in order to determine the Two white dots indicated the distance for participants to estimate. Participants stood near one of the
extent to which their findings reflect real behavior. white dots. Participants then told the researcher to move an orange adjustment dot away from the near
white dot until the distance between them looked the same as the distance between the two white dots.
All white dots were fixed in the environment. In the real-world, the orange adjustment dot was affixed to
a measurement wheel operated by a researcher. In the virtual environment, the researcher instead moved
an isolated orange adjustment dot with electronic control.
Predictions
In addition to measuring the similarity of estimates across modality as a means to quantify presence,
we also tested four Evolved Navigation Theory predictions.
INTRODUCTION Prediction 1: Participants should overestimate environmentally vertical surfaces, also known as the Figure 2.
Mean distance estimates
environmental vertical illusion (Jackson & Cormack, 2008).
Prediction 2: The degree of vertical overestimates should increase with the length of the surface across modality, distance,
Virtual reality is rapidly becoming a prevalent way to conduct research on human behavior (Kane, and surface. Dotted lines
(Jackson & Cormack, 2008).
McCall, Collins, & Blascovich, 2012; Parsons & Mitchell, 2002). This technology offers unprecedented represent the actual
Prediction 3: The degree of vertical overestimates should be greater when participants stand on top
access to research manipulations that would otherwise be unavailable, unethical, dangerous, or too distances. Error bars
rather than the bottom of the vertical surface, also known as the descent illusion (Jackson, 2005).
expensive. represent 95%
Prediction 4: Participants should estimate horizontal surfaces roughly accurately (Jackson & Willey,
When researchers use virtual reality technology, they assume that participants’ behavior and confidence intervals.
2011, 2013).
perception in VR matches the behavior and perception in the real world. However, this is oftentimes not
the case (Jackson & Cormack, 2010; Kober & Neuper, 2013; Powers et al. 2013; Riecke, 2008). This
poses a serious problem for drawing conclusions about real-world behavior.
“Presence” is the concept of how real the virtual environment feels to the participant (Burdea &
Coiffet, 2003; Sanchez-Vives & Slater, 2005; Stone, 2011). Generally, the more real that the environment Figure 1.
feels (i.e. higher presence), the more likely that findings will generalize to real-world behavior. Participant positions
Unfortunately, current methods for estimating presence are subjective and contain demand during three
characteristics that undermine their validity (Insko, 2003; Witmer & Singer, 1998). This adds to the estimates. Dotted
inability of virtual reality to generalize, because even the data that measure its validity are potentially icons indicate the
invalid. same participant
In the current study, we tested a method for measuring presence via evolved phenomena. Evolved during three
navigation theory identifies evolved mechanisms that reflect real-world navigation risks (Jackson, 2005; different estimates.
Jackson & Cormack, 2007). These mechanisms are not conscious to observers, which provides a means Blue dashes indicate
to avoid the demand characteristics of current presence measurements. the vertical surface
We measured the difference between real and virtual responses to evolved visual illusions as a means estimated from
to quantify presence in virtual reality. above and below.
Green dashes
DISCUSSION
indicates the
These data demonstrate a method for measuring presence that can quantify the generalizability of
horizontal surface.
virtual reality. This method is a unique improvement because it does not pose the demand characteristics of
Grey dashes
current measures of presence. Researchers can include the procedures outlined here as a means for
indicate each path
determining the generalizability of their virtual data.
of estimation.
Evolved navigation theory provided this method in ways not otherwise available. This theory suggests
REFERENCES that the mechanisms that we tested are neither conscious, nor do they reflect any mental calculation of
falling risks. Evolved navigation theory identifies that these are dummy mechanisms resulting from selection
Burdea, G. C., & Coiffet, P. (2003). Virtual Reality Technology, 2nd Ed. Hoboken, NJ: John Wiley & Sons.
Insko, B. E. (2003). Measuring presence: Subjective, behavioral, and physiological methods. In G. Riva, F. Davide, W.A IJsselsteijn (Eds.), Being There: over evolutionary time. A benefit of this lack of direct calculation is that these mechanisms are not
Concepts, Effects and Measurement of User Presence in Synthetic Environments (109-119). Amsterdam, The Netherlands: Ios Press.
Jackson, R. E. (2005). Falling towards a theory of the vertical-horizontal illusion. Studies in Perception and Action, 8, 241-244.
Table 1: Extent to which distance estimates in virtual-reality replicated consciously available to participants. The current method of testing distance perception is not subject to the
Jackson, R. E. (2009). Individual differences in distance perception. Proceedings of the Royal Society B: Biological Sciences, 276, 1665-1669. demand characteristics featured in other methods of measuring presence.
Jackson, R. E., & Cormack, L. K. (2007). Evolved navigation theory and the descent illusion. Perception & Psychophysics, 69(3), 353-362. Virtual-reality methods provide unique advantages. Using a virtual environment is beneficial for
Jackson, R. E., & Cormack, L. K. (2008). Evolved navigation theory and the environmental vertical illusion. Evolution and Human Behavior, 29 (5), 299-304.
doi: 10.1016/j.evolhumbehav.2008.03.001
estimates in the real-world (virtual-reality / real-world). experimental replication, as the environment will remain identical during each condition. Further,
Jackson, R. E., & Cormack, L. K. (2010). Reducing the presence of navigation risk eliminates strong environmental illusions. Journal of Vision, 10(5): 9, 1-8. researchers can easily recreate virtual settings that would otherwise pose a risk to participants, as in
doi:10.1167/10.5.9 occupations such as mining or when testing conditions such as acrophobia (Jackson, 2009).
Jackson, R. E., & Willey, C. R. (2011). Evolved navigation theory and horizontal visual illusions. Cognition, 119, 288-294. doi:10.1016/j.cognition.2010.11.003
Jackson, R. E., & Willey, C. R. (2013). Evolved navigation theory and the plateau illusion. Cognition, 128, 119-126.
Surface Virtual-reality research methods are generally only as useful as the extent to which they generalize to
Kane, H. S., McCall, C., Collins, N. L., & Blascovich, J. (2012). Mere presence is not enough: Responsive support in a virtual world. Journal of Experimental real-world behavior. The current study identifies an evolutionary method for quantifying virtual-reality
Social Psychology, 48, 37-44.
generalization as a way to capitalize on the strengths of this expanding research tool.
Kober, S. E., & Neuper, C. (2013). Personality and presence in virtual reality: Does their relationship depend on the used presence measure? International
Journal of Human-Computer Interaction, 29 (1), 13-25. doi:10.1080/10447318.2012.668131
Length Horizontal Vertical, Bottom Vertical, Top Mean
Parsons, S. & Mitchell, P. (2002). The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual
Disability Research, 46(5), 430-443.
Powers, M. P., Briceno, N. F., Gresham, R., Jouriles, E. N., Emmelkamp, P. M., & Smits, J. A., (2013). Do conversations with virtual avatars increase feelings of
social anxiety? Journal of Anxiety Disorders, 27, 398-403.
Short (1.32) 103% 90% 110% 101%
Riecke, B. E. (2008). Consistent left-right reversals for visual path integration in virtual reality: More than a failure to update one’s heading? Presence, 17(2).
143-175.
Sanchez-Vives, M. V., & Slater, M. (2005). From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6(4), 332-339.
Stone, R. J. (2011). The (human) science of medical virtual learning environments. Philosophical Transactions of the Royal Society: B, 366, 276-285. doi:
Medium (2.21) 109% 79% 79% 89% ACKNOWLEDGEMENTS
10.1098/rstb.2010.0209
Witmer, R. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual
Environments, 7 (3), 225-240. doi:10.1162/105474698565686 Long (4.75) 102% 90% 70% 88% We appreciated programming and technology assistance by Roger Lew, data gathering by James Founds, and
data entry by James Founds and Susan Klebba.

Mean 105% 86% 86% 92%