9/4/2019 Understanding display techniques in Augmented Reality

Understanding display techniques in

Augmented Reality
Niteesh Yadav Follow
Oct 8, 2018 · 5 min read

This is Part-II of the series of articles in which I am discussing the typographic aspects of
Augmented reality. It is based on my ongoing research on typefaces for AR headsets which
started as part of my MA in Typeface Design at the Department of Typography and Graphic
Communication, University of Reading, UK. The articles will help the type designers and
interface designers to understand the intricacies of the text in AR to improve their workflow
and design process. Click to read the first article “Preparing for the better typography in
Augmented Reality”.

There are three basic techniques for showing visuals in AR which depend on the
proximity of display mediums (see figure 1). Handheld AR is a common method that
uses smartphones and tablets to show AR content, it gained popularity after the release
of gaming applications like Pokémon Go and Ingress. Another is using headsets that are
classified as optical see-through (OST) and video see-through (VST). OSTs are often
https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 1/6
9/4/2019 Understanding display techniques in Augmented Reality

termed as true AR at the moment (e.g. Hololens, Google Glass). New technologies that
are in development include spatial see-through displays which project content in 3D free
space using plasma in the air and retinal displays which projects directly on users’ retina.
To limit the scope of these articles I will focus on headset based techniques which pose
more typographic challenges as compared to any other mediums already in use, like
smartphones. (The handheld smartphone-based AR doesn’t bring any major shift since the
users see the screen in a conventional way by holding it in front of them.)

Figure 1: Diagram showing three di erent techniques and the position of the medium to present the Augmented
content over the real world environment. Widely used medium for Spatial AR: Projectors, Hand-held AR:
Smartphones and tablets, Head-attached AR: AR and VR headsets.

“The interface designers will have to be more

careful, driving away from trends and using research
to make well-informed decisions.”
Headset based augmented reality
The headset based augmented reality is bringing a whole new set of challenges both in
terms of typography and typefaces. In a lot of cases the text information acts as a
supplement to the main point of focus, example: showing location-based information
while driving, where the road is the main focus. In such cases, the role of text might
seem secondary but it would be more complex to deliver good cognitive performance.
https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 2/6
9/4/2019 Understanding display techniques in Augmented Reality

The amount of time to process the information is very short and legibility is one key
factor that can enhance reading experience. Getting back to the main topic of discussion,
the headsets are divided into two categories which have their own mechanism to display
the text which directly affects the how the users perceive them. The same typeface can
appear different based on the type of headset.

Video see-through
Video see-through is one of the affordable techniques to deliver AR experiences. In VST,
a camera captures a digital video image of the real world and transfers it to the graphics
processor in real- time. Then the graphics processor combines the video image feed with
computer-generated images (virtual content) and displays it on the screen (see figure 2:
left). Since the video processing is done before showing the content to the user, it is
possible to control the brightness and contrast of both real world and virtual elements
for a seamless experience. It also has a better registration through tracking of head
movement. However, there are drawbacks which include low resolution of reality
(screens don’t match the human eye resolution), a limited field of view (which is
possible to increase but it is expensive) and eye parallax (eye-offset) due to camera’s
position which is usually at a distance from the viewers exact eye location. The VSTs can
use smartphones like in Samsung Gear VR where the phone is used as a display places
few inches away from users eyes which is totally different from experiencing AR in
handheld smartphones.

Figure 2: Left, Optical see-through displays operate based on transparent optics that allows the user to see the
surrounding directly along with virtual content. Right: Video see-through displays are opaque; they use a camera to

https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 3/6
9/4/2019 Understanding display techniques in Augmented Reality

capture the users’ surrounding and then displays on a screen along with computer-generated images. Diagram
based on “Breakthroughs in optics that are reshaping augmented reality.”

Optical see-through
Optical see-through displays operate using optical elements (like half-silvered mirrors)
that are half-transmissive and half reflective to combine real world and virtual elements.
The mirror allows a sufficient amount of light from the real world to pass through
making it possible to see the surroundings directly. Simultaneously, computer-generated
images are projected on the mirror through a display component placed overhead or on
the side, which creates a perception of the combined world (see figure 2: right). OSTs
show the world in real resolution free from parallax (caused by the offset of camera
position in relation to the viewer’s eye). These are safer to operate since the user can see
even if the power fails, making it an ideal option for military and medical purposes.
However, the use of mirrors and lenses reduces the brightness and contrast of both
virtual and real-world perception.

Figure 3: A visual classi cation of popular VST and OST headsets in the market.

Time to learn from our mistakes

The interface designers will have to be more careful, driving away from trends and using
research to make well-informed decisions. One such clear case of trend based the
extensive use of square grotesque typefaces in most of the dashboard UI because it
became popular by movie depictions of future. MIT Age lab’s research [1] reveals a lot of
factors that are usually ignored by designers, increasing the cognitive load and thereby
increasing the probable threat to users’ life. Design for AR gives clear ground to design
things from scratch to learn from our mistakes from past and design for better future
interfaces.

https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 4/6
9/4/2019 Understanding display techniques in Augmented Reality

In the next article, I will share the technical aspects and the pros/cons of OSTs and VSTs,
which directly impact the visual elements such as text and visuals. If you find anything hard
to understand or you would like to share your feedback then feel free to leave a comment
below. For updates on upcoming articles you can join me on Twitter @Niteesh22 .

Link to the next article:

The current state of the text in Augmented Reality

Credits:
Editing: Sérgio Martins

References/ Further Reading:

[1] Jonathan Dobres, Nadine Chahine, Bryan Reimer, Bruce Mehler, & Joseph Coughlin,
“Revealing Differences in Legibility Between Typefaces Using Psychophysical Techniques:
Implications for Glance Time and Cognitive Processing”,
http://agelab.mit.edu/files/Typeface/MT-MIT-Latin-Word_Recognition-White-Paper-
2014-06-24.pdf (last accessed on 6 October 2018).

Schmalstieg D. and Hollerer T., Augmented Reality: Principles and Practice, Addison-
Wesley Professional, 2016.

. . .

Join our weekly newsle er

Get a weekly digest of handpicked design news and tools!

Email

Si
https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 5/6
9/4/2019 Understanding display techniques in Augmented Reality
Sign up

I agree to leave Blog.prototypr.io and submit this informa on, which will be
collected and used according to Upscribe's privacy policy.

 Formed on Upscribe

Augmented Reality Typography User Experience Research User Interface

About Help Legal

https://blog.prototypr.io/understanding-display-techniques-in-augmented-reality-c258b911b5c9 6/6