How XR will shape

media, entertainment,
and advertising
 How XR will shape media, entertainment, and advertising
XR had a great year
All the talk of VR disillusionment aside,
2017 was a very good year.
Apple, Amazon, Microsoft, Google, Facebook,
Sony, and others increased investments to
make virtual reality, augmented reality, and
mixed reality (VR, AR, MR respectively, XR
collectively) available to billions of people.
Many of the most visible applications may seem
silly – from doggie-faced selfie filters to pocket-
monster hunting apps – but the the reality is
quite serious: these technologies are acclimating
the world to a state of altered reality.
XR is still only in its infancy, and as brands adopt it
they’ll harness an entirely new medium to broadcast
messages that the world has never seen.
2
Insights from NYC Media Lab’s
Exploring Future Reality conference,
November 30th, 2017
Research based in part on panels with:
• Ori Inbar, Founder, Super Ventures
• Matthieu Lorrain, Head of Creative
Innovation, The ZOO, Google
• Agatha Bochenek, Head of AR/VR
& Mobile Ad Sales, Unity
• Chris Hercik, Chief Creative Officer,
The Foundry, Time Inc.
• Roger Kenny, VR Design Tech Lead,
Dow Jones Innovation Lab
• Rori Duboff, Managing Director of Content
Innovation, Accenture Interactive
• Steven Feiner, Professor of Computer
Science, Columbia University
• Matt Hartman, Partner, Betaworks Ventures
• Jessica Lauretti, Global Head, RYOT
• Kyle Li, Assistant Professor, The New School
• Can Misirlioglu, Group Creative Director, Studio6
• Tereza Nemessanyi, Entrepreneur-
In-Residence, Microsoft
• Chaki Ng, Senior Vice President, Viacom Next
• Ken Perlin, Professor of Computer
Science, NYU Future Reality Lab
• Agnieszka Roginska, Music Associate
Professor, NYU Steinhardt
• Keith Soljacich, Vice President of
Emerging Technology, Digitas
• Adaora Udoji, Investor & Producer
• Yasmin Elayat, Co-Founder and
Creative Director, Scatter
How XR will shape media, entertainment, and advertising
1. The Medium - This is not TV 2.0
For those investors hoping that we’d soon see a VR
headset where every TV once stood, 2017 was probably
a rude awakening. VR is not a TV replacement. It’s an
entirely new medium that requires its own approach.
It’s perhaps better to think of XR as a new set of very
different digital channels. As videographers, game
designers, journalists, playwrights, and creators of
all stripes explore it, they’re finding their old tools
don’t quite fit. There are no optimal camera angles
for an immersed viewer. Movement poses problems
for tethered participants. And storytelling takes on a
new meaning when viewers have freedom of choice.
Stereoscopic technology
1889 2014
For AR, the outdoors pose a challenge. It’s difficult to
control for lighting conditions or hazardous objects
such as moving vehicles, and the system requirements
needed to layer virtual objects upon actual ones – a
process known as registration – are prohibitively high.
Hardware and content creators must work more
closely and evolve beyond cheap tricks if they want
to deliver on the promise of ‘immersion’. Volumetric
video and 3D audio capture are needed to help
viewers believe that objects in VR are moving in
space. Creators must harness other senses, such as
touch and smell, and XR devices must get smarter.
The XR medium will open new doors to human
creation and perception. But before that happens, it
must overcome its present technological hurdles.
For one, headsets are still clunky, tethered,
underpowered, and largely reliant on the same
technology used in Victorian-era stereoscopes which
used side-by-side pictures to create an illusion of
three dimensions. This induces nausea in some and
it’s only with technologies such as light-field viewing,
the innovation behind MR startup Magic Leap, that
viewers will comfortably stand long sessions.
Something new
2016 2018
For that to happen, artificial intelligence (AI) will be
critical. AI will allow XR devices to learn user behaviors
and populate virtual worlds. “They’re inseparable,”
said Ori Inbar, founder of Super Ventures. “In the near
future, one will not be able to work without the other.”
More sensors and increased connectivity will also
smooth out VR experiences. Steven Feiner, Professor
of Computer Science at Columbia University, identified
the use of eye-tracking technology as a way to
anticipate users’ needs. And Inbar asserted that a
cloud storage system would allow XR systems to
exhibit object permanence over multiple sessions
and connect users in multiplayer experiences.
3
How XR will shape media, entertainment, and advertising

“Until now most of the AR experiences are typically one

person, one place, one device,” said Inbar. Once people
are connected and there is a network effect, the entire
landscape will change.

The spectrum of XR tech challenges

Wireless headsets Cross-platform Light field view/visuals Room scale

extensibility interact with environment

Faster computing XR Cloud

Volumetric capture Vergence accommodation

mismatch

Full spectrum of senses Eye-tracking technology

(3D audio, haptic feedback,
smell, biometrics, etc.)
Conversational interface

Full human field of view Siloed hardware/

software creation

4
How XR will shape media, entertainment, and advertising
2. The Message - Brands are gaining fluency
Consumers crave just the sort of new perspectives that
XR can deliver. According to the Harvard Business
Review, all brands must ask themselves not whether to
participate but instead, “What is our VR/AR strategy?”
“It’s a new media, but we’re
still using old metrics”
Matthieu Lorrain, Head of Creative
Innovation, The ZOO, Google
Just as with the web and social media before,
brands must define what they stand for as they
figure out how to articulate it. They must evolve
beyond storyboards and sample clips, which no
longer suffice in an immersive medium. The same
goes for measurements. “It’s a new media, but
we’re still using old metrics,” said Matthieu Lorrain,
Head of Creative Innovation at The ZOO at Google.
“Brands are still thinking in TV terms and aren’t
measuring empathy. Hopefully that will come soon.”
What’s the best way for brands to approach XR? Chris
Hercik, Chief Creative Officer at the Foundry, Time
Inc., thinks it’s by experimenting heavily and early.
Google has taken a similar approach. “There are a lot
of pilots and a lot of experimentation to understand
the right platforms for the right stories,” said Lorrain.
Which brands are leading
the XR conversation?
XR allows media brands to turn up the emotional
volume on their storytelling. It goes beyond text,
images, and video to fool the senses and create the
feeling of complete transportation. Agatha Bochenek,
Head of Mobile VR/AR Ad Sales at Unity, the engine
upon which much of VR content is built, attested to
this. Her team monitored players of a VR horror game
and found that they exhibited the same biometric
responses as they would if the terrors were real.
Roger Kenny, VR Design Tech Lead at Dow Jones,
decided that VR was a good fit for the Wall Street
Journal after asking “What’s the most compelling
way we could possibly tell a story?” The Wall Street
Journal’s 360-degree videos aim to tell stories that
offer a more immersive experience. The New York
Times and The Economist have similar initiatives.
According to Hercik, LIFE magazine is finally realizing
it’s founder’s vision to transport people anywhere
by including AR activations in print editions. “We’re
establishing a single app that does VR and AR
that all of our brands feed into. We’re building a
massive audience for a single brand,” said Hercik.
5
How XR will shape media, entertainment, and advertising
“How do you achieve
mass-reach engagement
but still have it be deep
and meaningful?”
Agatha Bochenek, Head of AR/VR Ad
Sales, Unity
XR is also making advertising and commerce much
more immersive. The agency RYOT Studio uses
QR code activations to bring movie posters to
life and as of this year, AR shopping has become
commonplace for retailers such as IKEA and Amazon.
The advertising industry has also embraced XR.
Bochenek and her team are creating an ad platform
with units that run cross-device and populate in
multiple formats. “How do you achieve mass-
reach engagement but still have it be deep and
meaningful, rather than just inserting a 2D object?”
she asked. “That’s what we’re exploring.”
Films like HYPER-REALITY by Keiichi Matsuda
catalog all the ways that VR and AR could go wrong
by depicting a world cluttered with virtual ads. But
Lorrain believes that consumers will have the final
say. “I’m sure someone will create the AR version
of pop-up ads. But at the end of the day it’s also
about what users choose to interact with. Hopefully
they won’t engage with those kind of things.”
Agencies and schools are
nurturing XR
XR storytelling is evolving most rapidly at advertising
and creative agencies and their associated studios,
which are pinched between a shifting media
economy and heightened pressure to prove their
worth. As agencies and brands bring each other
new ideas, they form a valuable melting pot from
which great XR innovations are emerging.
As the market expands, individual studios are
starting to specialize, a key indicator of long-term
growth. Scatter, a studio in Brooklyn, focuses on
volumetric filmmaking for documentaries while
Arup, an agency in London, has a fast growing
3D audio and acoustic consulting arm.
At schools, similar pressures to adapt are inspiring
innovation. The New School, New York University,
Columbia University, and the School of Visual Arts all
offer VR programs and workshops. And in 2018, New
York City will launch the first publicly funded lab for
virtual and augmented reality in the United States,
aided by a $6 million investment by the New York City
Economic Development Corporation (NYCEDC) and the
Mayor’s Office of Media and Entertainment (MOME).
NYC Media Lab will assist in its development, with the
intention of creating over 500 jobs within 10 years.
6
How XR will shape media, entertainment, and advertising
3. The Consumer -
What does a virtual humanity look like?
Today, VR in particular is still held back by “an ‘I’ve
tried it’ problem,” said Rori Duboff, Managing Director
of Content Innovation at Accenture Interactive.
Consumers are happy to try VR, check the box, and
move on – for a number of reasons. There are the
clear technological limitations, such as clunky headsets
and nausea-inducing images, but there is also the
lack of truly outstanding content. Without enough
emotionally evocative experiences, consumers see
VR as a novelty. But once they are truly moved, they’ll
see past the tech and talk about how it made them
feel, much like how they talk about movies rather
than about movie projector screens. If consumers
can be moved, that may be the tipping point.
Once the desire is there, consumer tastes can adapt
and VR may become habit. Though XR headsets
may still look strange to many, tastes change
quickly. It was only a decade ago that iPhones
launched and it became commonplace to see
people everywhere hunched over pocket-sized
mirrors. Where might we be in another ten years?
The path forward depends on brands. How can they
represent themselves in a virtual way? What stories
do they have to tell or new perspectives to offer? It
will be up to them to help consumers reach beyond
simple entertainment to make XR more than just tech.