Sei sulla pagina 1di 29



Saku Panditharatne
Formerly VR investing & data science @ a16z deal team
BA from Cambridge in Maths & CS
Internships Oculus, other startups
Why build a 'game engine
for programmers'?
Computer graphics is set explode in importance

Current game engines do not meet the needs of future developers

Game engines may be able to capture a lot of the value that comes from
these new visual computing platforms

Unreal engine, photo credit

The rising importance of
computer graphics
Mobile Augmented Reality (e.g. Snap, Pokemon Go)

AR headsets (Hololens)

VR gaming (Oculus)

Simulation for autonomy & machine learning (e.g. drones, delivery robots)

AR, photo credit

VR and AR will be here
within the next few years
2018 mobile AR install base- 100m Android, 500m iPhone

Next Oculus Rift expected around 2019

Microsoft - "Hololens is a 5-year project" - perfected by 2021

Optimistic case on AR is itll replace everything we do with smartphones

VR, if it takes o, could be the future of games consoles

3D -> VR might be like the switch from 2D->3D games. It was 10 years from
the first hit (Wolfenstein) to the mainstream console (PS2)
The importance of simulation
to augmented reality
Convergence in hardware: Hololens will have an AI co-processor chip

Next Oculus will have SLAM tracking

Wearables are potential inputs to augmented reality

Important to both are: object recognition & computer vision, sensor input and
accurate physical simulation

3d scans, photo credit

What's missing in the game
engines we have now?
What is in a game engine?
Today - its not really graphics or physics that dierentiate a game engine.
These things can be found in open source.

Most of the code in game engines is about handling and processing content
(3D models, textures, animations) allowing designers to organize it into a
video game

The other important function of game engines is compiling the game for
multiple platforms
What do game engines
Looking at Unreal Engines features

Red = core features

Yellow = extensions

Blue = for non-programers

Green = platform compatibility

What is holding back
game engines now?
O-the-shelf game engines are aimed at teams of not only programmers, but
artists and designers

Open source is not common in the gaming world - possibly because

software engineers in the game industry have less power to pressure their
employers to adopt it

O-the-shelf game engines are not equipped to handle complex coding,

because major studios, if they want something better, build it in-house

This will change: AR developers likely to be less content driven than game
studios, with more solo app developers
Current game engines
Mostly spin-os of game studios - exception is Unity

Multiplatform compatibility - PC, mobile, tablet

Aimed at teams of artists, designers and programmers, rather than

programmers themselves

Tools are aimed at the specific case of organizing 3D content into a video

Not much open source - instead of a package manager Unity has an asset

Dicult to modify or extend

What we need from game
engines now
Multiplatform for a dierent set of platforms - Oculus, Hololens, ARKit

Integrate with sensor input, computer vision, SLAM

Realism in physics, physically based rending by default

O the shelf 3D models, perhaps via an asset store

Increased importance of scripting, tailored to the needs of programmers only

Easy to modify - package managers instead of app stores

Where will the value be
captured in visual computing?
What will AR be used for?
Many obvious specific applications: Retail, real estate, education, health,
social networking, video streaming

Possibility: replaces the interface for desktop computers

More exciting possibility: replaces the function of mobile phones

(from Hololens)
The AR/VR Ecosystem
The split of value between core chips, industrial design, and software: the PC
industry was 45%, 10% 45%, smartphone industry is 30%, 35%, 35%

For both VR and AR, it seems that 3rd party software - games and apps -
will be more important than it was for smartphones. The iPhone was
already very useful without the app store.

The VR split is something like 30% 25% 45%, resembling smartphones

somewhat. Current VR headset makers are already trying to court developers
(see how Vive is currently winning the battle with Steam).

Unlike mobile, AR headsets are quite complex and unlikely to become

commodities quickly. The split might be something like 20% 35% 45%. This
is more similar to game consoles, where you have each platform being
optimized for something quite dierent (compare Wii vs. PS3)
Why do game engines
With both VR & AR, platform wars will be intense and competition for
developers will be strong. It might be more like game consoles than

Unlike during the console wars, computer graphics is already a mature field,
so app & game developers are less likely to compete on graphics

The leverage a game engine has against a headset manufacturer is it's

community of developers

console wars, photo credit

Possible moats for game
Network eect from developer community is the obvious way, and how Unity
currently maintains its advantage

Most promising: a package manager & open source ecosystem around the
game engine (current game engines dont have this because open source is
rare in the entertainment industry)

An asset store comprehensive enough to actually allow game developers to

cut down on the number of artists they have to hire

Cloud services for apps & games (e.g. with AR, spatial co-ordination
between dierent apps - you place a object at a certain latitude and
3D content
Always been a hugely expensive part of game development and a barrier for
indie developers

Most 3D models are produced in-house, or bought from smaller studios or


3D model marketplaces havent worked because content needs are very

specific, so the chicken-and-egg problem is hard to overcome

Emerging techniques for programmatically tweaking 3D modeling might

change that
Several potential revenue

Royalty-based business model SaaS business model for Marketplace model for 3D models
for VR/AR apps created with Asteroid cloud services for VR & AR apps bought on the Asteroid store
Why build a 'game engine
for programmers'?
Computer graphics is set explode in importance

Current game engines do not meet the needs of future developers

Game engines may be able to capture a lot of the value that comes from
these new visual computing platforms

Game render, photo credit

Whats dierent about a game
engine for programmers?
Lightweight & extensible - Ruby on Rails style approach to game engines

The game engine is closed source, but with a flexible API for add-ons

Scripting interface / console where every object & process is referenced

Provides framework to modify & replace components of a modular pipeline

Example: the asset pipeline, processing textures & generating animations

Developer ecosystem: package manager, forums, screencasts, tutorials etc

High-level strategy
Build a prototype game engine for a small niche of programmers who really
need a programming-centric game engine - autonomy & robotics simulation

Start to build a developer community and ecosystem around the game


By the end of 2017: expand to mobile AR, creating experiences for ARKit or

Next year: expand to VR, adding more complex features relevant to high-
quality game development (e.g. asset pipeline)

Further in the future: expand to AR headsets, create a 3D asset library, build

out cloud services for apps & games depending on what they build
Game engine for robotics simulation

MVP is a Unity Plugin + a cloud service for rendering the simulation

Each frame is retrieved via API call

Launched Asteroid Cloud Service - August 2017

Release v1 Asteroid desktop Mac app - November 2017

Grow the developer community, build open source ecosystem

Make Asteroid compatible with ARKit - by Jan 2018

Launch version with the complex features needed for VR development - June

Launch cloud services for mobile AR / VR games - Sep 2018

Asteroid Mac App
Lightweight 3D viewer & content manager, combined with scripting interface

Extensible - users can add configuration panels or scripts

GUI to modify & replace components of a modular pipeline

Metal 2 on Mac supports graphics sophisticated enough for desktop VR

By using Apples SceneKit for rendering, code can be reused for ARKit

Expanding to other operating systems comes later - most game engines are
designed with Windows in mind, which may not be the platform for AR & VR
Specific extra features for
simulation & mobile AR
Accurate physics engine and representation of robotics (can rely on open
source engine Bullet)

Generate a large number of variations on a similar scene

Integration with cloud simulation service

Process camera input (e.g. filters) and sensor and peripheral input

Interaction between AR and the real world (e.g. using SLAM and accurate
physics, rendering)
Headsets and beyond
VR requires expensive, in-depth experiences

AR will likely borrow from tools & design principles developed for VR

Designing a VR world compared to a video game scene is like building the

entire movie set vs. a scripted sequence

In-VR editors have not proved popular, mostly because need complex
interfaces do not translate well to VR

Most VR headsets now just have a Unity plugin, rather than a toolkit of their

39% of game developers are already currently working on VR