UNITED STATES DISTRICT COURT

EASTERN DISTRICT OF WISCONSIN

CANDY LAB, INC.,

Plaintiff,
v.

MILWAUKEE COUNTY, et al,

Defendants,

_____________________________________/

DECLARATION OF MARK SKWAREK

I, Mark Skwarek, hereby declare, upon penalty of perjury, that the following is true and

correct to the best of my knowledge and belief:

1. I am a New York resident. I have first-hand knowledge of the facts stated here

and am competent to testify to them if called to do so.

2. I submit this declaration in support of Candy Lab ARs challenge to Milwaukee

Countys ordinance regulating the publication of location-based augmented reality games. I am

gravely concerned that laws like this pose a serious threat to the emerging medium of augmented

reality, which promises to be one of the most important and innovative ways that human beings

will use to communicate with each other in the 21st Century.

My Background in AR

3. I earned my M.F.A. from Rhode Island School of Design's Digital Media

Department. As a Lecturer of Integrated Digital Media teaching 3D Graphics and the Augmented

Reality graduate-level class, I am a full-time faculty member at New York Universitys School

of Engineering and the director of NYU's Mobile Augmented Reality Lab. I am also the founder

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 1 of 11 Document 3

and CEO of Semblance Augmented Reality, a company based in Brooklyn, New York that

publishes augmented reality games.

4. I am an artist working to bridge the gap between the virtual and physical world

with augmented reality. My art explores the translation of our everyday digital experiences into

the physical world using mobile augmented reality.

5. My artwork has been written about by the New York Times, Art in America,

Boing Boing, WIRED, the Boston Globe, The Huffington Post, NPR, BBC, the Discovery

Channel, Leonardo, and Creative Capital.

6. I have exhibited in various venues, including: the Institute of Contemporary Art,

Boston; ISEA; Dumbo Arts Festival, UCLA Digital Grad Gallery; the CyberArts Festival; the

Sunshine International Art Museum, Beijing; and the Krannert Art Museum at the University of

Illinois, FACT in Liverpool England, Siggraph 2013, The 2013 Augmented World Expo, The

Corcoran Gallery of Art, Kasa Galeri, and Contemporary Istanbul.

Political Speech in Augmented Reality

7. Because AR, in and of itself, is simply a medium for the publication of any type

of content someone wants to express, it is, of course, a potential realm for political and civic

speech. We have already seen several examples of such speech in AR, including some that might

also be arguably considered games as well.

8. My practice is also largely based in art activism. I organized the augmented

reality artist group manifest.AR, the arOCCUPYWALLSTREET movement, and co-organized

We AR in MoMA. I have a long record of international augmented reality work, ranging from

erasing the DMZ battlements between North and South Korea (a piece I did on-site), to the

virtual elimination of the barricades between Palestine and Israel at the Gaza Strip. I have created

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 2 of 11 Document 3

political work and symbols in a variety of locations across the United States, including pieces at

Wall Stret, the U.S. Mexico Border, and the White House, to name a few.

9. My application, called AR Occupy Wall Street, styled itself as a call to all AR

activists, and collected a series of protest-themed images from various AR designers:

10. The Occupy protests of 2011 extended into the AR medium in other ways as

well. One example is how some of the protestors resorted to free AR apps to keep the public

informed about related events and locations. Using Metaios location-based AR browser junaio,

one AR developer launched an Occupy channel that provides locations, contact information,

and resources for all the Occupy protests in various cities across the country. The Occupy Wall

Street group in New York took this idea one step further, using junaio to superimpose signs,

placards, and related imagery over areas from which they were restricted from physically

protesting.

11. In 2010, with Joseph Hocking, I released the leak in your home town, an iPhone

app that let users see a representation ofthe BP Deepwater Horizon oil spill whenever they see a

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 3 of 11 Document 3

BP logo. Opening the app and aiming the iPhones camera at the nearest BP logo triggered an

image of one of the broken BP pipes coming out of the BP logo, and out of the pipe comes the

oil, pluming upward.

12. HIT Lab NZ developed the app CityViewAR allowed people to see what

Christchurch, New Zealand's central business district looked like before the 2010 earthquakes

and subsequent building demolitions. Using a mobile phone, users can walk around the city and

see life-sized virtual models of buildings before they were demolished, as well as photographs

and historical information, overlaid on the now vacant site. The apps creators also envisioned it

as allowing citizens to vote on virtual models of proposed replacement buildings before they

were constructed.

13. The Green Party in Germany partnered with Metaio to launch an app that lets

constituents leave comments geo-tagged to specialized billboards and specific physical locations

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 4 of 11 Document 3

and that represent a certain issue and to hear pre-recorded statements by party officials about

those very issues.

14. During the 2012 presidential campaign in the U.S., AR startup GoldRun

announced a feature called Visualize the Vote that lets users pose for a picture with their

favorite presidential candidate super imposed over the users physical locationthen share that

photo with their friends.

15. GoldRun also announced plans to launch a location-based reminder service that

automatically alerts you to a particular cause when you come within a certain distance of a

related location. The first example mentioned was showing users the image of a dog or cat when

they walk within a mile of an animal shelter. But this same technique could easily be applied to

any social or political issue. Driving over a bridge might bring you a layer of information about

the pork barrel spending that went into funding it. Entering a road construction area might

prompt data on your tax dollars at work, or perhaps information about that companys safety

record.

16. Professor BC Biermann and the New York-based Public Ad Campaign launched a

project called AR Ad Takeover. This mobile app used feature tracking to recognize particular

print advertisements that were then prominent across New York City. The app then

superimposed original art on top of those ads, essentially replacing their commercial message

with an expression of the Campaigns choosing. The creators said that public space and the

publics interaction with that space is a vital component of our citys health, and that they

consider outdoor advertising to be the primary obstacle to open public communications. Their

mission was to air grievances in the court of public opinion and witness communities regain

control of the spaces they occupy.

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 5 of 11 Document 3

Location-Based Augmented Reality Games

17. Some mobile games are location-based. As I use and understand this term,

location-based mobile games are those that display digital content in reaction to a devices

proximity to specified geographic locations.

18. Not all location-based games are augmented reality games. The two terms

describe different things. Dozens of games available on the market are location-basedin that

they require a user to be physically present in certain geolocations in order to take certain actions

or interact with certain contentbut are not AR games, because they display their content

entirely in a two-dimensional fashion.

19. Similarly, not all AR games are location-based in the sense that they rely on

GPS or other sensors to determine a users real-time physical location and incorporate it into the

gameplay, as the term location-based game typically indicates. Instead, an AR game may

superimpose the same digital content on a devices surroundings regardless of where the device

is.

20. Therefore, I use the term location-based augmented reality game to mean a

mobile game that both incorporates the users real-time physical location into the gameplay, and

that displays content in an AR format. I have often heard the term used in the same manner by

others, but I am not aware that the phrase has an officially defined meaning.

21. In a manner of speaking, however, any AR displaywhether meant for gaming or

any other purposeis location-based, and has some relationship to the users physical

location, because AR displays by their very nature are meant to be blended with the users view

of their physical surroundings. With this understanding, even an application that displays the

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 6 of 11 Document 3

same digital image regardless of physical location could be said to be location-based, in that the

location and the image blend to form a unique expression.

22. During my time with Semblance, I have helped design and publish applications

that fit my use of the term location-based augmented reality games. Examples include:

a. Play AR: In 2015, we launched Play AR, which is both a digital creation platform

and a mobile game. It allows players to create their own digital content and

place it in the physical world, so that it becomes visible on other players

mobile devices when those other players are in the vicinity of the chosen

geolocation. The exact geolocation to which any given piece of content is

assigned is entirely up to the user. Play AR also allows players to battle each

other using those digital objects. The image below shows a user viewing a giant

squid that someone placed in the waterway in front of him:

b. Dimension Breaker: Also released in 2015, Dimension Breaker is a simple wall

breaking game which recognizes any physical wall at which the user points it,

then overlays that wall with a digital wall that the player can break with digital

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 7 of 11 Document 3

 balls. This game can be played wherever the user is and is not dependent on

being in a specific geolocation. Players may, however, also detect and take over

games that other users have played in their specific locationwhich makes the

game location-based, but in a different way than games like Play AR or Texas

Rope Em, in that the location of current and prior games within Dimension

Breaker is entirely user-generated. The image below shows a user playing the

game on the building visible through the window in front of him:

The Milwaukee Ordinance

23. I only recently learned of the ordinance passed by Milwaukee County regulating

location-based augmented reality games. Specifically, I see that it prohibits companies from

introducing location-based AR games into the countys parks without a permit. Whoever

wrote this language must not understand how AR works, because game publishers do not

introduce their games into any physical location. The games are simply software that play a

mix of locally stored and internet-based content, and that choose what content to display based in

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 8 of 11 Document 3

part on the real-time geolocation of the mobile device running the software. I cannot say with

any degree of certainty, therefore, what this language means.

24. Reading this language with the understanding that it was meant to regulate games

like Pokemon Go, however, I assume it was meant to prohibit companies from publishing

location-based AR games that are playable in Milwaukee County Parks. That worries me,

because Play AR and Dimension Breaker are both playable anywhere a player wants to play

them, including in Milwaukee Parks. It concerns me that Milwaukee may now try to regulate my

games or force me to obtain a costly permit.

25. If Semblance was forced to seek and pay for a permit just so players could play

my games in a certain area, we would be forced to either somehow find a way to modify our

games to make them unplayable in Milwaukee, or else pull them off the market entirely. Either

option would diminish Semblances ability to do business or destroy it entirely, especially if

other municipalities follow Milwaukees lead.

26. At the very least, the Ordinance forces me to second-guess whether and how

Semblance or I can continue to make anything that may be considered a location-based AR

game available to the public.

27. I also have no idea what the Ordinance means by location-based. As explained

above, both Play AR and Dimension Breaker are games I consider location-based, but in

different ways. The Ordinance doesnt tell me how it defines the term.

28. Similarly, I dont know what the Ordinance means by augmented reality. There

are a lot of different perspectives in various communities as to what that term means. Some

people may not consider Play AR or Dimension Breaker to be AR, for example, even though I

do, and the Ordinance does not define it.

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 9 of 11 Document 3

 29. Likewise, the Ordinance does not define game, and that is not a self-defining

term. Lots of applications that most people would not typically think of as games nevertheless

incorporate game-like features to incentive people to use them, such as how Foursquare and Yelp

(which are way-finding and consumer review apps) allow users to check in to certain

locations, and award titles to the person with the most check-ins. By the same token, many

perhaps mostapps that are typically considered games also have significant educational

and/or practical utility.

30. The Ordinance is also seriously vague about who it applies to. For example, the

popular AR platforms Wikitude and Vuforia offer content-creation tools (called Software

Development Kits or SDKs) for sale to developers, who then create applications through the

software and market them to the public. The end products created with these SDKs are still

primarily composed of software authored by the SDK publisher. Does that mean that Wikitude

or Vuforia will be subject to regulation by Milwaukee every time one of their customers creates

an application that can be played in Milwaukee parks? If so, that prospect would severely disrupt

the AR industry. Metaio provided a similar SDK that I used to make many of my apps until the

company was purchased by Apple; its sudden disappearance forced me to abandon or completely

rewrite the applications I had made with their software.

31. I also fear that, if laws like this Ordinance are upheld, there is no reason to believe

they will stop at location-based AR games. If these can be regulated, why expect governments to

stop there? They may decide to impose onerous regulations on all AR content for whatever

reason, or prevent developers from adding AR content of any type to a particular location.

Because all AR content is inherently location-based to some degree, as I explained above,

reserving this power to government would devastate the entire AR industry.

10

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 10 of 11 Document 3

Further Declarant Sayeth Not.

Dated: April 18, 2017

Mark Skwarek

11

Case 2:17-cv-00569-JPS Filed 04/21/17 Page 11 of 11 Document 3