Projected Interactive Display for Public Spaces

1. Introduction

1.1 BACKGROUND OF THE STUDY

Human-computer interaction (HCI) is the study of the interaction between the users

and the computers. The basic goal of HCI is to improve the interaction between users and

computers by making the computers more user-friendly and accessible to users. HCI in

the large is an interdisciplinary area. It is emerging as a specialty concern within several

disciplines, each with different emphases: computer science, psychology, sociology and

industrial design (Hewett et. al., 1996). The ultimate goal of HCI is to design systems that

would minimize the barrier between the human’s cognitive model of what they want to

accomplish and the computer’s understanding of the user’s task.

The thesis applies a new way to interact with sources of information using an

interactive projected display. For a long time the ubiquitous mouse and keyboard has

been used to control a graphical display. With the advent of increased processing power

and technology, there has been great interest from the academic and commercial sector in

developing new and innovative human computer interfaces in the past decades. (Myers

et. al., 1996). Recently advances and research in human computer interaction (HCI) has

paved the way for techniques such as vision, sound, speech recognition, and context-

aware devices that allow for a much richer, multimodal interaction between man and

machine. (Turk, 1998; Porta, 2002). This type of recent research moves away from

traditional input devices which are essentially blind into the so called Perceptual User

Interfaces (PUI). PUI are interfaces that emulate the natural capabilities of humans to
sense, perceive, and reason. It models human-computer interaction after human-human

interaction. Some of the advantages of PUIs are as follows: (1) it reduces the dependence

on being in proximity that is required by keyboards and mouse systems, (2) it makes use

of communication techniques found natural in humans, making the interface easy to

use,(3) it allows interfaces to be built for a wider range of users and tasks, (4) it creates

interfaces that are user-centered and not device centered, and (5) it has design emphasis

on being a transparent and unobtrusive interface. (Turk, 1998).

What is interesting in this line of research is the development of natural and intuit-

ive interface methods that make use of body language. A subset of PUI is Vision Based

Interfaces (VBI) which focuses on the visual awareness of computers to the people using

them. Here computer vision algorithms are used to locate and identify individuals, track

human body motions, model the head, and face, track facial features, interpret human

motion and actions. (Porta, 2002) A certain class of this research falls under bare hand hu-

man-computer interaction which this study is about. Bare hand human interaction uses as

a basis of input, the actions and gestures of the human hands alone without the use of at-

tached devices.

1.2 STATEMENT OF THE PROBLEM

Information-rich interactive viewing modules are usually implemented as

computer based kiosks. However placing computer peripherals such as touch-screens and

mouse and keyboard controlled computers in a public area would require significant

space and have maintenance concerns on the physical hardware being used by the

common public. Using a projected display and a camera based input device, would
eliminate the hardware problems associated with the space and maintenance. It also

attracts people since projected displays are new and novel.

1.3 OBJECTIVES

1.3.1 General Objectives

The general objective of the thesis is to create an interactive projected display

system using a projector and a camera. The projector would display the interactive

content and the user would use his hand to select objects in the projected display.

Computer vision is used to detect and track the hand and generate the proper response.

1.3.2 Specific Objectives

1.3.2.1 To use a DLP or LCD projector for the display

1.3.2.2 To use a PC camera as the basis of user input

1.3.2.3 To use a PC to implement algorithms to detect hand action as seen from the

camera.

1.3.2.4 To use a PC to host the information-rich content

1.3.2.5 To create an interactive DLSU campus map as a demo application

1.4 SCOPE AND DELIMITATION

1.4.1 Scope of the Study

1.4.1.1 The proponents will create a real time interactive projected display using

a projector and camera.

 1.4.1.2 The proponents will use development tools for image/video processing

and computer vision to program the PC. Algorithms for hand detection

and tracking will be implemented using these tools.

1.4.1.3. A demo application of the system will be implemented as an interactive

Campus Map of the School.

1.4.1.4 Only the posture of a pointing hand will be recognized as an input. Other

visual cues to the camera will not be recognized.

1.4.2 Delimitations of the study

1.4.2.1 The display will be projected in a clean white wall.

1.4.2.2 The projector and camera set-up will be fixed in such a way that

blocking the projector is not a problem.

1.4.2.3 Trapezoidal distortion which results from projecting from an angle will

be manually compensated if present.

1.4.2.4 Lighting conditions will be controlled. The set-up area must not have

too much light as to overpower the projection.

1.4.2.5 The system would be designed to handle only a single user. In the

presence of multiple users, the system would respond to the first user

triggering an event.

1.5 SIGNIFICANCE OF THE STUDY

The study applies a new way of presenting information using projected displays

and allows the user to interact with it. A projected display conserves space as the system

is ceiling mounted and there is no hardware that directly involves the user. Using only the
hands of the user as an input, the system is intuitive and natural-- key criteria for effective

interfaces. It presents an alternative to computer based modules where space can be a

problem.

Currently there is a high cost of acquiring and maintaining a projector. But it is

still viable when maintaining an information center is deemed to be important. The

system can be comparable to large screen displays that are used in malls and such. Since

the system is also a novel way of presenting information. It can be used to make

interactive advertisements that are very attracting to consumers. The display can

transform from an inviting advertisement into detailed product information. With this

said, the cost of the operation of the projector can possibly be justified with the revenue

generated from effective advertising.

The study is an endeavor towards the development of natural interfaces. The use

of a projector and camera provides a means of producing an augmented reality that is

natural-- requiring no special goggles or gloves that the user has to wear. In public spaces

where information is very valuable, a system that can provide an added dimension to

reality is very advantageous and the use of nothing but the hands means the user can

instantly tap the content of the projected interface. Computer vision provides the

implementation of a perceptual user interface and the projection provide the means of

creating an augmented reality. Further developments in these areas means the presence of

computers can be present in everyday life without being perceived as such. With PUI
there is no need for physical interface hardware, only the use of natural interaction skills

present in every human is needed.

1.6 DESCRIPTION OF THE PROJECT

The system is comprised of 3 main components; (1) the PC which houses the

information and control content, (2) the projector which displays the information, and (3)

the PC camera which is the input of the system. Development of the study would be

heavily invested in the software programming of the PC. The functions of the PC would

be the following: detection of the position and action of the hands of the user relative to

the screen, generating a response from a specific action, hosting the information rich

content. Techniques of image/video processing and machine vision will be used to

facilitate the first two functions of the PC.

As a demo application an interactive map of the school is used. The projector will

project the campus directory of De La Salle University Manila. The camera will capture

the images needed and will upload to the computer. The user will then pick on which

building he/she would like to explore using his/her hand as the pointing tool. Once the

user has chosen a building, a menu will appear that will give information about the

building. Information includes brief history, floor plans, facilities, faculties, etc. Once the

user is finished exploring the building, he/she can touch the back button to select another

building in the campus. The cycle will just continue until the user is satisfied.

1.7 METHODOLOGY
 Development of the study would be heavily invested in the software programming

of the PC. We the researchers must spend time in acquiring skills in programming for the

implementation of the research. Research about video capture and processing is greatly

needed for the operation of the system. Quick familiarization and efficiency with the

libraries and tools for computer vision is necessary for timely progress in the study.

The proponents of the research must first obtain the hardware that is needed for

the achievement of the study, materials such as the camera that will capture the input, the

projector which will give the output display (projected display) and the PC which the

system will be based. The appropriate specifications of the camera and the projector will

be carefully looked at to get the precise requirements. After which the system has its

working prototype, testing and making the necessary adjustments will be needed upon

detecting and fixing problems in the system.

Seeking advice from different people may be necessary for speedy progress of the

study. Advice in programming will be very helpful, since the implementation is PC based.

Additionally advice from the panel, adviser, and other people about the interface will be

helpful in removing biases the proponents may have in the system.

1.8 GANTT CHART

1.9 ESTIMATED BUDGET OF THE PROJECT

Projector……………………………….…..P 50,000

PC Camera……………………………..…. P 1500-2000

Open Source SDK…………………………Free

Development / Prototype Computer……….Available

Miscellaneous…………………………….. P5000

Estimated budget………………………….P 57000

REFERENCES

Porta, M. (2002) Vision-based user interfaces:methods and applications. International

Journal of Human Computer Studies. Elsevier Science

Turk, M. (1998). Moving from GUIs to PUIs. Symposium on Intelligent Information

Media. Microsoft Research Technical Report MSR-TR-98-69

Hewett, et. al. (1996) Chapter 2: Human Computer Interaction. ACM SIGCHI Curricila
for Human Computer Interaction. Available:
http://sigchi.org/cdg/cdg2.html#2_3 retrieved June 2, 2006.

Myers B., et. al. (1996) Strategic Directions in Human-Computer Interaction. ACM
Computing Surveys Vol.28 No.4