Organic

User Interfaces
Seminar on Post-Desktop
User Interfaces

Seminar paper at the

Media Computing Group
Prof. Dr. Jan Borchers
Computer Science Department
RWTH Aachen University

Julian Krenge
Vina Wibowo

Advisor:
Max Möllers

Semester:
Winter Semester 2008

Submission date:
Jan 29th, 2009
 iii

Contents

Abstract ix

Überblick xi

1 Introduction 1

2 Related Work 3

2.1 Organic Computing . . . . . . . . . . . . . . . . . . . . . . 3

2.2 User Interfaces We Know So Far . . . . . . . . . . . . . . 4

3 Organic User Interfaces Defined 7

3.1 Properties of Organic . . . . . . . . . . . . . . . . . . . . . 7

3.2 Design Principles . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Tangible UI Vs. Organic UI . . . . . . . . . . . . . . . . . 9

4 On the Way Towards Organic User Interfaces 11

4.1 Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.1.1 Interactive Surfaces . . . . . . . . . . . . . . . . . . 12

iv Contents

4.1.2 Deformation Tracking . . . . . . . . . . . . . . . . 13

4.2 Flexible Displays . . . . . . . . . . . . . . . . . . . . . . . 14

4.2.1 Digital Paper . . . . . . . . . . . . . . . . . . . . . 14

4.2.2 Bendable Screens . . . . . . . . . . . . . . . . . . . 15

4.3 Shape Actuation . . . . . . . . . . . . . . . . . . . . . . . . 16

4.3.1 Physical 3D Displays . . . . . . . . . . . . . . . . . 16

4.3.2 Ferrofluid Displays . . . . . . . . . . . . . . . . . . 17

4.3.3 Volumetric Displays . . . . . . . . . . . . . . . . . 18

4.4 Combination of Technologies . . . . . . . . . . . . . . . . 18

5 An Alternative Approach to Organic User Interfaces 21

5.1 Data Presentation . . . . . . . . . . . . . . . . . . . . . . . 21

5.2 Data Manipulation . . . . . . . . . . . . . . . . . . . . . . 22

6 Evaluation 23

6.1 OUI in Everyday Life . . . . . . . . . . . . . . . . . . . . . 23

6.2 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

7 Summary and Future Work 27

7.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Bibliography 31

Index 35
 v

List of Figures

4.1 SmartSkin . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2 TWEND . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.3 E-Ink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.4 Flexible OLED . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.5 Lumen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.6 SnOil, a ferrofluid display . . . . . . . . . . . . . . . . . . 18

4.7 Gummi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.1 Organic UIs interaction method . . . . . . . . . . . . . . . 24

7.1 The Nokia Morph Concept . . . . . . . . . . . . . . . . . . 29

 vii

List of Tables

3.1 Differences between Tangible UIs and Organic UIs . . . . 9

 ix

Abstract

Nowadays, user interfaces are rigid and do not utilise human’s manipulation skill.
They force users to learn certain methods to interact with them. However, in order
to improve the richness of interfaces, they should conform to the way humans in-
teract with their environment. This leads to the concept of Organic User Interfaces
(UIs) in which interfaces are designed to imitate this interaction. This is achieved
by developing interfaces so that they comply with the following principles: in-
put equals output, form equals function and form follows flow. Currently, there is no
interface, which follows all three principles above. Nonetheless, there are several
technologies leading to the realisation of Organic UIs.
x Abstract
 xi

Überblick

Heutige User Interfaces sind starr und schöpfen die Möglichkeiten menschlicher
Interaktion nicht aus. Stattdessen verlangen sie dem Benutzer ab, sich Metho-
den zur Kommunikation anzueignen. Um aber die Qualität des Interfaces zu
verbessern, sollten sie sich der menschlichen Art mit der Umwelt zu interagieren
unterwerfen. Dies führt zu dem Konzept der Organisches User Interfaces (Organis-
che UIs), die diese Interaktion imitieren. Erreicht wird dies durch die Entwicklung
von Interfaces, die folgenden Prinzipien entsprechen: Eingabe gleicht der Ausgabe,
Form gleicht der Funktion und Form folgt Funktion. Im Moment ist noch kein Interface
verfügbar, was allen drei Prinzipien genügt. Nichtsdestotrotz sind bereits einige
Technologien erhältlich, die die Realisierung von Organischen UIs ermöglichen.
 1

Chapter 1

Introduction

“Because you don’t have to be green to be green.”

—MTV Switch, The Green Song

Being organic is a trend that becomes more and more popular these
days. From organic food to organic clothing, from organic agriculture
to organic cleaner, everything is going green to save the earth environ-
ment and make humans healthier.

Inevitably, the term organic has also inspired the computing field. Nev-
ertheless, the purpose of organic differs from the purpose of organic
in general. In organic computing, the term organic means to imitate
the properties owned by organic beings such as adaptation. Apart
from organic computing, there has been a silent development of so-
called Organic User Interfaces (UIs). In which, interfaces are built to
resemble the shape of nature, which is flexible and deformable, and the
interaction between nature systems.

This seminar paper presents an overview of the development of

Organic UIs. Several work, which are related to the concept of Organic
UIs, will be presented in chapter 2. The next chapter is dedicated to the
definition of Organic UIs and their design principles. Chapter 4 gives
several technologies that have a great contribution in the realisation of
Organic UIs. In chapter 5 an alternative approach to Organic UIs is
given. Chapter 6 discusses the interaction styles and issues that may
arise by implementing Organic UIs in everyday life. Last but not least,
summary and possible future work will be given in the last chapter.
2 1 Introduction
 3

Chapter 2

Related Work

“Art is either plagiarism or revolution.”

—Paul Gauguin

Research on Organic UIs is still in its infancy state. Currently, not many
or even hardly any real UI is introduced as Organic UI. Nevertheless,
several work related to Organic UIs have been done. One work in-
cludes organic computing, which has the same concept of imitating
organic properties found in nature system. Others include several UIs,
of which characteristics have similarity with of Organic UIs.

2.1 Organic Computing

Organic Computing and Organic UIs share the same organic term. Al-
though both concepts are inspired by nature, they have different ap-
proaches in realising the nature or organic properties in a system.

Organic Computing is a system that dynamically adapts to changes in Dynamic

the environment and probably interact with each other [Seebach et al., Adaptation
2007]. The objective is to use principles observed in nature system
for building technical system with organic properties [Müller-Schloer,
2004]. These include self-configuration, self-adaptation, self-healing Self-x
and other self-x properties, as well as context-awareness. By combin- Properties
ing these properties, the system is given with more degree of freedom
to react upon component failures or environmental changes.
4 2 Related Work

2.2 User Interfaces We Know So Far

Until today, a number of UIs have been introduced to the public. Al-
though there are no revolutionary changes since good HCI design is
evolutionary rather than revolutionary [Canny, 2006], each believes
that its interaction is more intuitive than any of its predecessors. Be-
low are some of UIs that can be seen as the building blocks of organic
UIs.

WIMP Graphical UIs, especially the WIMP Interfaces, are probably the most
Interfaces well-knowned interface ever. Since 1970s when WIMP Interfaces
were introduced, all applications running on PCs are built to support
them [Canny, 2006]. A WIMP Interface consists of windows, icons,
menus, and is equipped with a pointing device, in this case a mouse.
Many argue that this interface does not reflect daily objects interaction.
Rekimoto in his article [Rekimoto, 2008] mentioned that by using a
mouse users can only interact with ”one point” and either ”pressed”
or ”hover”. Meanwhile, in reality people touch objects on ”multiple
points” and even put a special ”pressure” to them, like handshaking
for example.

The popularity of Graphical UIs has made its way to smaller mobile
devices such as PDAs. However, PDAs’ small screen makes it difficult
Adaptive for the users to browse through menus. Adaptive Interfaces have been
Interfaces introduced to address this issue. The idea is that the system adapts
the interface according to the users’ needs, such as displaying only the
features used most at the front page rather than all features. Although
this proved to be viable in small displays, in large display these ”un-
predictable menus” may lead to users’ confusion [Findlater and Mc-
Grenere, 2008].

Other interfaces similar with Adaptive Interfaces called Ambient

Ambient Interfaces adapt their interface not based on their users but on their
Interfaces users’ physical environment such as light, sound, or movement [Gross,
HCII, 2003]. They can be used for awareness information environment
that aims to improve awareness among geographically separated team
members. One example is by displaying pop-up window giving in-
formation regarding what the other co-workers are currently working
on.

Although Adaptive and Ambient Interfaces are improvements of clas-

sical Graphical UIs, the main interface is still in WIMP-style. Many
Post- researchers moved towards post-WIMP Interfaces to enrich user expe-
WIMP
2.2 User Interfaces We Know So Far 5

rience in interacting with interfaces by introducing more natural way

of interaction.

Touching is the first natural thing people do when encountering new

objects. By utilising this human instinct, Tactile and Haptic Interfaces Tactile-
were developed [Motamedi, 2007]. The idea of these interfaces is that Haptic
the users are actually touching the interfaces when they interact with Interfaces
them and getting haptic feedback, such as vibration. The later feature
adds a value to Tactile-Haptic Interfaces especially to visually impaired
users [Kahol and Panchanathan, 2006].

The ability to grasp and manipulate physical objects inspired the devel- Tangible
opment of Tangible UIs. Hiroshi Ishii, as one of the pioneers of Tangible User
UIs, defined Tangible UIs as interfaces, which give physical form to its Interfaces
digital counterpart [Ishii, 2008b]. Users manipulate the digital infor-
mation by directly manipulating the physical object that represents the
digital information and learn what interactions are possible from the
physical affordances of the object itself. Both properties increase the
directness and intuitiveness of interactions.
6 2 Related Work
 7

Chapter 3

Organic User Interfaces

Defined

“To make something look real and alive, nothing can be

symmetrical because nothing in real life is symmetrical. You
have to make it look organic. ”
—John Kricfalusi

Take a look at today’s PC hardware, for example an LCD. It is built to be

planar and rigid in order to protect the electronics inside it. Compare
this with paper. It can be folded, wrapped around, torn, even recycled.
Such thing cannot be imagined to be done on the current LCD [Holman
and Vertegaal, 2008].

3.1 Properties of Organic

Compared to Organic Computing, in which self-x properties of nature

system becomes the fundamental key in developing a technical sys-
tem, Organic UIs are inspired by the shapes of the nature system which Nature
are transformable, flexible, naturally adaptable, resilient and reliable Shape
[Vertegaal and Poupyrev, 2008]. Just like a leave, which bends instead
of breaks to accommodate the reception of sunlight. It also grows and
adjusts its shape to flexibly adapt with the environment [Holman and
Vertegaal, 2008].
8 3 Organic User Interfaces Defined

Another emphasis is on the analogue, continuous and transitional

Natural nature of physical and human interaction [Rekimoto, 2008] not on the
Interaction physical objects or metaphors [Schwesig, 2008]. A successful Organic
UI makes the users forgetting that they are operating machines to ma-
nipulate virtual data.

A mouse, which is the most popular input device, is considered as the

most inorganic interface. It is a tool to point and manipulate a certain
(x, y) location of an object which is located on the display, a different
device. It contradicts the true nature of human interaction in which a
tool is rarely needed and the fact that people manipulate the shape of
the object directly at multiple points at the same time.

3.2 Design Principles

Holman and Vertegaal [Holman and Vertegaal, 2008]defined three de-

sign principles, which can be used as guidelines, to develop Organic
UIs. These include input equals output, form equals function and form
follows flow.

A person draws directly on the paper and views his/her drawing di-
rectly from the paper. This visualises a true physical interaction where
Input input and output interaction happen at the same location, in this case
Equals the paper. To imitate this behaviour, Organic UIs should have a display
Output which can sense [Holman and Vertegaal, 2008]; or to put it differently
the input device acts also as the output device.

Form The form of an object gives hints on what activities people can do with
Equals it [Holman and Vertegaal, 2008]. The flexible form of a piece of paper
Function suggests that it can be folded, bent, crumpled or even torn up. Yet, it
can still serve its original purposes: to be read or written on. Organic
UIs should use their form as a physical representation of activity.

Form A spring changes its shape to follow the movement of the person’s
Follows hand, which extends it. As the person lets the spring go, it will go
Flow back to its original shape. This implies that Organic UI should be able
to either alter its shape to follow the flow of user interaction or to adapt its
shape automatically for better context of use [Holman and Vertegaal, 2008].
3.3 Definition 9

3.3 Definition

Concluding from the three design principles above, a definition of

Organic UI can be formalised. An Organic UI is a UI that more closely re-
sembles natural human-physical and human-human interaction by using non-
planar displays, as input and output, that may actively or passively change
shape following analogue physical input to adapt to user’s needs. [Rekimoto,
2008] [Holman and Vertegaal, 2008] [Vertegaal and Poupyrev, 2008]

Although they are said to be organic, Organic UIs do not need to be

made out of organic materials. The emphasis is to promote flexibility,
enhance users satisfaction and allow users to be creative rather than
productive. [Holman and Vertegaal, 2008].

3.4 Tangible UI Vs. Organic UI

By looking at the definition of Organic UIs defined above, similari-

ties between Organic UIs and their predecessor, Tangible UI cannot
be avoided. Organic UIs are Tangible UIs but Tangible UIs are not always Physical
organic. Tangible UIs are said to be the gate to Organic UIs. They in- Manipulation
spire Organic UIs to use physical interaction to manipulate digital data.
However, they lack of shape adaptation since the tool used to interact No
cannot change shape in real time. Adding shape actuating behaviour Shape
makes Tangible UIs may be considered as Organic UIs [Ishii, 2008a]. Ta- Adaptation
ble 3.1 describes further differences between Tangible UI and Organic
UI [Rekimoto, 2008].

Characteristics Tangible UIs Organic UIs

Interaction Metaphor using tool direct contact
Orientation manipulation-oriented communication-oriented
Representation tool = digital info shape = activity
Coverage application specific generic

Table 3.1: Differences between Tangible UIs and Organic UIs

10 3 Organic User Interfaces Defined
 11

Chapter 4

On the Way Towards Organic

User Interfaces

“Technology: No Place for Wimps!”

—Scott Adams, Dilbert

As shown, Organic UIs are meant to be more intuitive than other UIs.
To provide this, technologies enabling devices to feel organic are nec-
essary. Input as well as output technologies are essential to make the
interaction between human and computer feel natural. In this section,
some of these technologies are described.

4.1 Input

First of all, the users should be provided with an intuitive way to ma-
nipulate data on the computer. Since nowadays most common input
devices, such as mouse and keyboard, are very unnatural, a touch-
screen can be said as an advancement of the nativeness of a user
interface, even though a touch-screen is bound to the rigid and planar
shape of a computer. Since rigidity does not occur in nature, Organic
UIs should be without edges and corners as well.
12 4 On the Way Towards Organic User Interfaces

4.1.1 Interactive Surfaces

Three kinds of evolvement could be applied to a standard touch-pad.

By these, a standard touch panel could be improved to an interactive
surface, which enables intuitive interaction.

Flexible Firstly, the bondage of rigidity has to be broken. Not only should
Devices touch pads be capable to fit any form or shape but also to be deformed
while being used. A first approach to this improvement is SmartSkin
that could be produced to be flexible. Additionally, it could also be
transparent to be applied on top of a display [Rekimoto, 2002].

Multiple Secondly, multiple inputs should be allowed, so that more than one
Inputs hand can be used and even multiple users are able to interact at the
same time. At the moment, multi-touch panels, which use different
technologies, are available. The best-known examples are Apple prod-
ucts such as the iPhone or iPod touch.

Gesture Thirdly, the sensing capabilities should be improved. Hovering as well

Detection as other gestures should be recognised. This would allow a more nat-
ural way of input as one can express complex instructions in uncom-
plicated ways. SmartSkin , which uses capacitive sensing, [Rekimoto,
2002] provide this capability. A more recent approach is ShapeTouch
, in which the sensing is more precise [Cao et al., 2008]. By using op-
tical recognition, ThinSight tracks hand gestures in short distance and
through an overlying display [Hodges et al., 2007]. Toshiba recently
released a notebook with a dedicated processor to recognise gestures
using the built-in webcam [Toshiba DPD, 2008].

Figure 4.1: SmartSkin, an interactive surface - by Sony CSL

4.1 Input 13

Although these technologies are promising, there are still several is- Issues
sues to be addressed. Up to now there is no way to provide direct
haptic feedback. Also, the coupling of touch panel and display has to
be improved to enable thin devices. Using projectors decouples the
input and output. Additionally, the use of gestures is a newly discov-
ered field and effective ways of interaction have yet to be found. User
studies have to be performed to discover how users would like com-
municating their commands.

4.1.2 Deformation Tracking

Pure bearing of deformation, as in the section before, might not be Deformation

enough. For instance books allow input by deformation when search- as
ing for a specific page. It is likely to bend the pages in a way that Input
they quickly flip over enabling the user to scroll through the book
while getting a glance at every page. Yet, this is not the only exam-
ple. Deformation is a very common way of interaction with objects.
This leads to the conclusion that organic devices should be capable of
tracking its own actual shape.

TWEND, an input device, named by the terms ”twist” and ”bend”, State
provides this capability. Deformation regarding to the X- and Y-axis of the
can be recognised. In this approach, optical bending sensors are used. Art
Based on this technology, gestures can be defined: flipping one corner
over for going to the next page or simulating a dog-ear for bookmarks
[Herkenrath et al., 2008]. A more recent device providing similar fea-
tures is Bookisheet [ichiro Watanabe et al., 2008].

Figure 4.2: TWEND, a device to track twisting and bending - by the

Media Computing Group at RWTH Aachen
14 4 On the Way Towards Organic User Interfaces

Issues The problem when allowing users to interact via deformation is that
the whole device has to be deformable. Flexible displays are available
but the whole processing hardware has to be flexible as well. Useful
gestures have to be found similar to the multi-touch panels. User stud-
ies on the interaction based on gestures have to be conducted to clarify
how these features can be used efficiently.

4.2 Flexible Displays

Only creating devices, which are deformable, is not enough to create

a flexible touch-pad or even to track the deformation. Since the whole
device should be flexible, the display has to be bend- and twistable
as well. In addition, this development is accompanied by a higher
durability. There are two highly promising technologies.

4.2.1 Digital Paper

High Formerly, the most used medium to provide information was paper,
Contrast which is very different to computer screens. On one hand, it lacks of
and the ability to change its information easily. On the other hand, it is flex-
Durability ible and has a very high contrast. While computer screens are not read-
able any more when the sun shining on them, paper is immune to that
effect. Closing this gap is electrophoretic ink displays, which are often
associated with the brand E Ink. They combine advantages of both me-
dia. Also, they can be equipped with background lighting easily. In
addition, digital paper has less power consumption than common Liq-
uid Crystal Displays (LCDs) because due to their technique, their state
is stable and only changes trigger the use of energy [Comiskey et al.,
1998].

State Recently, E Ink was improved such that it is unbreakable. It withstands

of the extreme vibration as well as impacts of heavy objects. This broadens
Art the application fields of electrophoretic displays.

Issues Electrophoretic displays are already far developed and applicable in

productive use. There are several products on the market using this
display technology, usually developed as E-Book readers, such as the
Amazon Kindle. However, digital paper still suffers from two main
4.2 Flexible Displays 15

Figure 4.3: A bendable E-Ink display - by E-Ink

problems. Firstly, its frame rate is low. Videos cannot be shown using
this kind of display. This also hinders electrophoretic displays to be
used in portable devices such as phones or PDAs because fluid menu
navigation is not possible. Even so they convince their customers with
their very low energy consumption. The Motorola F3 was the first cell
phone using digital paper but had only basic features. Secondly, the
chroma resolution is very low. Coloured displays can be produced
by microcapsules containing red, green and blue droplets rather than
white ones. Three capsules could be combined to one pixel.

4.2.2 Bendable Screens

While digital paper enhances the way the information can be altered Full-
easily, Organic LED (OLED) displays improve nowadays screens. By colour
using OLED technology, full-coloured computer displays can be very and
thin and robust and therefore flexible. Compared to an LCD, the en- Very
ergy consumption is lower and the size of the borders is smaller. This Thin
enables OLED displays to be applied in the most convenient way.

Sony has already presented a flexible full-coloured display based on the State
OLED technology. Recently, Samsung SDI presented an OLED display of the
with a thickness of only 0.05 mm, which actually flaps in the wind. Art
These thin displays allow light shining through them and are flexible.
16 4 On the Way Towards Organic User Interfaces

Figure 4.4: A flexible Organic LED display - by Pioneer

Issues Although OLED displays are far developed and already applicable in
productive use they still suffer from several issues. At present only rel-
atively small displays are available. Because of this, OLED displays are
only available in small devices such as mp3-players. In further devel-
opment, the screen size has to be enlarged. Another issue is the rela-
tively short lifetime of Organic LEDs. Although they are still very high
compared to LCD- or Plasma-displays, they still cannot compete with
LEDs.

4.3 Shape Actuation

Two-dimensional displays are not the only way to visualise informa-

tion. In several applications, the shape of an actual object represents
digital data. Recent computers display a three-dimensional object by
having several points of view at the object at the same time. This adds
additional information to the data. Additionally, direct manipulation
of an actual object would be more efficient and natural.

4.3.1 Physical 3D Displays

Actual The most efficient and convenient way of manipulating three-

Objects dimensional objects would be a physical representation of the object of
interest. This would grant direct haptic feedback and an instant view
on the result. This overlaps with the field of Tangible UIs.
4.3 Shape Actuation 17

One of technology for shape alternation is so called shape memory Simple

alloys. Using this is Lumen. It is a 16x16 pixel display where a third Approach
dimension is added by enabling the pixels to alter their physical height.
Therefore, every pixel provides not only information by its red, green
and blue colour value, but also by its height [Poupyrev et al., 2004,
2007].

Figure 4.5: Lumen, a simple three-dimensional display - by Sony CLS

At the moment the possibilities of shape alternation are limited. Lumen Issues
is a simple approach and cannot convey a lot of additional data via
the height of the pixels. More advanced approaches would need more
complex shape alternation.

4.3.2 Ferrofluid Displays

While physical 3D displays are still bound to the shape of their ele- Shapeable
ments, ferrofluid displays are an approach to alternate shape in another Liquid
way. Ferrofluid acts similar to iron, but is liquid. When a magnet ap-
proaches, it changes its shape.

An example for ferrofluid displays is SnOil. It is an implementation of State

the classic game Snake. Underlying electromagnets influence a basin of the
filled with ferrofluid [Poupyrev et al., 2007]. Another approach is Pro- Art
trude, Flow. It is not displaying information but focusing on aesthetics
[Kodama, 2008].

Although ferrofluid displays are more versatile than nowadays physi- Issues
cal 3D displays, they are not capable of representing any shape. They
are bound to the possibilities of electromagnets. In addition, they can-
not be touched.
18 4 On the Way Towards Organic User Interfaces

Figure 4.6: SnOil, a ferrofluid display - by Martin Frey at UDK Berlin

4.3.3 Volumetric Displays

Holograms Volumetric displays are not part of the field of shape actuation but also
capable of displaying information three-dimensionally. A precise hand
tracking could enable users to modify the projected objects directly us-
ing their hands. Since volumetric displays are encapsulated in glass,
this concept is yet unrealisable [Grossman and Balakrishnan, 2006].

4.4 Combination of Technologies

Building When building an Organic UI it is not necessary to include all organic

an technologies mentioned before. As already explained an Organic UI
Organic has to be adjusted to its use very carefully. These technologies are to
UI utilize that fit to the theme of the device that is about to be built.

Figure 4.7: Gummi, an organic digital map

4.4 Combination of Technologies 19

Gummi is the concept of a digital map as an Organic UI. The whole First
device is bendable and bending is used as input. In addition, a bend- Organic
able touch-pad is positioned on the backside. Gummi is the first device UI
combining a lot of organic aspects. It is flexible and also tracks the
deformation to provide intuitive input. The user is not that aware of
it as a computer but recognises its capability to display a street map
[Schwesig et al., 2003, 2004]. Another example for the concept of a
Organic UI is Morph, which will be explained in 7.2—“Future Work”.

Currently, Gummi is just a concept, the actual prototype does not con- Issues
sist of all aimed features. The prototype already allows evaluating the
effectiveness of the interaction since all features are emulated. Gummi
is not yet bendable but recognises applied pressure as if one was bend-
ing it.
20 4 On the Way Towards Organic User Interfaces
 21

Chapter 5

An Alternative Approach to
Organic User Interfaces

“A different language is a different vision of life.”

—Federico Fellini

Up to now, the focus lays on devices that enable users to interact with
them in a natural way. The most important aspect on this was the flexi-
bility of the devices. However, there is a different approach in viewing
Organic UIs, that not only the device itself can be organic but also can
be the software. There is a possibility for software to provide informa-
tion in a natural way and still running on an ordinary computer.

5.1 Data Presentation

Data could be provided and shown in a natural way. Nowadays, Organic

presentation tools are based on the concept of a linear sequence of Presentation
slides. Obviously the human mind is not organised in a straight pro- Tool
ceeding. Dealing with this problem is Fly. It tries to map the thoughts
of the presenter to the presentation. The information to be described is
organised in any way the author wants it to be [Holman et al., 2006].
22 5 An Alternative Approach to Organic User Interfaces

Another section of the field of organic data presentation is the data

structuring. Apple recently developed iPhoto software to organise pho-
tos in a more natural way than any other software before. Photos are
grouped into events that can be tagged by the users. By this, they do not
have to remember the exact date, but only the people they met there.
This is a good representation of the human memory.

5.2 Data Manipulation

Organic Digital data could be manipulated in a more natural way. While

Video the common way to navigate within a video is the timeline, the
Navigation manipulation of objects in the video to browse through time would be
more convenient. For example, one can jump to a previous part of the
movie by dragging a person backwards in space and by this also in
time. When a digital object is moved to the position where it was in a
previous part, all other objects are reset to their previous positions as
well. This is the concept of DRAGON [Karrer et al., 2008].

Issues Although DRAGON is a very organic way of video navigation, it is

not practical. For instance, when the scene changes, the objects on
the frame change as well. Therefore, DRAGON could not be used for
navigation in movies.
 23

Chapter 6

Evaluation

“I do not fear computers. I fear the lack of them.”

—Isaac Asimov

Several technologies in the field of Organic UIs have been described.

However, it is questionable whether these approaches might be appli-
cable to everyday life.

6.1 OUI in Everyday Life

A few years ago when PCs made its way to offices to store necessary
business documents, employers have dreamed to cut business cost by
realising a paperless office. Nevertheless until today, many employ- Paperless
ees still prefer to print out the document, hold the paper in hands and Office
mark things up [Blevis, 2008]. With the emergence of thin and flexible
displays, they are becoming more and more paper-like. In the near fu-
ture, it is likely that displays can be treated as real paper and paperless
office can finally be realised.

By applying other continuous parameters such as pressure, one can im- Continuous
prove the intuitiveness of the interface [Rekimoto, 2008]. Different pres- Parameters
sure applied to the interface should result in different possible action.
An example would be the bending interaction, which means zooming
action. A softer pressure to the interface means that the zooming action
is slower than when a harder pressure is applied.
24 6 Evaluation

Figure 6.1: Paper-like Organic UIs interaction method

The development of Organic UIs is still in its infancy. Therefore, not

many interaction methods could be thought up during this period.
Emotionally- However, regardless on which interaction method introduced later,
involved interaction with Organic UIs should make people emotionally involved
in the interaction and forget that they are operating computers [Schwe-
Everyday sig, 2008] and see them as ordinary everyday object [Holman and Verte-
Object gaal, 2008].

Alternative As for the alternative approach of organic UIs, the interaction method
Interaction might be different with the Organic UIs mentioned above. However,
the emphasis is still to establish a natural interaction between the users
and the interface. DRAGON, for example, allows users to navigate
through a video by selecting the object inside the video instead of the
video timeline [Karrer et al., 2008]. While Fly presents a new way to or-
ganise presentation slide as mind map rather than in linear order [Hol-
man et al., 2006].

6.2 Issues

Nothing is perfect. The concept of Organic UIs gives a modern

approach on how computers can integrate seamlessly in everyday life
so that people forget that they are actually computers. Nevertheless,
issues arise inevitably.

Flexibility in Organic UIs arises issues concerning the hardware. Not

Flexible only the displays, which should be flexible but also the processor and
Hardware other electronics. Current technologies mentioned before revealed that
there is no any approach, which tries to combine organic in- and out-
put technologies so that they can be seen as one device. While TWEND
does not possess a display, flexible displays such as E-Ink or Organic
6.2 Issues 25

LED supplies no input method. Gummi tried to combine these two

technologies. However, they are still located in different positions
(front and back).

Recently developed technologies in the field of shape actuation suffer

from another problem. Either they are bound to certain types of form
manipulation and therefore cannot display any information or they are Untouchable
not suitable for human interaction as they can not be touched. Interface

The interaction techniques of Organic UIs are not complete yet. Only
simple interaction techniques, such as bending for zooming action us- Incomplete
ing both hands, have been introduced. It is questionable whether later Interaction
the interaction can involve other modalities, such as eye gazing, blow- Techniques
ing and entire body. Thus, more interaction techniques are still yet to
be discovered. [Rekimoto, 2008]

One property that makes Organic UIs differ from Tangible UIs is that
Organic UIs are general rather than application-oriented. However, it
does not mean that one Organic UI fits all since consistencies across ac- Interaction
tivities and contexts might be difficult to realise [Holman and Vertegaal, Inconsistencies
2008]. In graphic application, bending might mean zooming; while in
video application bending might mean fast forwarding.

Last but not least, enhancing computers in a way that they are not
recognised as computers any more might lead to privacy problem. Peo- Privacy
ple might feel inconvenience by the thought of being observed by com-
puters all the time and not being able to distinguish between digitally
improved and ordinary objects in their everyday life.
26 6 Evaluation
 27

Chapter 7

Summary and Future Work

“Our imagination is the only limit to what we can hope to

have in the future.”
—Charles F. Kettering

Coming to the end of this seminar paper, several questions are still re-
mained. How is the future of Organic UIs? Will they be as successful as
Graphical UI? What improvements can be done to guarantee the future
of Organic UIs?

7.1 Summary

Organic UI is a UI which more closely resembles natural human- Definition

physical and human-human interaction by using non-planar displays,
as input and output, that may actively or passively change shape
following analogue physical input to adapt to user’s need 3.3—
“Definition”. The definition states that Organic UIs should follow three
design principles, which are input equals output, form equals function and Design
form follows function 3.2—“Design Principles”. Principles

Based on these definition and design principles, currently there is no No

real UI which follows these principles. Nevertheless, several technolo- Organic
gies of input, flexible displays and shape actuators play an important UI
role in the realisation of Organic UIs.
28 7 Summary and Future Work

Alternative Another approach in viewing Organic UIs from other point of view
Approach has been introduced. Rather than focussing on the shape of the device,
the alternative approach focusses more on the interaction between the
users and the application interface.

Organic UIs aims to create interfaces, which are seamlessly integrated

with everyday life objects. To resemble everyday life objects, Organic
UIs need to be flexible. This property is fulfilled by the emergence of
Issues flexible displays and flexible input devices today. More research and
experiments still need to be done in order to couple these two tech-
nologies. The seamlessly integration of the interface might raise pri-
vacy problem. People are afraid of being watched since they cannot
differentiate between which objects are computers and which are not.

7.2 Future Work

Organic UIs is a fresh concept in interaction design. Since its official

introduction published by ACM in April 2008, there has been no pub-
lication labelled “Organic UIs“ which leads to further development of
Organic UIs. Nevertheless, by looking at the past and current technolo-
gies presented before, a possible future work can be concluded.

Input + The in- and output technologies which lead to the realisation of organic
Output UIs have been developed. However, there has not been any attempt to
put these two technologies together such that they can be seen as one
device. One possibility is by combining TWEND which is an input
device and Flexible OLED which an output device. Both are flexible
and possible to be put into one device.

Nokia Introduced earlier this year was a concept by Nokia called Morph,
Morph [Nokia, 2008] which uses Nanotechnology as the basis for future mobile
Nanotech phones. Nanotechnology is a development and research on materials
of which size ranges from 1 -100 nanometer (1 nm = 10−6 mm = 10−9 m)
[Paull and Lyons, 2008]. By using Nanotechnology, the Nokia Morph
demonstrates the possibility to have a mobile phone, which is flexible,
stretchable and transparent. It is charged using solar power and has
integrated sensors, which can sense the environment around the users.
7.2 Future Work 29

Figure 7.1: The Nokia Morph Concept

Further user studies should also be done to discover new interaction New
techniques for using organic UIs and to address the privacy issues, Interaction
which might come out when a computer does not feel like a computer Techniques
anymore.
30 7 Summary and Future Work
 31

Bibliography

Eli Blevis. Sustainability implications of organic user interface tech-

nologies: an inky problem. Commun. ACM, 51(6):56–57, 2008.

John Canny. The future of human-computer interaction. Queue, 4(6):

24–32, 2006.

Xiang Cao, Andrew D. Wilson, Ravin Balakrishnan, Ken Hinckley, and

Scott E. Hudson. Shapetouch: Leveraging contact shape on interac-
tive surfaces. Horizontal Interactive Human Computer Systems, 2008.
TABLETOP 2008. 3rd IEEE International Workshop on, pages 129–136,
Oct. 2008.

Barrett Comiskey, J. D. Albert, Hidekazu Yoshizawa, and Joseph Jacob-

son. An electrophoretic ink for all-printed reflective electronic dis-
plays. Nature, 394:253–255, May 1998.

Leah Findlater and Joanna McGrenere. Impact of screen size on perfor-

mance, awareness, and user satisfaction with adaptive graphical user
interfaces. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHI
conference on Human factors in computing systems, pages 1247–1256.
ACM, 2008.

Tovi Grossman and Ravin Balakrishnan. The design and evaluation of

selection techniques for 3d volumetric displays. In UIST ’06: Proceed-
ings of the 19th annual ACM symposium on User interface software and
technology, pages 3–12. ACM, 2006.

Gero Herkenrath, Thorsten Karrer, and Jan Borchers. Twend: Twisting

and bending as new interaction gesture in mobile devices. In Ex-
tended Abstracts of CHI 2008 Conference on Human Factors in Comput-
ing Systems, Florence, Italy, April 2008. ACM Press. Student Research
Competition 2nd Place.

Steve Hodges, Shahram Izadi, Alex Butler, Alban Rrustemi, and Bill
Buxton. Thinsight: versatile multi-touch sensing for thin form-factor
32 Bibliography

displays. In UIST ’07: Proceedings of the 20th annual ACM symposium

on User interface software and technology, pages 259–268. ACM, 2007.

David Holman and Roel Vertegaal. Organic user interfaces: designing

computers in any way, shape, or form. Commun. ACM, 51(6):48–55,
2008.

David Holman, Predrag Stojadinović, Thorsten Karrer, and Jan

Borchers. Fly: an organic presentation tool. In CHI ’06: CHI ’06 ex-
tended abstracts on Human factors in computing systems, pages 863–868.
ACM, 2006.

Jun ichiro Watanabe, Arito Mochizuki, and Youichi Horry. Bookisheet:

bendable device for browsing content using the metaphor of leafing
through the pages. In UbiComp ’08: Proceedings of the 10th international
conference on Ubiquitous computing, pages 360–369. ACM, 2008.

Hiroshi Ishii. The tangible user interface and its evolution. Commun.
ACM, 51(6):32–36, 2008a.

Hiroshi Ishii. Tangible bits: beyond pixels. In TEI ’08: Proceedings of the
2nd international conference on Tangible and embedded interaction, pages
xv–xxv. ACM, 2008b.

Kanav Kahol and Sethuraman Panchanathan. Distal object perception

through haptic user interfaces for individuals who are blind. SIGAC-
CESS Access. Comput., (84):30–33, 2006.

Thorsten Karrer, Malte Weiss, Eric Lee, and Jan Borchers. Dragon:
a direct manipulation interface for frame-accurate in-scene video
navigation. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHI
conference on Human factors in computing systems, pages 247–250.
ACM, 2008.

Sachiko Kodama. Dynamic ferrofluid sculpture: organic shape-

changing art forms. Commun. ACM, 51(6):79–81, 2008.

Nima Motamedi. The aesthetics of touch in interaction design. In DPPI

’07: Proceedings of the 2007 conference on Designing pleasurable products
and interfaces, pages 455–460. ACM, 2007.

Christian Müller-Schloer. Organic computing: on the feasibility of

controlled emergence. In CODES+ISSS ’04: Proceedings of the 2nd
IEEE/ACM/IFIP international conference on Hardware/software codesign
and system synthesis, pages 2–5. ACM, 2004.

Nokia. The morph concept, December 2008. URL http://www.

nokia.com/A4852062.
Bibliography 33

John Paull and Kristen Lyons. Nanotechnology: The next challenge for
organics. Journal of Organic Systems, 3(1):3–22, 2008.

Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto,

and Yasufumi Yamaji. Lumen: interactive visual and shape dis-
play for calm computing. In SIGGRAPH ’04: ACM SIGGRAPH 2004
Emerging technologies, page 17. ACM, 2004.

Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. Actuation and

tangible user interfaces: the vaucanson duck, robots, and shape dis-
plays. In TEI ’07: Proceedings of the 1st international conference on Tan-
gible and embedded interaction, pages 205–212. ACM, 2007.

Jun Rekimoto. Organic interaction technologies: from stone to skin.

Commun. ACM, 51(6):38–44, 2008.

Jun Rekimoto. Smartskin: an infrastructure for freehand manipulation

on interactive surfaces. In CHI ’02: Proceedings of the SIGCHI con-
ference on Human factors in computing systems, pages 113–120. ACM,
2002.

Carsten Schwesig. What makes an interface feel organic? Commun.

ACM, 51(6):67–69, 2008.

Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: user

interface for deformable computers. In CHI ’03: CHI ’03 extended
abstracts on Human factors in computing systems, pages 954–955. ACM,
2003.

Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: A bendable

computer. In CHI ’04, pages 24–29. ACM, 2004.

Hella Seebach, Frank Ortmeier, and Wolfgang Reif. Design and con-
struction of organic computing systems. In IEEE Congress on Evolu-
tionary Computation 2007, pages 4215–4221. IEEE, 2007.

Toshiba DPD. Toshiba qosmio world’s first laptop with cell proces-
sor technology, July 2008. URL http://explore.toshiba.com/
pressrelease/423413?fromPage=editorials.

Roel Vertegaal and Ivan Poupyrev. Organic user interfaces. Commun.

ACM, 51(6):26–30, 2008.
Typeset January 28, 2009