Contents

1 Challenges in GUI Development 2

1.1 Using Design Patterns to Apply HCI Principles : : : : : : : : 2
1.1.1 Handling Restrictions on Screen Size : : : : : : : : : : 2
1.1.2 Pre-Design Research : : : : : : : : : : : : : : : : : : : 3
1.2 Creating Extensible and Maintainable UI Software : : : : : : : 4
A Survey of Design Pattern Usage in 1.3 Communicating Design Between Software Groups : : : : : : : 6

2 Solutions to the Challenges 8

Graphical User Interfaces 2.1 Using Design Patterns to Apply HCI Principles : : : : : : : : 8
2.1.1 Handling Restrictions on Screen Size : : : : : : : : : : 8
Wyatt Davenport Dan Fraser Kavi Maharajh 2.1.2 Pre-Design Research : : : : : : : : : : : : : : : : : : : 10
2.2 Creating Extensible and Maintainable UI Software 11
Rahim Moledina
: : : : : : :

2.3 Communicating Design Between Software Groups : : : : : : : 13

April 7, 2000 3 Analysis of Solutions 16

4 Questions for Further Study 19

1
1 Challenges in GUI Development
1.1 Using Design Patterns to Apply HCI Principles
1.1.1 Handling Restrictions on Screen Size
In the paper by Je Noble, GOF Patterns for GUI Pattern, three forces
that are common to all the patterns arepresented. These forces constitute
challenges in the area of GUI design patterns. One such challenge entails
the handling restrictions on the available screen size. The handling of screen
real estate poses a challenge for all of the patterns mentioned and has two
of screen size manifests it self in scene composition. Every Storyboard has
a set of scenes, each of which have three distinct parts, spatial composition,
temporal composition and interactivity description. Within the spatial de-
scription the challenge is brought forth. In spatial composition, the designer
handles the positioning of each multimedia element on the screen. Inherent
to this part are the challenges of screen real estate preservation and object
permanence.
Other problems arise when dealing with screen real-estate, the user must
often deal with hundreds or thousands of related objects on-screen at one
important elements that contribute to it. The rst is the need to preserve
available screen space. Some of the patterns are not able to successfully
conquer this challenge due to their inherent restrictions. Secondly, instances
of patterns should not be kept on the screen at all times as it would cause
inecient handling of screen resources. Thus certain patterns are restricted
in terms of their performance or capabilities.
Another challenge of screen size can be seen in the article \Storyboard"
Alexandrou and Popadopulous. The article introduces a pattern for creation
time. It is often unclear how to place these objects on the screen such that
they have strong visual relationships. There should be a good way to organize
these objects to ensure maximum productivity and ease of use.
Windows in a graphical user interface are often divided into zones of
information. Correctly sizing these zones and arranging them in the correct
locations is a dicult process.
1.1.2 Pre-Design Research
One other article focuses on a similar challenge; one that deals primarily
of a Storyboard. The Storyboard is a technique taken from the lm making
with predesign research. Up to this point most of the work on patterns for
industry and is enhanced with additional features from Multimedia systems
interfaces has focused on the screen design issues. PSA (Pattern Supported
such as interactivity. The Storyboard pattern therefore assists the designer in
Approach) suggests a wider scope for the use of patterns by looking at the
creation of a Storyboard for the application being developed. The challenge
overall user-oriented user interface design process. The primary challenges

2 3
at hand here are to focus on addressing patterns before design as much as Run-time congurability is also required. A graphical application such
the design phase itself. That is, proposing patterns for system denition, as computer aided design (CAD) requires the manipulation of many pre-
task analysis and conceptual design. By this method, the patterns oer a
way to capture and communicate knowledge from previous designs. The
previously mentioned patterns will act as tools to guide and support the
usability engineering tasks. This approach is not entirely new; many have
practiced usability testing like this before. However, the major concern here is
learning as much as possible about the current situation (task user analysis)
and predicting what is needed for a future system. In addition, collected
information before design is usually gained from interviews, questionnaires,
old systems research, etc. Using these sources means gathering quite a bit of
information. Analysis can then prove to require a lot of eort and tend to
dened objects. It should be possible to add or remove functionality relating
to these objects at run-time. This would allow the user to easily customize
the application, or for the vendor to easily provide the user with custom
upgrades.
Graphical applications such as CAD software also oer the user the ability
to apply certain algorithms to objects or groups of objects. It should be
possible to apply these algorithms to any object, at any time. It should also
be possible to add new algorithms at run-time. This requires careful design
in order to be eective and reliable.
A useful feature in graphical applications is the ability to group objects
be cumbersome. Reuse of information and design denitely occur for similar
projects, but this often adds havoc and could be aected by the individual's
ability and personal experiences.
1.2 Creating Extensible and Maintainable UI Software
Software requirements usually involve creating extensible, robust, maintain-
together and manipulate them as if they were a single object. Implementing
this aggregation in a way that is both maintainable and transparent to the
user is an important challenge.
Common implementation problems also include designing a system that
can be used by several dierent interfaces. For example, blind users may
wish to use an application through an audio interface.
able software. It is often desireable to have dynamically dened functionality Developing a reusable library for system construction and making this
in applications, for example, to include only the features specically needed library
exible and maintainable is dicult. The challenge of dynamically
or purchased by a user. altering library functionality while maintaining eciency is a concern.
With a black-box library, information about the implementation of the

4 5
functionality is hidden from the user. On the other hand, a white-box library, design problem. The article suggests usability elements such as Learnable,
the implementation is fully disclosed to the user. Both of these library types Memorable, and Ecient.
can potentially be used when addressing the concerns of library creation. The exact elements in this list are not as important as the idea that
they represent dimensions of usability that can dene the forces underlying a
1.3 Communicating Design Between Software Groups design problem. They will always need to be interpreted for any development
Another primary challenge that seems to be prevalent in GUI design is the project.
idea of correctly communicating the elements of patterns for usability, focus- In addition to these elements of usability, the article also suggests a mod-
ing mainly on the details of the design such as interface elements (widgets),
task domains, interaction styles and types of users. The article suggests that
there have been many partially successful approaches to capturing and com-
municating the essential elements of good designs. Since the recent promi-
nence of pattern use in object oriented software, the essentials of design
solutions have been much easier to understand, learn, and applied to simi-
lar cases. The important advancement needed to move from exemplars to
ication to the original denition of a pattern. The main elements are con-
served but vary in the need for more clarity from a User Interface perspective.
This adapted list consists of:
Name A name for the pattern; e.g. Window Place
Context A context for the design problem; e.g Design of a residential room
Forces Forces which require resolution; e.g. People want to sit and also be
templates to patterns seems to be the vocabulary that is specic to design in daylight.
for usability. This vocabulary should simple to learn and extensible to a
Problem A problem growing from the forces; e.g. If all seating is away from
broad range of applications. In other words, an addition to the vocabulary
the windows, then these forces are not resolved, and people will always
is essential to a more ecient advancement in pattern use.
be dissatised in one way or the other.
The primary challenges seem to be ways of correctly communicating the
elements of patterns for usability. To address these issues, a vocabulary Solution A known solution, proven in practice; e.g Build seating into the
exists that will represent the usability which denes the forces underlying a window { the traditional window seat.

6 7
 Another major set back in HCI (Human Computer Interaction) design
is the communication between interdisciplinary design teams. A pattern
based approach can lesson the set back involved in interdisciplinary design
team communication. In general, HCI deals with communication between
designers and users during development. It is dicult for user interface
experts to communicate their experience and methods to other design team
members, from software engineering to application domain experts. These
communication problems lead to a lack of understanding of the paradigms,
methods, and values of the other groups within the project development.
a part of the object which uses the algorithm. The user can then choose
an algorithm via. the main object and interact with the strategy object to
set the parameters of the algorithm. Since screen space is limited, only one
strategy object at a time is presented thus ensuring preservation of screen
space. Although the pattern does successfully resolve this challenge, it does
so at the cost of limiting some of it's potential. The pattern could conceivably
present more than one object at a time if screen real estate was not an issue
to consider. Presenting multiple objects at a time on the screen may allow
better use of the pattern in some way. Thus the pattern does trade o some
2 Solutions to the Challenges
2.1 Using Design Patterns to Apply HCI Principles
2.1.1 Handling Restrictions on Screen Size
The challenge of screen size is successfully resolved in some of the patterns
presented in the James Noble article, "GOF Patterns to GUI Patterns". The
GUI Strategy pattern is one of the patterns that resolves this challenge by
potential benets in conquering this challenge.
The article by Alexandrous and Papadopolous does provide a solution
for the challenge of Spatial Composition in the Storyboard pattern. The
Storyboard pattern consists of objects called scenes. During the developmen
phase of the user interface, specialists decide the utilization of the screen real
estate available in each of these scenes. These specialists take in to consid-
eration the challenges set forth by limitations in screen size and incorporate
them in to the design of the respective scene. Further, these specialists also
decide on positioning of the objects involved so as to optimize the screen
preserving screen real estate. This pattern denes a family of algorthims
space available.
and makes them interchangable. A separate strategy object is created to
Some of the design patterns proposed by Wake oer solutions to screen
represent each algortihm, and make the algorithm's parameters in to the
layout issues. His pattern entitled Make Position Re
ect Relationships sug-
attributes of the strategy object. The strategy object is then made to be
gests that the problem of the user dealing with thousands of on-screen objects

8 9
can be solved by carefully giving each object a visual representation. This mentioned earlier, the process before the design is so important that there
representation would be based on the object's attributes. The object would
then be placed carefully on the screen based on some structuring metaphor.
Wake's pattern entitled Arrange Zones by Similar Persistence addresses
the problem of sharing screen space between objects. He suggests that the
window be partitioned into zones, and each item in a given zone should
change at about the same rate. He suggests that dividing lines or other
visual indicators should separate the zones.
exibility in
When many objects compete for screen space, Wake suggests in his pat-
tern entitled Make Space contents within them that the size of the zones
should be proportional to the importance of the data within them. For ex-
ample, a spreadsheet devotes most of the screen space to the user's data,
while other elements (such as menus) are made smaller and kept along the
sides or top of the screen.
2.1.2 Pre-Design Research
are patterns that are specic to those areas. Overall there are domain specic
patterns (Business Domain Patterns, Business Process Patterns and Task
Patterns), which lead up to a suitable Conceptual Design Pattern.
2.2 Creating Extensible and Maintainable UI Software
In his paper about design patterns and computer graphics, Duy suggests
several possible solutions to the problems with extensibility and
CAD software which were stated in an earlier section of this report.
To solve the problem of dynamically scalable software, Duy suggests
creating only the most basic program code possible and then adding all ad-
ditional functionality using the Decorator pattern. This would allow separa-
tion of the additional features without aecting the base code. It would also
allow very easy run-time control of the functionality available in the system.
Duy suggests that dynamic functionality changes could be implemented
using the Visitor pattern. By creating a Visitor interface on all the objects
To deal with the challenges within predesign , the pattern approach provides
the added value of accumulated knowledge and design experience in a format
that's easy to understand. It should be equivalent to having a senior colleague
on your team. Results that have addressed these challenges include a number
of patterns that point to information that is generally valid about the UI
domain and also designs that have proven good for similar projects. As
10
in a graphical system, it makes it quite possible for signicant functionality
to be added to the program at any time. Again, in the context of graphics
application, this would allow a vendor to provide a more custom solution
to its clients, and respond more quickly to customer requests. The ability
to run arbitrary algorithms on objects or groups of objects is also provided
11
though use of the Visitor pattern.
The arbitrary and nested grouping of graphical objects could be handled
though the Composite pattern, according to Duy. Many graphics and GUI
exibility. On the other hand, if
objects consist of collections of other objects, and having a uniform interface
ex-
by which they can be manipulated makes the system easier for both the
programmer and the end user.
Black box or white box style reuse libraries can also increase program
exibility and maintainability. These libraries deal with the interaction be-
tween GUI libraries and data libraries. GUI libraries are often dicult to
understand and link to data libraries. A model can be constructed using
white-box style class libraries where all the GUI and Application data are
subclassed respectively. This was not as eective because white box does not
always have the best optimization. Using the black box method will opti-
\Technically, one of the key points to shift to black-box style
reuse is how black-box component can have white-box's superior
points, i.e., extendibility and
component users need not require limitless extendibility or
ibility for components, this will also solve the key; this means
all components are well constructed and prepared as "component
pattern" and all you have to do is just to choose one pattern from
the pattern library." [2]
2.3 Communicating Design Between Software Groups
To address the challenges of a clear concise description of the situation and
the appropriate pattern use, the objective is to begin combining this limited
vocabulary into sentences by rating the importance of each element. In this
mize the robustness and maintainability of the nished pattern. It is believed article, the rating scale was low, medium or high. More ne-grained numeric
that black box style reuse must be used, but must rst gain wide spread ac- scales are eective as well. To give a small example of how this solution
ceptence in the business community. However, most of the current reuse of might work we can say that the combination of \forces" (ratings) in the
code is done using white box style and this will continue due to its
exibility \context" of a project denes the \problem." In other words, if a system,
and long history of use. There are advantages and disadvantages for either has to be very easy to learn (Learnable) and very fast in use (Ecient),
creating white or black box style reusable libraries. The decision should be then the designer has a problem to solve because the two priorities are likely
based on the following factors: the domain, languages, object characteristics, to con
ict. However, if Learnable is rated high and Understandable is also
objects
exability and many others. high, then the problem is easier to solve, and there is a known \solution" {
Inoue compares black-box and white-box design in the following way:

12 13
providing the user with a good conceptual model. The use of these elements
make it a somewhat trivial task in getting an immediate feel for the amount
of eort that will be required to create the required design.
A pattern language that includes experiences from software engineering,
HCI, and the application domain can improve communication and team work
An example application using pattern languages called WorldBeat has
been implemented. It allows a user to:
 to conduct music;
 play virtual drums and other instruments;
throughout the team. This will lead to better products and design rationales.  hum a tune to nd music in the system;
Pattern Languages are a general structure for design knowledge. There three
elds from which pattern design language derive their knowledge: application  and improvise blues music without prior musical knowledge.
domain, human computer interaction, and software engineering. Each of the The patterns used in the application were represented in the interaction
elds' knowledge is used in the making of the pattern language. The following
improvements become possible:
 \Views and concepts of each profession become easier to understand
for others because the pattern concept of presentation is the same and
is geared towards clarity." [2]
 \Once the three pattern languages are understood, it will be easier
and in the internal software architecture. These patterns deal mainly with
implementing the music.
There are three patterns that unify the pattern language for WorldBeat
Musical Design Patterns Jazz is a high level pattern but at the low level
this can be represented by notes, rhythmic timing and other musical
factors.
to nd bridges and analogies between them: Abstract concepts from Interaction Design Patterns This pattern deals with the interactive de-
the application domain may give hints to high level user interface con- sign of the user interface.
sideration, which in turn may indicate overall system characteristics.
Concrete facts and objects from the application domain may point the Top level Visitor instructions.
concrete physical interaction details, which can be mapped to specic Medium Level This deals with the user information. (Attract user,
software objects and implementation." [2] Engage user, Deliver user command)

14 15
Software Design Patterns These are standard patterns that deal with the stage the vision is captured and the goal is to collect background information
implementation of the application in the coding. and set the scope, constructing a common understanding. To facilitate this,
business domain patterns are used. The pattern is reviewed and dierences
3 Analysis of Solutions in the businesses are adjusted to suite the specic situation. Task patterns is
then chosen based on the particular business pattern. There are many task
Overall, design patterns seem to be very useful in the world of graphical patterns for each business pattern. A task pattern helps describe the main
applications such as CAD software. Such software lends itself well to object- tasks and user groups that are involved in the system. The task pattern is
oriented techniques, since the goal of the software itself is the manipulation
of on-screen objects. Applying even the basic Gang-of-Four patterns to the
design of CAD software could improve the experience of the end-user, and
make the software easier to design for the developer.
The Interactive Usability system of creating patterns seems to, at the very
least, be a method of properly preparing for the design of a new user interface.
It presents an opportunity to add patterns very easily since the elements
optimized by utilizing user prole and task analysis activities. Similarities
and discrepancies between this process are the basis for conceptual design
patterns in which information and activities are arranged to support the
ultimate goal of any user interface { to eectively support the user's task.
Graphical design patterns for screen layout are also important, and their
use will make a lot of designers think harder when they are developing a GUI.
They bring to the forefront many important issues of HCI and psychology
that are modied are natural concerns that are normally accompanied by
user interfaces. The result of this approach to the challenges of UI design is
essentially a pattern language that can be extended and evolved with further
use. It then becomes a collaboration rather than an individual invention.
Further, usability of a system emerges as a product of User, Task and
Context of Use. The intent of the suggested patterns is to strengthen the
that many programmers do not consider, such as the needs of the user! By
following the patterns, programmers can ensure they make well-researched
choices when they design their interface.
The solution presented in the James Noble article, \GOF Patterns for
GUI Patterns," successfully resolves the challenge of screen real estate but
does have possible negative consequences. As was seen in the example of the
analysis of these three components and communicate knowledge from previ- Strategy pattern's resolution, some benets of the pattern may be lost. This
ous designs. The design process starts with the system denition. During this particular solution would be satisfactory in a problem where multiple strategy

16 17
instances were not benecial when created concurrently. Thus there may plication, which allows all domain experts to interact, subtle but possibly
exist instances where successful resolution of the screen real estate challenge important details could be lost. This problem could lead to an inadequate
limits a pattern from realizing it's full potential. solution being produced. Pattern languages are developed over a process of
The idea that the Strategy pattern causes such tradeos may result in many similar projects. They incorporate an easy to remember vocabulary for
the loss of other unforseen benets. This could apply to any pattern where
screen real-estate is an issue. This challenge should not always be resolved
depending on the needs of the application and the potential performace lost
in the respective pattern.
Alexandrous and Papadapolous in their article Storyboard, propose to
solve their existing spatial composition challenge by employing the Storyboard
communication about successful solutions across discipline boundaries. The
pattern approach helps new employees in an interdisciplinary development
team learn more quickly because they can use a common language and set
of rules to communicate more eectively.
There will always be questions regarding the best way to use design pat-
terns in a graphical application. Questions of which patterns to apply, how
pattern and calling on a team of specialists. This solution is seemingly an
ideal one, provided that such resources are available. The spatial composition
must be understood by a diverse set of developers in order to avoid complexity
that could occur in implementation. Thus the solution gives an entire team
of technical and non-technical members a better understanding of the spatial
composition of a scene while addressing screen space limitation challenges.
The solution presented in the Storyboard pattern with regards to the chal-
lenge of communication between design groups seems to be a valid one. The
pattern allows for the bringing together of ideas and experiences, facilitating
the mulitdisciplinary approach that is required. The solution has one possible
they should be used, and how they should be modied will be a constant
source of useful research topics for the software design community.
4 Questions for Further Study
There are many open questions surrounding the world of design patterns in
GUI applications. Issues range from communications to pre-design to imple-
mentation of GUI-based systems. The classic computer science problems of
eeciency, reliability, and maintainablility remain prevelant despite extensive
use of design patterns.
short coming: the potential problem lies in the oversimplication inherent in The process of pre-design and research tends to produce successful results
the use of the Storyboard pattern. There is a possibility that through sim- in addressing the challenges of creating user interfaces. The article suggests

18 19
that it is inappropriate to change patterns or write new patterns during further pattern-gained knowledge for pattern approached designs. In addi-
the design process. After all knowledge is gained, the patterns can later tion to all open questions it is also extremely benecial to document new
be evaluated. They can then be updated based on the new situations that patterns carefully because they will have most value when they are shared
arose from that project. However, there are still open questions that remain with others.
requiring further research. A list of ve questions follows below.
1. Task Patterns: What information should be captured and passed on?
2. What is the proper granularity of a task pattern?
3. How well can conceptual design solutions be described with patterns?
4. How do we handle conceptual design of composite systems (sometimes
data oriented and sometimes activity oriented)?
Questions exist about human-computer interaction and the best way to
do screen layout. As more research is done in these areas, hopefully more
people will answer questions about how computers are used by end-users. If
these concerns are addressed in the form of new graphical design patterns,
both new and experienced software designers will be able to easily apply
those techniques.
In the article by James Noble, the question of when to apply particular
GUI design patterns in addressing the challenge of screen real estate is left
5. When describing these new patterns, how can the quality (validity) of open. When is it worth sacricing extra capabilities and benets of a pattern
each pattern be determined? Is there a way of quantifying this? for optimization of screen real estate? Further, instead of sacricing one
or the other, can either one be sacriced to dierent degrees in order to
A particluar concern that entails further research is to look at the var-
accomodate the respective demands of an application? These questions are
ious combinations of usability attributes. This will assist in clarifying the
open to debate. In addition, the answers to such questions may shape when
diculty level of each combination. It may also help to analyze existing
and how the challenge of screen real estate is resolved.
computer applications to see which usability patterns have implicitly driven
The Storyboard article by Alexandrous and Papadapolous suggests a pos-
past designs. Many authors also suggest that when you meet a system that
sibility of creating a pattern language based on the Storyboard pattern's scene
has usability problems, try the idea that there are forces that are dicult
components. Can a pattern language based on this pattern evolve and can
to resolve before assuming that the designer was careless. This may lead to
it be successful? Is the multidisciplinary approach fostered by this pattern

20 21
benical in addressing the challenges of screen real estate restrictions? These
questions may shed some light on the future evolution of this pattern and its
approach to this challenge.
A lot of open questions remain about whether black box or white box
style reuse should be used in making pattern libraries. The factors such as
domain, language, and object structure all determine the best way of doing a
certain task. The main question would be, what the best way to standardize
these pattern decisions is, and how we can make this accessable for every
user to understand and reuse?
A proper formalization of the interconnection of pattern languages to set
up computer based developement tools is an area of understanding not yet
explored to its fullest potential. Pattern Languages have only scratched the
puter Human Interaction Design: A pattern Approach. Linz University,
Linz, Austria, 1998.
[3] George Casaday. Notes on a Pattern Language for Interactive Usability.
CHI 97 Electronic Publications, Newton, MA. USA, 1997.
[4] Daniel J. Duy. Design Patterns and Graphics. Datasim BV, Amsterdam,
1999.
[5] Asa Grunland and Daniel Lafreniere. A Pattern-Supported Approach to
the User Interface Design. Ericsson Software Technology, 1999.
[6] Takeshi Inoue. From Class Libraries to Component-Based Development.
Yokogawa Electric Corpation, Tokyo, 1998.
surface of understanding in this eld of study. With proper formalization, [7] James Noble. GOF Patterns for GUI design. Macquarie University,
design patterns may greatly improve the communication between designers, Sydney, 1997.
developers, and the end users.
[8] William C. Wake. Patterns for Interactive Applications. PLoP '98, Mon-
ticello, Illinois, 1998.
References
[1] Chrystara C. Alexander and George A. Papadopoulos. Storyboard Gen-
erator: An Architectural Pattern for Multimedia Applications. PLoP '96,
Monticello, Illinois, 1996.

[2] Jan O. Borchers. LATEX, Breaking teh Interdisciplinary Limits of Com-

22 23