Designers’ perception during sketching: An

examination of Creative Segment theory

using eye movements
Lingyun Sun, Wei Xiang, Chunlei Chai, Zhiyuan Yang and Kejun Zhang,
Modern Industrial Design Institute, Zhejiang University, Caoguangbiao East
Building 101, Yuquan Campus, Hangzhou 310017, China

As a visual thinking tool, sketching strongly supports the conceptual stage of

product design. Designers perceive their sketches and discover new ideas.
Creative Segment theory models sketching process as a Creative Segment tree,
and reveals the Creative Segment-centered pattern of sketching activities.
However, the perception of sketches is important during sketching, whether
designers’ perception is in accord with this theory has not been examined. This
article reports an experiment that recorded participants’ eye movements to
analyze their perception. Participants displayed unique eye movement features
during Creative Segments, their fixations changed during the generation of
Creative Segments and differed when generating Creative Segments on the
three-level branches of the tree. These results provide evidence for the Creative
Segment theory.
Ó 2014 Elsevier Ltd. All rights reserved.

Keywords: conceptual design, design theory, design activity, perception

C
onceptual sketching is a visual thinking process; designers sketch
their ideas, inspect their sketches and perceive new possibilities,
generating more creative ideas. Various articles have studied how de-
signers benefited from sketching processes and ambiguous sketches
(Goldschmidt, 1991; Temple, 1994). Researchers segmented the sketching
process and tracked the development of design ideas (Kavakli & Gero,
2002; Suwa, Purcell, & Gero, 1998), and compared activities between novices
and experts to reveal better sketching strategies (Cross, 2004).

The sketching process could be modeled using Creative Segment theory (Sun,
Xiang, Chai, Wang, & Huang, 2014). This theory defines Creative Segment as
the loop involving idea generation, idea expression and visual feedback, and
arranges Creative Segments in a Creative Segment tree. In this way, the Cre-
ative Segment tree displays the evolution paths of ideas to final outcomes,
providing possibilities for valuable understanding of sketching. Sun et al.
Corresponding author: (2014) have conducted experiments that analyzed the sketching activities to
Chunlei Chai validate this theory, but have not examined the visual feedback of sketches.
dishengchai@126.com The visual feedback, namely the perception of sketches, is a critical part during
www.elsevier.com/locate/destud
0142-694X $ - see front matter Design Studies 35 (2014) 593e613
http://dx.doi.org/10.1016/j.destud.2014.04.004 593
Ó 2014 Elsevier Ltd. All rights reserved.
 sketching that triggered new thoughts (Goldschmidt, 1991), and needs to be
further analyzed in order to examine the Creative Segment theory.

Designers’ eye movements provide an effective data source for the analyses of
perception during sketching. The fixations display items of interest (Findlay &
Gilchrist, 2003). Participants move their eyes according to demands of the task
(Sch€
utz, Braun, & Gegenfurtner, 2011), and fixate on critical regions during
problem solving (Hodgson, Tiesman, Owen, & Kennard, 2002); fixations
also serve to guide thoughts and stimulate solutions (Thomas & Lleras,
2007). Apart from fixations, other eye movement data, such as fixation dura-
tion and pupil diameter, indicate participants’ concentration level (Miall &
Tchalenko, 2001; Minassian, Granholm, Verney, & Perry, 2004).

In this article, we optimized the Creative Segment theory and employed eye-
tracking techniques to examine designers’ perception. Designers focus on
multi-level factors involving design goals, functions, and structures during
sketching; the Creative Segments could be arranged according to their goals,
functions, and structures. A digital sketching experiment was conducted, par-
ticipants’ eye movements were recorded, and the ideas during sketching were
recognized and arranged as three-level Creative Segment trees. The eye move-
ments as the tree grew and the eye movement data among Creative Segments
on the three-level branches were analyzed to examine the Creative Segment
theory.

1 Previous studies
1.1 Sketching as a visual thinking process
The early stage of industrial design faces ill-structured problems (Restrepo &
Henri, 2004). A typical design process for this problem is solution-based
(Dorst & Cross, 2001). The solutions are generated for current problems,
and new problems are discovered based on the solutions, stimulating new so-
lutions. This co-evolution gradually set boundaries to the problems and lead to
proper solutions.

Sketching supports the reflective and ambiguous process of conceptual indus-

trial design (Cross, 2006; Temple, 1994). Concepts can be stored and organized
visually (Kan, Barsalou, Solomon, Minor, & Thompson-Schill, 2003). Sketch
as a quick visual representation records the solutions in real time, enabling a
flexible recall of the functional issues (R€ omer, Pache, Weißhahn,
Lindemann, & Hacker, 2001; Suwa, Gero, & Purcell, 1998). From the perspec-
tive of knowledge management, sketches support knowledge creation, knowl-
edge sharing, and knowledge documentation (Eppler & Pfister, 2011).

Sketching builds conversation between the internal mental and external mate-
rializing activities (Schutze, Sachse, & Romer, 2003). Taura et al. (2011)

594 Design Studies Vol 35 No. 6 November 2014

 recognized the conversation as an ‘inexplicit’ cognition. Designers developed
multiple tentative representations, causing irregularities, and reinterpreted
the visual shapes and images to stimulate new thoughts (Goldschmidt, 1997;
Oxman, 2002). Jin and Chusilp (2006) combined this reinterpretation with
the idea generation, and proposed mental iterations involving generating,
composing, and evaluating. Goldschmidt summarized two methods of reinter-
pretation, involving ‘seeing that’ and ‘seeing as’, in the sketching process
(Goldschmidt, 1991); designers understood what they had sketched and
improved them, or recognized new meanings from the sketches. Suwa,
Gero, and Purcell (1999) also reported the driving force behind unexpected
discoveries, designers discovered hidden features by attending to previously
sketched elements together, or viewing sketches with new design concepts
derived from design requirements invented in sketching. The discoveries in
sketching altered the perspectives and conceptual meanings of the sketches, re-
sulting in continually new interpretation and discoveries.

The perception of previous drawn sketches, accompanied with the expressing

activities, forms a complete visual thinking process. Previous studies of percep-
tion focused on sketch descriptions using think-aloud method (Menezes &
Lawson, 2006; Wiegers, Langeveld, & Vergeest, 2011). Sketching is a dynamic
process, the think-aloud method cannot record the fast changing perception
precisely. Apart from think-aloud method, eye movements have been used
to interpret designers’ perception of geometric forms (Jowers, Prats,
McKay, & Garner, 2013; Prats, Garner, Jowers, McKay, & Pedreira, 2010).
Therefore, this article turned to eye movements to analyze designers’ percep-
tion of sketches, thus examining the Creative Segment theory.

1.2 Eye movements

Eye movements are not smooth and continuous. When exploring a scene, the
eyes jump in a fast succession among places and fixate on these places to recog-
nize items (Hubel, 1995). The jumps in visual exploration, known as saccades,
direct the retina to places of interest (Duchowski, 2007). The fixation stabilized
fovea over an object; since the fovea supports a fine-detail vision, this stabili-
zation enables objects to be carefully examined. In this way, the fixations and
saccades provide a clear vision of the scene. Figure 1 shows an example of eye
movements during a second of sketching process.

Fixations correlate with designers’ attention. There are two kinds of attention
during viewing: the overt attention describes attending by looking, and the
covert attention describes attending without looking (Findlay & Gilchrist,
2003). Rayner (2009) suggested an overlapping of eye location and covert
attention during fixation in a review of reading, scene perception, and visual
search studies. But visual perception is a complex process, with some effort
people could also pay attention to objects without fixating their eyes on that
(Posner, 1980). Apart from the mismatch of fixations and covert attention,

Designers’ perception during sketching 595

Figure 1 Example of eye
movements during sketching
(the nodes represent fixations
and the lines connecting nodes
represent saccades, the diam-
eters of nodes represents the
duration of fixations)

fixating on an object does not ensure that this object is perceived and processed
well. It just gives clues to what people are trying to do and what information is
available to them. This article adapts the active vision theory proposed by
Findlay and Gilchrist (2003) to interpret fixation. This theory regards covert
attention as the result of the motor system that offers peripheral preview of
next fixation location to generate saccades; and regards fixation as an indica-
tion of items that are of interest. Therefore, fixations reflect the items that peo-
ple make an effort to perceive.

Fixations display the items of interest in real time. The newly attended geomet-
rical elements constantly overwrite elements in visual working memory during
problem solving (Epelboim & Suppes, 2001); when the problems that people
solved referred to certain elements, multiple rescans of the same elements
were observed. Ballard, Hayhoe, and Pelz (1995) reported this gathering strat-
egy, with task-relevant items observed just before it was required in working
memory. As Rensink (2002) proposed, the world could be treated as a huge
memory that was accessed by eyes.

Since eyes fixate on the items needed in working memory, eye movements
reflect participants’ processing of the current scene. Participants fixate on lo-
cations critical for tasks, indicating their analysis of the task (Johansson,
Westling, Backstrom, & Flanagan, 2001). Prats et al. (2010) observed different
fixation locations when participants had different interpretations of ambig-
uous shapes. Nakakoji, Yamamoto, Takada, and Reeves (2000) also used fix-
ations to analyze participants’ reflection in the early stage of writing task.
When people are about to solve a problem, they display a different form of
eye movements that focus more on solutions (Ellis, Glaholt, & Reingold,
2011). This phenomenon has also been observed by Ohsawa, Maeda,
Yoshida, and IEEE (2007), the eyes draw auxiliary lines as an externalization
of the hidden relationship between objects. Task processing guides eye

596 Design Studies Vol 35 No. 6 November 2014

 movements; eye movements may also induce new ways of task processing.
Thomas and Lleras (2007) reported that guiding participants’ eye movements
through an unrelated task could help them find the solution of the radiation
problem. These results have been repeated using other guiding methods,
such as another person’s gaze (Litchfield & Ball, 2011), and, in a more strict
comparison, by Groen and Noyes’ experiment (2010).

Researchers collected multiple eye movement data. Apart from fixation loca-
tions, the fixation durations and pupil dilation also provide effective data to
study the cognitive process. Fixation durations correlate with cognitive activ-
ities (Nuthmann, Smith, Engbert, & Henderson, 2010). That is, when people
make an effort, their fixation durations increase. Pupillary dilation is an indi-
cator of concentration (Causse, Senard, Demonet, & Pastor, 2010), with pupil
diameter increasing in complex tasks (Minassian et al., 2004).

In general, eye movements are guided by the task goal, displaying what is be-
ing needed; eye movement data elicit the cognitive states. These data enable
the analyses of designers’ perception during sketching.

2 Creative Segment theory

Creative Segment theory was first proposed in one study of designers’ sketch-
ing activities (Sun et al., 2014). It considers the participation of brain, hand,
and eye during sketching, and describes the sketching as a Creative
Segment-centered process. Designers hesitate before Creative Segment, ex-
press their ideas during Creative Segment, and evaluate the ideas to discover
new possibilities after Creative Segment. The Creative Segments are arranged
into a Creative Segment tree that shows designers’ explorations during sketch-
ing. In this article, we also considered the genealogy of ideas and accordingly
optimized the structure of the tree.

2.1 Creative Segment

Designers constantly try to solve design problems from multiple perspectives.
Each time a designer generates an inspiration to solve a problem, he/she gets
an idea. The designer continually generates and refines ideas that change parts
of objects or complete objects, and these ideas evolve and combine to final
design solutions. Therefore, ideas rather than the design solutions are the vital
element during the sketching. Since the sketching process comprises the inter-
active actions of brain, hand, and eye, the loop containing idea generation,
idea expression and visual feedback is named Creative Segment, representing
a new finding during sketching. It starts from the point when designers express
the idea by a drawing activity, and ends after the activity has been completed.
The Creative Segment paper (Sun et al., 2014) provides a figure explaining the
Creative Segment during sketching (Figure 2). During the sketching presented
in the figure, the designer designed chairs using a tablet for 40 min.

Designers’ perception during sketching 597

Figure 2 Examples of Crea-
tive Segments in the sketching
process

2.2 Creative Segment tree

By arranging Creative Segments into a tree structure, the Creative Segment
tree displays the design space that designers explored during sketching, with
each branch representing a design path. We would analyze the genealogy of
these ideas and optimize the Creative Segments tree accordingly.

There are multi-level explorations during sketching. Rodgers, Green, and

McGown (2000) proposed lateral transformation and vertical transformation
of ideas, corresponding to idea alternation and idea implementation in sketch-
ing. van der Lugt (2000) also employed three levels of solution similarity in his
studies, ranging from the solutions involving different functions to those offer-
ing small and auxiliary modifications. The categorization principles are similar.
As shown in Table 1, the three main categorization principles of ideas are their
goals and behavior, their functions to achieve goals, and their structures.

Designers propose several functions for one goal and several structures for one
function during sketching. Every new Creative Segment contains an idea that
offers a new goal, or a new function for one goal, or a new structure for one
function; the Creative Segments have three levels of similarity: different
goal, same goal and different function, same function and different structure.
Creative Segment tree are accordingly have three-level structure. The first
Table 1 Categorizations of ideas

Authors Goal level Function level Structure level

Gero (1990) Behavior Function Structure

Deng (2002) Purpose function Action function Structure
Shah, Smith, and Vargas-Hernandez (2003) \ Physical, working Embodiment, detail
Jin and Benami (2010) Behavior Function Form

598 Design Studies Vol 35 No. 6 November 2014

 Creative Segments designed for different goals belong to the main branch, Cre-
ative Segments that provide new functions to achieve the same goal constitute
function branches, and Creative Segments that provide new structures of the
same function constitute sub branches. Ideas in the same branches are sorted
by time sequence. In this way, the tree displayed the three kinds of explorations
and clear evolution paths of ideas.

A typical Creative Segment tree is shown in Figure 3. The numbers represent the
sketching sequence of the Creative Segments. During the sketching process, the
designer designed chairs that enhanced communication. He sketched Creative
Segment 1 to make people sit closer (goal 1 to make people sit closer), Creative
Segments 2 to offer topics for communication (goal 2 to offer common topics),
and Creative Segments 3 to record and display others’ talking topics (goal 2).
The designer also sketched Creative Segment 4 that allowed people to speak
amusingly through an echo wall (goal 3 to be comical in communication), and
Creative Segment 5 that offered an echo stone (goal 3, the same function with
Creative Segment 4). Then, he generated Creative Segment 6 that displayed hott-
est news to induce discussion (goal 2) and Creative Segment 7 that offered swings
for families (goal 1). Finally, the designer proposed Creative Segment 8 that
modified Creative Segment 7 to wave-like swings (goal 1, the same function
with Creative Segment 7), and Creative Segment 9 that offered a peaceful place
displaying poetry (goal 4 to be comfortable with the environment).

During the sketching process, Creative Segments 1, 2, 4, and 9 were the first
solutions proposing four different goals. These were categorized as the main
branch. Creative Segments 7, 3, and 6 employed different functions for the
same goals and constituted function branches. Creative Segment 8 was a struc-
ture modification of Creative Segment 7 on wave-like swings, and Creative
Segment 5 was an echo stone evolved from the echo wall of Creative Segment
4. They were categorized as sub branches.

Figure 3 Creative Segment tree of a sketching process

Designers’ perception during sketching 599

 2.3 Sketching activities as Creative Segment tree grows
Creative Segment tree grows with the designers’ sketching activities. Designers
draw depictions and write annotations to express their ideas, explain and eval-
uate the idea in detail, and explore other possibilities, leading to new Creative
Segments. Kavakli and Gero (2002) divided the drawing activities into activities
related to depictions and activities related to annotations (symbols and words).
We further divide activities related to depictions into activities related to objects
and activities related to contexts. Apart from the coding on sketching process,
Kavakli, Scrivener, and Ball (1998) also proposed the overstriking and the
perspective view when sketching single ideas. Therefore, five kinds of drawing
activities are recognized involving drawing objects, drawing objects from
another perspective, drawing context, drawing annotations, and overstriking.
Creative Segments correspond to the first activities that express the ideas.

2.4 Hypotheses
Creative Segment theory regards the sketching as a Creative Segment-centered
process. If designers’ perception during sketching is in accord with Creative
Segment theory, their eye movements should display some features related
to Creative Segments.

Hypothesis 1: Eye movement features during Creative Segments are different.

The fixations are longer and pupil diameters dilate. Designers sketch ideas in
Creative Segments and explain them in following activities. They concentrate
and make more efforts when expressing new ideas, leading to longer fixations
and pupil dilations.
Hypothesis 2: Designers’ eye fixations change with the sketching activities be-
tween successive Creative Segments. The explanation and evaluation need
reference to the prior Creative Segment, and new thoughts that emerge
from these explanations direct longer fixations to other items.
Hypothesis 3: Designers perform different eye movement patterns when gener-
ating Creative Segments of the three-level branches. They pay attention to a
large amount of items to discover new goals and explain these goals carefully.
Therefore, designers fixated more Creative Segments before Creative Segments
on main branches, and Creative Segments on main branches receive fixations
from more sketching activities. Because many factors need to be considered
when exploring structures, designers fixate a wider range of Creative Segments
before Creative Segments on sub branches than those on function branches.

3 Method
3.1 Experimental setup
A total of 41 participants took part in the study. They were Chinese students
with an average of 5.7 years’ learning in industrial design, and an average age
of 23.1. The experimental setup contained Sketchpainter sketching software,

600 Design Studies Vol 35 No. 6 November 2014

 Camstudio 7.0 screen recording software, a Wacom PTK 640 (Saitama, Japan)
tablet, a 22-inch monitor, and an eye-tracking system. Sketchpainter offered
only basic drawing functions; brush was preset and sketches were saved auto-
matically, ensuring an undisturbed digital sketching process. The resolution of
the computer screen was 1920
1080 pixels at a refresh rate of 60 Hz. Partic-
ipants viewed from a distance of 70 cm.

3.2 Data collection

Eye movements were sampled at 120 Hz using the Tobii (Stockholm, Sweden)
X120 eye tracker and Tobii Studio 3.13, with spatial accuracy of better than
0.3 .

Sketching processes were understood by employing the post-reporting

method. The think-aloud method was not appropriate because its influence
on eye movements had not been studied. Eye movement data possibly would
be distorted by the additional workload of expressing thoughts verbally.
Therefore, we recorded the video of participants’ whole sketching process, in-
structed participants to watch the video and report what they were sketching.

3.3 Design task

A task of ‘facilities that enhance communication in a park’ was used in the
study. The description was simple; participants could recall the design task
with ease, focusing on the screen during the whole sketching process.

3.4 Procedure
Participants were first introduced to the whole experiment and the eye-
tracking system, and then instructed to conduct a pre-task to become familiar
with the tablet and eye tracker. They found a comfortable position to sketch
without moving their heads, and their eye movement data were then cali-
brated. Subsequently, participants were verbally introduced the design task
and participated in a 30-min sketching process with the calibrated eye tracker.
The sketching processes were recorded, and eye movements were monitored
(Figure 4). Immediately after the sketching processes, participants watched
the recorded sketching process and reported their sketching and ideas to
researchers.

3.5 Data analyses

The eye movement data might be lost when participants moved their heads
during sketching. Eleven participants losing more than 20% samples of the
eye movement data were excluded. The eye movement data were filtered and
fixations shorter than 100 ms were deleted. The reports of the other 30 partic-
ipants were typed.

3.5.1 Construction of Creative Segment tree

Ideas were recognized by researchers and participants talking together. The
participants continually explained what they were doing while watching the

Designers’ perception during sketching 601

Figure 4 Participant in the experiment and the sketches

video. When participants reported ideas, the researchers typed the reported
goals, functions and structures of the ideas, as well as the time. Researchers
then confirmed these ideas with the participants.

Every idea corresponded to a unique Creative Segment. The Creative Segment

trees were constructed individually by two researchers. As the goals, functions
and structures had been reported, the two researchers only had four differences
(Kappa ¼ 0.94, p < 0.01). The differences were settled by discussion. Figure 3
displayed an example of Creative Segment tree generated from the sketching
process that was described in the last two paragraphs of Section 2.2.

3.5.2 Segmentation of sketching activities

The sketching activities were recognized according to the categories in Section
2.3 (Table 2). A participants’ sketching processes were randomly chosen to be
segmented individually by two researchers. The two researchers only differed
on 6 out of 61 sketching activities and reached consensus after discussion
(Kappa ¼ 0.86, p < 0.01). Other sketching processes were segmented by one
researcher.

3.5.3 Sketching activities between Creative Segments

Participants sketched and paused during the sketching process. As shown in
Figure 5, the duration of a sketching activity was defined as the sketching ac-
tivity and the pause before it. The duration between successive Creative

Table 2 Categories of sketching activities

Category Example

Drawing objects Draw an echo wall

Drawing objects from another perspective Draw different angles of the wall
Drawing context Draw how users interact with the wall
Drawing annotations Write textual descriptions
Overstriking Overstrike the wall

602 Design Studies Vol 35 No. 6 November 2014

Figure 5 Sketching activities
between Creative Segments
(A represents sketching activ-
ity, CS represents Creative
Segment)

Segments could thus be divided into the sketching activities between Creative
Segments and the pauses before Creative Segments.

To describe the relative position of a sketching activity between two Creative

Segments, we used 0 to represent the first activity between two Creative Seg-
ments and 1 to represent the last activity. The third sketching activity in
Figure 6 would get 0.4.

3.5.4 Fixations on sketches

Sketching emphasizes on exploring by drawing. The knowledge and properties
designers abstract from existing ideas act as a linkage to new ideas (Goel, 1997;
Sim & Duffy, 2003). Therefore, the fixations between two Creative Segments
reveal the items needed for generating the following Creative Segment. To
analyze the location of fixations, three types of area of interest (AOI) on
sketches were defined. These are:

Creative Segment AOIs: sketches drawn by the first activities that express
ideas;
Explanatory AOIs: sketches drawn by the activities that explaining ideas;

Figure 6 Three different types of areas of interest (AOIs): Creative Segment AOI, explanatory AOI and exploratory AOI (the Chinese text
explains that the chair recorded your hopes, emotions and other words you wanted to say to others)

Designers’ perception during sketching 603

 Exploratory AOIs: other exploratory drawings.

The intentions of sketching activities had been reported by participants, corre-

sponding to the three types of AOIs. As shown in Figure 6, one participant
sketched a chair that recorded users’ sound (Creative Segment AOI). He
then sketched the users and wrote some annotations (explanatory AOI). After
that, the participant arbitrarily drew a chair without any intention (explor-
atory AOI). The boundaries of AOI were set close to the sketches.

3.5.5 Links between Creative Segments

Designers could recall a prior Creative Segment when fixating on any parts of
the Creative Segment. A directed link was thus established when designers
looked at the AOI of another Creative Segment before a Creative Segment
(Figure 7). Two Creative Segment AOIs overlapped in one participant’s
sketches. If the participant fixated on the overlapping area, two links were es-
tablished. Creative Segments were numbered according to the time sequence,
and the distance of a link was defined as the difference between their sequence
numbers. Thus, the link distance of successive Creative Segments was 1.
Figure 8 shows the links built during a sketching process.

Two variables were defined to describe the links between Creative Segments:
Looking Back (LB), the number of links designers established before each Cre-
ative Segment, indicating the Creative Segments designers referred to; and
Received Looking (RL), the number of links each Creative Segment received,
indicating fixations in the subsequent sketching process. Thus, idea 6 in
Figure 8 has an LB of 2 and an RL of 1. The LB and RL were compared
among Creative Segments on the three-level branches.

4 Results
Thirty participants sketched 231 Creative Segments. Two participants sketch-
ing less than 3 Creative Segments were excluded in the construction of Creative
Segment tree. The other 28 participants sketched 228 Creative Segments.

4.1 Creative Segment

Participants spent more time fixating during Creative Segments than during
other periods. Fixation durations in Creative Segments were longer

Figure 7 Directed links estab-

lished between Creative Seg-
ments (the numbers
represent the sketching
sequence)

604 Design Studies Vol 35 No. 6 November 2014

Figure 8 Directed links established in a participant’s sketching process (the numbers represent the sketching sequence)

(t ¼ 4.73, p < 0.01), whereas the number of fixations per second displayed no
difference (t ¼ 1.13, p ¼ 0.27). Participants had an average fixation of 430 ms
during a Creative Segment and 380 ms in other intervals. The pupil diameter
was not affected by Creative Segments and time (Creative Segment: F ¼ 0.01,
p ¼ 0.98; time: F ¼ 0.23, p ¼ 0.64), while the first second of Creative Segments
had a larger pupil diameter than the last second of Creative Segments (t ¼ 3.98,
p < 0.01).

4.2 Eye movements during the sketching activities

Participants during the sketching activities gradually turned their fixations
from the prior Creative Segments to other sketches. It was found that
22.87% of the sketching activities between Creative Segments did not refer
to the prior Creative Segment. These sketching activities had an average posi-
tion of 0.77 (SD ¼ 0.11), which was close to the last sketching activities
(Figure 9). 19.71% of sketching activities between Creative Segments referred
to other Creative Segments, with an average position of 0.70 (SD ¼ 0.22).

During the pauses before Creative Segments, the average fixation durations on
the three types of AOIs differed significantly (F ¼ 13.67, p < 0.01). The fixa-
tions on exploratory AOIs were on average 319 ms, while those on explanatory
AOIs and Creative Segment AOIs were on average 232 ms and 231 ms.

4.3 Links between Creative Segments

Participants tended to look at Creative Segments just sketched before. These
228 Creative Segments built 312 links. These links had an average distance

Figure 9 Average position of sketching activities (A represents the sketching activity, CS represents the Creative Segment)

Designers’ perception during sketching 605

 of 1.59 while the link distance would be 4.96 if participants looked back
randomly.

A link could be established twice during the sketching activities between Cre-
ative Segments and the pauses before Creative Segments. 82 out of 228 Crea-
tive Segments referred to prior Creative Segments during the pauses. These
Creative Segments build 123 links during the sketching activities and 113 links
during the pauses. Participants tended to recheck the referred Creative Seg-
ments during the pauses. As shown in Figure 10, 99 out of 123 links established
during the sketching activities were repeated. Only 14 new links were estab-
lished during the pauses.

4.4 Eye movement variations among Creative Segments on

the three-level branches
Among the 228 Creative Segments, there were 112 Creative Segments on main
branches, 90 Creative Segments on function branches, and 26 Creative Seg-
ments on sub branches. The average distance between two successive Creative
Segments was 1.72 on function branches and 1.33 on sub branches. Creative
Segments on sub branches were sketched consecutively.

4.4.1 Looking back of Creative Segments

The time and branch types did not have significant effect on LB (time:
F ¼ 0.21, p ¼ 0.65; branch type: F ¼ 1.12, p ¼ 0.31). But LB would increase
if Creative Segments on the main branches were generated posterior to Crea-
tive Segments on other branches (t ¼ 1.82, p ¼ 0.08).

The range of links was influenced by branch types. Creative Segments on main
branches established 129 links. Creative Segments on function branches estab-
lished 78 links on the same branches and 68 links between different branches.

Figure 10 Link distance of the

82 Creative Segments

606 Design Studies Vol 35 No. 6 November 2014

 Creative Segments on sub branches had a smaller proportion of outer connec-
tions (c2 ¼ 11.30, p < 0.01), establishing 31 links on the same sub branches and
6 links between different branches. Since participants sketch Creative Seg-
ments successively on different function branches, the tendency to look at
adjacent Creative Segments might provide Creative Segments on function
branches a higher proportion of outer connections. Therefore, we further
compared the links between Creative Segments that were not adjacent. As
shown in Figure 11, Creative Segments on the sub branches still had a smaller
proportion of outer connections (c2 ¼ 4.99, p ¼ 0.03).

4.4.2 Received looking of Creative Segments

Creative Segments on the main branches received fixations from a larger pro-
portion of following sketching activities (t ¼ 3.06, p ¼ 0.01). The average po-
sition of sketching activities that did not refer to Creative Segments was 0.84
for main branches and 0.74 for other branches.

Creative Segments on the main branches also received fixations from more
following Creative Segments. Both time and branch type affected the number
of received links (time: F ¼ 10.30, p ¼ 0.01; branch type: F ¼ 10.65, p ¼ 0.01).
Creative Segments on the main branch received 1.46 links and those on other
branches received 1.27 links.

All variations among Creative Segments on the three-level branches were sum-
marized in Table 3.

5 Discussion
5.1 Creative Segment
Hypothesis 1 proves to be correct. Creative Segments have unique eye move-
ment features. Compared with other intervals, participants tended to be more

Figure 11 The number of links

built by Creative Segments on
three-level branches (A: the
links between adjacent Crea-
tive Segments; NA: the links
between other Creative
Segments)

Designers’ perception during sketching 607

Table 3 Comparison of Creative Segments on the three-level branches (>: more or larger, <: fewer or smaller)

Main branch Function branch Sub branch

Distance between successive Creative Segments \ > <

Fixations on prior Creative Segments > < <
Links among different branches \ > <
Fixations from following sketching activities > < <
Fixations from following Creative Segments > < <

concentrated when expressing the ideas. They fixated longer during Creative
Segments and had larger pupil diameter in the beginning. Since participants
fixate on the sketches they were drawing, the longer fixation indicates more ef-
forts in perception of ideas. The pupil dilation also confirms the concentration
during the beginning of Creative Segments.

5.2 Eye movements as Creative Segment tree grows

Hypothesis 2 proves to be correct. Participants’ fixations change from the
prior Creative Segment to other items during the sketching activities between
Creative Segments. Only 22.87% of sketching activities between Creative Seg-
ments did not fixate on an area pertaining to a prior Creative Segment, and
this mainly appeared in the latter half of the sketching activities. Activities
that fixated on other Creative Segments also occurred in the latter half of
this duration. After expressing their creative segments, participants first
explain and check the details, then look at other areas and explore new possi-
bilities, through which they finally realize new creative segments.

The fixations during the pauses before Creative Segments offer new details.
Participants rechecked Creative Segments and fixated more on new thoughts.
87.61% of links built during the pauses had been established before the pause.
At the same time, participants fixated longer on exploratory AOIs rather than
explanatory AOIs and Creative Segment AOIs. They checked what they had
known and focused on ambiguous items.

Eye movement data reveal the Creative Segment-related perception during

sketching. After generating a Creative Segment, participants focused on this
Creative Segment to explain and evaluate their ideas, while they then referred
to other Creative Segments and exploratory items to explore new ideas. The
transformation of focused explanation to broad exploration support the gen-
eration of new Creative Segments.

5.3 Eye movement variations among Creative Segments on

the three-level branches
Hypothesis 3 is partly supported by the results. Creative Segments on main
branches are critical during sketching. Participants referred to more prior Cre-
ative Segments to generate them, Creative Segment on main branches also

608 Design Studies Vol 35 No. 6 November 2014

 attracted fixation from more following sketching activities and posterior Cre-
ative Segments.

The intermittently developed Creative Segments on function branches have

extensive searching. Participants developed functions for one goal, and turned
back to this after exploring other goals; they scanned both areas of Creative
Segments on the same branch and those on different branches. In contrast,
participants sketching for structures of one function employed continuous
exploring process. They were more concentrated, only scanning areas of Cre-
ative Segments along the sub branches.

Participants displayed different eye movement patterns for Creative Segments

on the three-level branches. The first solutions for new goals need more refer-
ence to other Creative Segments and act as important references in following
sketching. The subsequent development of functions means that the attention
to a wide range of Creative Segments could be quite supportive. When produc-
ing structural evolutions of one Creative Segment, participants preferred to
focus on optimization of the Creative Segment rather than considering the
design factors thoroughly.

5.4 General discussion

In general, the experimental results validated the Creative Segment theory.
These results revealed the unique eye movement features during Creative Seg-
ments, the exploration between Creative Segments and the recheck before Cre-
ative Segments, and the three kinds of sketch perception when generating
Creative Segments of the three-level branches. The perception during sketch-
ing is a Creative Segment-related process.

These discoveries also offer possible methods to improve sketching quality.

The exploratory items should be emphasized when referring to others’
sketches. Designers attended to different items when generating goals, func-
tions, and structures; they could accordingly acquire information via other re-
sources to support their design. Novice designers might be instructed to search
widely when stuck in the function design, and concentrate on the structure to
modify every bug successively. For example, designers exploring functions for
communication should also seek functions for being quiet, they might find that
quiet circumstances provide security to talk freely and enhance communica-
tion. While exploring structures for a function, just searching other structures
for the same function will be more proper.

6 Conclusion
This article employed eye-tracking techniques and analyzed designers’ percep-
tion during sketching to examine the Creative Segment theory. It arranged the
sketching process as a Creative Segment tree, compared the eye movements
during Creative Segments with those during other intervals, examined

Designers’ perception during sketching 609

 fixations in the growing process of Creative Segment tree, and compared the
eye movements among Creative Segments on the three-level branches.

These analyses validated Creative Segment theory as a suitable model of the

perception during sketching. Participants were more concentrated in the Cre-
ative Segment, and regularly looked at areas of prior Creative Segments in
subsequent activities. The eye movements were influenced by the three-level
branches of Creative Segment tree. The Creative Segments proposing new
goals were critical. The Creative Segments exploring functions of one goal
used a wider searching strategy, and those exploring structures of one function
were more focused.

The use of eye movements requires cautious interpretation on designers’

perception. This paper only validated that the perception during sketching is
Creative Segment-related. Future work will try to build the connection be-
tween eye movements and the inner processing, and study designers’ informa-
tion processing based on the Creative Segment theory.

Acknowledgements
This paper is supported by the National Natural Science Foundation of China
(61004116, 61272308), the Zhejiang Provincial Natural Science Foundation of
China (LY13E050005, LY13F030002), and the Science & Technology Project
of Zhejiang (2012C21066).

References
Ballard, D. H., Hayhoe, M. M., & Pelz, J. B. (1995). Memory representations in
natural tasks. Journal of Cognitive Neuroscience, 7, 66e80.
Causse, M., Senard, J.-M., Demonet, J. F., & Pastor, J. (2010). Monitoring cogni-
tive and emotional processes through pupil and cardiac response during dy-
namic versus logical task. Applied Psychophysiology and Biofeedback, 35.
Cross, N. (2004). Expertise in design: an overview. Design Studies, 25, 427e441.
Cross, N. (2006). Designerly ways of knowing (1st ed.). London, UK: Springer.
Deng, Y. (2002). Function and behavior representation in conceptual mechanical
design. AI EDAM, 16(5), 343e362.
Dorst, K., & Cross, N. (2001). Creativity in the design process: co-evolution of
problemesolution. Design Studies, 22, 425e437.
Duchowski, A. (2007)Eye tracking methodology: Theory and practice, Vol. 373.
Springer.
Ellis, J. J., Glaholt, M. G., & Reingold, E. M. (2011). Eye movements reveal so-
lution knowledge prior to insight. Consciousness and Cognition, 20, 768e776.
Epelboim, J., & Suppes, P. (2001). A model of eye movements and visual working
memory during problem solving in geometry. Vision Research, 41, 1561e1574.
Eppler, M. J., & Pfister, R. (2011). Sketching as a tool for knowledge manage-
ment: an interdisciplinary literature review on its benefits. In Proceedings of
the 11th international conference on knowledge management and knowledge tech-
nologies (pp. 1e6). Graz, Austria: ACM.
Findlay, J. M., & Gilchrist, I. D. (2003). Active vision: The psychology of looking
and seeing. Oxford University Press.

610 Design Studies Vol 35 No. 6 November 2014

 Gero, J. S. (1990). Design prototypes. A knowledge representation schema for
design. AI Magazine, 11, 26e36.
Goel, A. K. (1997). Design, analogy, and creativity. IEEE Expert, 12, 62e70.
Goldschmidt, G. (1991). The dialectics of sketching. Creativity Research Journal,
4, 123e143.
Goldschmidt, G. (1997). Capturing indeterminism: representation in the design
problem space. Design Studies, 18, 441e455.
Groen, M., & Noyes, J. (2010). Solving problems: how can guidance concerning
task-relevancy be provided? Computers in Human Behavior, 26, 1318e1326.
Hodgson, T. L., Tiesman, B., Owen, A. M., & Kennard, C. (2002). Abnormal
gaze strategies during problem solving in Parkinson’s disease. Neuropsycholo-
gia, 40, 411e422.
Hubel, D. H. (1995). Eye, brain, and vision. Henry Holt and Company.
Jin, Y., & Benami, O. (2010). Creative patterns and stimulation in conceptual
design. AI EDAM e Artificial Intelligence for Engineering Design Analysis
and Manufacturing, 24, 191e209.
Jin, Y., & Chusilp, P. (2006). Study of mental iteration in different design situa-
tions. Design Studies, 27, 25e55.
Johansson, R. S., Westling, G., Backstrom, A., & Flanagan, J. R. (2001). Eye-
hand coordination in object manipulation. Journal of Neuroscience, 21,
6917e6932.
Jowers, I., Prats, M., McKay, A., & Garner, S. (2013). Evaluating an eye tracking
interface for a two-dimensional sketch editor. Computer-Aided Design, 45,
923e936.
Kan, I. P., Barsalou, L. W., Solomon, K. O., Minor, J. K., & Thompson-
Schill, S. L. (2003). Role of mental imagery in a property verification task:
fMRI evidence for perceptual representations of conceptual knowledge. Cogni-
tive Neuropsychology, 20, 525e540.
Kavakli, M., & Gero, J. S. (2002). The structure of concurrent cognitive actions: a
case study on novice and expert designers. Design Studies, 23, 25e40.
Kavakli, M., Scrivener, S. A. R., & Ball, L. J. (1998). Structure in idea sketching
behaviour. Design Studies, 19, 485e517.
Litchfield, D., & Ball, L. J. (2011). Using another’s gaze as an explicit aid to
insight problem solving. Quarterly Journal of Experimental Psychology, 64,
649e656.
van der Lugt, R. (2000). Developing a graphic tool for creative problem solving in
design groups. Design Studies, 21, 505e522.
Menezes, A., & Lawson, B. (2006). How designers perceive sketches. Design
Studies, 27, 571e585.
Miall, R. C., & Tchalenko, J. (2001). A painter’s eye movements: a study of eye
and hand movement during portrait drawing. Leonardo, 34, 35e40.
Minassian, A., Granholm, E., Verney, S., & Perry, W. (2004). Pupillary dilation to
simple vs. complex tasks and its relationship to thought disturbance in schizo-
phrenia patients. International Journal of Psychophysiology, 52.
Nakakoji, K., Yamamoto, Y., Takada, S., & Reeves, B. N. (2000). Two-dimen-
sional spatial positioning as a means for reflection in design. In Proceedings
of the 3rd conference on designing interactive systems: Processes, practices,
methods, and techniques (pp. 145e154). ACM.
Nuthmann, A., Smith, T. J., Engbert, R., & Henderson, J. M. (2010). CRISP: a
computational model of fixation durations in scene viewing. Psychological Re-
view, 117.

Designers’ perception during sketching 611

 Ohsawa, Y., Maeda, Y., Yoshida, T., & IEEE. (2007). Eyes draw auxiliary lines
before insight moment. In IEEE international conference on systems, man and
cybernetics (pp. 3470e3475). Montreal, Cook Islands: IEEE.
Oxman, R. (2002). The thinking eye: visual re-cognition in design emergence.
Design Studies, 23, 135e164.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental
Psychology, 32, 3e25.
Prats, M., Garner, S., Jowers, I., McKay, A., & Pedreira, N. (2010). Interpreta-
tion of geometric shapes: an eye movement study. In Proceedings of the 2010
symposium on eye-tracking research & applications (pp. 243e250). Austin,
Texas: ACM.
Rayner, K. (2009). Eye movements and attention in reading, scene perception,
and visual search. The Quarterly Journal of Experimental Psychology, 62,
1457e1506.
Rensink, R. A. (2002). Change detection. InFiske, S T., Schacter, D. L., & Zahn-
Waxler, C. (Eds.). (2002). Annual review of psychology, Vol. 53 (pp. 245e277).
Restrepo, J. C., & Henri. (2004). Problem structuring and information access in
design. Journal of Design Research, 4.
Rodgers, P. A., Green, G., & McGown, A. (2000). Using concept sketches to
track design progress. Design Studies, 21, 451e464.
R€omer, A., Pache, M., Weißhahn, G., Lindemann, U., & Hacker, W. (2001).
Effort-saving product representations in designdresults of a questionnaire sur-
vey. Design Studies, 22, 473e491.
Sch€utz, A. C., Braun, D. I., & Gegenfurtner, K. R. (2011). Eye movements and
perception: a selective review. Journal of Vision, 11.
Schutze, M., Sachse, P., & Romer, A. (2003). Support value of sketching in the
design process. Research in Engineering Design-Theory Applications and Con-
current Engineering, 14, 89e97.
Shah, J. J., Smith, S. M., & Vargas-Hernandez, N. (2003). Metrics for measuring
ideation effectiveness. Design Studies, 24, 111e134.
Sim, S. K., & Duffy, A. H. B. (2003). Towards an ontology of generic engineering
design activities. Research in Engineering Design-Theory Applications and Con-
current Engineering, 14, 200e223.
Sun, L., Xiang, W., Chai, C., Wang, C., & Huang, Q. (2014). Creative segment: a
descriptive theory applied to computer-aided sketching. Design Studies, 35,
54e79.
Suwa, M., Gero, J. S., & Purcell, T. A. (1998). The roles of sketches in early con-
ceptual design processes. In M. A. Gernsbacher, & S. J. Derry (Eds.), Proceed-
ings of the twentieth annual conference of the Cognitive Science Society (pp.
1043e1048).
Suwa, M., Gero, J., & Purcell, T. (1999). Unexpected discoveries: How designers
discover hidden features in sketches. MA: Visual and Spatial Reasoning in
Design Key Centre of Design Computing and Cognition, University of
Sydney.
Suwa, M., Purcell, T., & Gero, J. (1998). Macroscopic analysis of design processes
based on a scheme for coding designers’ cognitive actions. Design Studies, 19,
455e483.
Taura, T., Yamamoto, E., Fasiha, M. Y. N., Goka, M., Mukai, F., Nagai, Y.,
et al. (2011). Constructive simulation of creative concept generation process
in design: a research method for difficult-to-observe design-thinking processes.
Journal of Engineering Design, 23, 297e321.
Temple, S. (1994). Thought made visible: the value of sketching. Co-Design Jour-
nal 16e25.

612 Design Studies Vol 35 No. 6 November 2014

 Thomas, L. E., & Lleras, A. (2007). Moving eyes and moving thought: on the
spatial compatibility between eye movements and cognition. Psychonomic
Bulletin & Review, 14, 663e668.
Wiegers, T., Langeveld, L., & Vergeest, J. (2011). Shape language: how people
describe shapes and shape operations. Design Studies, 32, 333e347.

Designers’ perception during sketching 613