Food Research International 48 (2012) 893–908

Contents lists available at SciVerse ScienceDirect

Food Research International

journal homepage: www.elsevier.com/locate/foodres

Review

Sensory profiling, the blurred line between sensory and consumer science. A review
of novel methods for product characterization
Paula Varela a,⁎, Gastón Ares b
a
Instituto de Agroquímica y Tecnología de Alimentos (CSIC), Avda. Agustín Escardino, 7. 46980 Paterna (Valencia), Spain
b
Departamento de Ciencia y Tecnología de Alimentos, Facultad de Química, Universidad de la República, General Flores 2124, C.P. 11800, Montevideo, Uruguay

a r t i c l e i n f o a b s t r a c t

Article history: Sensory descriptive analysis is one of the most powerful, sophisticated and most extensively used tools in
Received 27 April 2012 sensory science, which provides a complete description of the sensory characteristics of food products. Con-
Accepted 29 June 2012 sidering the economic and time consuming aspects of training assessor panels for descriptive analysis, several
novel methodologies for sensory characterization have been developed in the last ten years. These method-
Keywords:
ologies are less time consuming, more flexible and can be used with semi trained assessors and even con-
Sensory descriptive analysis
Consumer profiling
sumers, providing sensory maps very close to a classic descriptive analysis with highly trained panels.
QDA® Novel techniques are based on different approaches: methods based on the evaluation of individual attri-
Napping® butes (intensity scales, check-all-that-apply questions or CATA, flash profiling, paired comparisons); methods
Sorting based on the evaluation of global differences (sorting, projective mapping or Napping®); methods based on
Flash profiling the comparison with product references (polarized sensory positioning), and based on a free, global evalua-
Free choice profiling tion of the individual products (Open-ended questions). This review aims at reviewing theory, implementa-
Repertory grid tion, advantages and disadvantages of the novel product profiling techniques developed in the last ten years,
CATA
discussing recommendations for their application.
Open-ended questions
© 2012 Elsevier Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894
1.1. Sensory characterization, from classical descriptive analysis to the emergence of novel profiling techniques . . . . . . . . . . . . . 894
2. Novel methods for product characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
2.1. Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
2.1.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
2.1.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
2.1.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
2.1.4. Modifications to the original methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
2.2. Flash profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
2.2.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
2.2.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897
2.2.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897
2.3. Projective mapping and Napping® . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 898
2.3.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 898
2.3.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899
2.3.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899
2.3.4. Modifications to the original methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899
2.4. Check-all-that-apply questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900
2.4.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900
2.4.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900
2.4.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 901

⁎ Corresponding author. Tel.: +34 963 900 022; fax: +34 963 636 301.
E-mail address: pvarela@iata.csic.es (P. Varela).

0963-9969/$ – see front matter © 2012 Elsevier Ltd. All rights reserved.
doi:10.1016/j.foodres.2012.06.037
894 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

2.5. Other methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 902

2.5.1. Intensity scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 902
2.5.2. Open-ended questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 902
2.5.3. Preferred attribute elicitation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903
2.5.4. Polarized sensory positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903
2.5.5. Paired comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903
2.6. Comparison of the methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903
3. Conclusions and recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 906

1. Introduction
Sensory descriptive analysis is one of the most powerful, sophisticat-
ed and most extensively used tools in sensory science. Its application has
steadily grown in the end of the 20th century and the beginning of the
21st. This methodology enables to measure the sensory reaction to the
stimuli resulting from the consumption of a product, providing a descrip-
tion of the qualitative and quantitative aspects of human perception, and
allowing correlations to other parameters (Lawless & Heymann, 2010;
Moussaoui & Varela, 2010; Murray, Delahunty, & Baxter, 2001; Stone &
Sidel, 2004). Describing the sensory characteristics of a product has
been common practice in the food and beverage industry since long
ago, allowing informed business decisions, guiding product development
to match a consumers' ideal, to get closer to a benchmark, to check the
effect of ingredients or processes, for quality control purposes, to track
product changes over time, and to correlate with instrumental measure-
ments. In academic research it has been a valuable resource as well, en-
abling the establishment of correlations with analytical measurements,
helping to explain how changes in texture, flavor, aroma or structural
and microstructural features determine different sensory characters,
and allowing to better understand the mechanisms underlying sensory
perception (Gacula, 1997; Moussaoui & Varela, 2010; Stone & Sidel,
2004). Importantly, both in industrial and in academic fields, descriptive
analysis has served as a link between product characteristics and con-
sumer reaction.
1.1. Sensory characterization, from classical descriptive analysis to the
emergence of novel profiling techniques
There are various ways to perform classical sensory descriptive
analysis in practice, and there is plenty of literature extensively
reviewing them in the last years. Murray et al. (2001) have published
one of the latest reviews on classical methods of descriptive analysis,
mentioning the Flavor Profile Method (Cairncross & Sjöstrom, 1950),
the Texture Profile Method (Brandt, Skinner, & Coleman, 1963), Quan-
titative Descriptive Analysis™ — QDA™ (Stone, Sidel, Oliver, Woolsey,
& Singleton, 1974), the Spectrum™ method (Meilgaard, Civille, &
Carr, 1991), Quantitative Flavor Profiling (Stampanoni, 1993a, 1993b)
and what they called a “generic descriptive analysis”, which is general-
ly a mixed approach taking ideas from some of these methods. The
most utilized generic descriptive analysis technique comprises a
combination of the basic elements of QDA™ and Spectrum™
(Lawless & Heymann, 2010), to make it more flexible and allowing
meeting project specific objectives. As a summary, it requires as a
first step the selection, training and maintenance of a panel of
8–20 assessors. Once the assessor panel is selected for each project
or sample group it would: (a) generate specific attributes that de-
scribe the similarities and differences between products,
(b) determine and agree on the evaluation procedure for each of the
selected attributes, (c) be trained in the evaluation and scaling of the
selected attributes for the particular sample set, and (d) would fi-
nally quantitatively evaluate the samples, generally with the use of
10–15 cm unstructured line scales, where samples would be evaluated
individually in a sequential monadic, balanced randomized presentation.
The obtained data would be in the form of intensity scores of all the
attributes, which can be analyzed individually, by attribute and sample
as a sensory signature or profile of each product. It is common practice
to look at descriptive data through sensory mapping, reducing the num-
ber of variables, and obtaining one or more biplots representing the per-
ceptual space of interest. This space provides a representation of the
samples according to the similarities and differences in the intensity
of the evaluated sensory attributes.
The high specialization of descriptive panels allows obtaining very
detailed, robust and consistent, reproducible results, stable in time
and within a certain sensory space (Moussaoui & Varela, 2010). Cre-
ating and maintaining a well-trained, calibrated sensory panel can
be quite expensive, though; small food companies usually cannot af-
ford it, and it could even mean a significant expenditure for big com-
panies if they have a wide range of products that require various
panels working in parallel. Furthermore, the training step can be rel-
atively long, as it should be detailed and extensive, varying between
10 and 120 h, depending on the range and complexity of the sample
set, which might result in a time constraint, particularly when indus-
tries require quick responses to market (Lawless & Heymann, 2010;
Murray et al., 2001). The high economic and time consuming aspects
of having a trained descriptive panel could be a problem in academic
research as well, when a short project does not justify the training of a
panel from scratch, or the lack of funding does not allow it.
All things considered, it was natural that sensory science would
transition, at certain point, to less time consuming, more rapid senso-
ry methods, that would be more flexible and give extra agility to sen-
sory description, both in terms of timing and training requirements.
And that is exactly what happened, starting with the development of
free choice profiling (FCP) and repertory grid (RG) methods in the eight-
ies, which could be used with non-trained assessors (Thomson &
Mcewan, 1988; Williams & Arnold, 1985). In FCP, consumers develop
their own attributes to describe the products, with their own vocabulary
and in any number, limited only by their sensory skills, they then quan-
tify their personal attributes using line scales; the method is based
on the assumption that panelists do not differ in their perceptions but
solely in the to describe them (Murray et al., 2001). The development
of Generalized Procrustes Analysis (GPA) as statistical tool (Gower,
1975) allowed the possibility of analyzing the data coming from data
sets which differed in the number of attributes per consumer and also
having differences in the use of the scale. The RG method quantifies
products in the same way, but the step of development of the attributes
is done by constructs generation using triads of products through a ver-
sion of Kelly's repertory grid (Kelly, 1955). The emergence of these two
methods opened the way to the use of consumers for product sensory
description, with the realization that by allowing panelists to select
their own attributes it was possible to identify characteristics (which
may not have been considered using the traditional approach), together
with the economization of time and resources.
Development of descriptive techniques continued since the eight-
ies to our days with an array of different methods of sensory charac-
terization, developed in the last ten years, which can be used with
semi trained assessors (i.e. trained in sensory recognition and charac-
terization but not in the specific category of products or in scaling)
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
and even naïve consumers, with a great success, obtaining sensory
maps very close to a classic descriptive analysis with highly trained
panels. Those novel techniques are, namely: sorting (Lawless, Sheng, &
Knoops, 1995; Schiffman, Reynolds, & Young, 1981), flash profiling
(Dairou & Sieffermann, 2002), projective mapping or Napping® (Pagès,
2005; Risvik, McEvan, Colwill, Rogers, & Lyon, 1994), Check-all-
that-applies (CATA) questions (Adams, Williams, Lancaster, & Foley,
2007), and other techniques less frequently used for sensory profiling
(use of intensity scales with consumers, evaluation of open-ended
questions, paired comparisons) or still in early development at this
point in time as polarized sensory positioning (PSP) (Teillet, Schlich,
Urbano, Cordelle, & Guichard, 2010) and preferred attribute elicitation
method (Grygorczyk, Lesschaeve, Corredig, & Duizer, in press). In gen-
eral, novel techniques are based on different approaches: in the line
of conventional profiling or free choice profiling there are methods
based on the evaluation of individual attributes (intensity scales,
CATA, flash profiling, paired comparisons); methods based on the eval-
uation of global differences (sorting, Napping®); methods based on
the comparison with product references (PSP), and based on a free,
global evaluation of the individual products (open-ended questions).
Conventional descriptive analysis nevertheless, has not been
substituted by novel profiling methods, as trained panel descriptive
measurements usually perform better in various cases, for example
when there is a need to compare samples in different moments in
time, when comparing different sample sets with a few samples in
common, or when a very detailed sensory description is required.
The new tools emerged, in fact, as complementary tools to sensory
and consumer science, as they can be applied to gather product de-
scriptions directly from consumers, with the added benefit of having
direct feedback from them, and sometimes with their own vocabulary
(Moussaoui & Varela, 2010).
The hypothesis that consumers are able to accurately describe
products is more and more accepted within the sensory science com-
munity and diverse product profiling methods are used as never be-
fore in the food industry: the line between sensory and consumer
science is becoming blurred. This paper aims at reviewing theory, im-
plementation, advantages and disadvantages of the novel product
profiling techniques developed in the last ten years.
2. Novel methods for product characterization
2.1. Sorting
2.1.1. Theory and implementation
Classification, i.e. putting a group of things into categories
according to an established criterion, is one of the most common op-
erations in thinking (Coxon, 1999). In the context of social sciences,
the process by which a person classifies objects is called sorting
(Coxon, 1999). Sorting has been extensively used as a systematic
method for data collection in psychology, anthropology and sociolo-
gy (Coxon, Davies, & Jones, 1986; Miller, Wiley & Wolfe, 1986). The
aim of the technique is to study how people classifies objects,
which provides information of how people perceive the objects and
what characteristics they attend to when making classification be-
tween a series of objects (Black, 1963).
Sorting tasks have also been used to get information about
the sensory characteristics of food products in sensory and consumer
science (Lawless et al., 1995; Schiffman et al., 1981). The aim of the
methodology is to measure the global degree of similarity between
samples by sorting them into groups. Assessors are asked to try the
whole set of samples and to sort them into groups according to
their similarities and differences, using their own personal criteria.
Assessors are told that two samples which are perceived as similar
should be placed in the same group, whereas two samples that are
markedly different should be placed in different groups. In order to
avoid trivial answers, assessors are usually told that they should
895
sort the samples in at least two groups and that they could not have
only one sample in all groups. In order to gather information about
the sensory characteristics of the samples which are responsible
for the similarities and differences between the samples, once the
sorting has been completed, assessors are asked to provide descrip-
tive words for each of the groups they formed (Lawless et al., 1995;
Popper & Heymann, 1996). A typical classification provided by an as-
sessor in a sorting task is shown in Fig. 1.
When assessors are not trained they might find it difficult to
provide a description of the sensory characteristics of each group of
samples. Therefore, in order to make the description phase easier,
Lelièvre, Chollet, Abdi, and Valentin (2008) provided to the assessors
a list of pre-defined sensory characteristics from which they could
select those they consider appropriate to describe the samples.
The number of assessors used in sorting tasks depends on their
training: when working with trained assessors the usual number ranges
from 9 to 15 (Cartier et al., 2006; Chollet, Lelièvre, Abdi, & Valentin,
2011), whereas when untrained assessors or naive consumers are con-
sidered the number of assessors ranges from 9 to 98 (Chollet et al.,
2011; Cadoret, Lê, & Pagès, 2009). Despite the variability in the number
of untrained assessors considered in sorting tasks, most studies work
with 20–50 (Ares, Varela, Rado, & Gimenez, 2011a; Cartier et al.,
2006; Falahee & MacRae, 1997; Moussaoui & Varela, 2010).
2.1.2. Data analysis
Data analysis of sorting tasks aims at providing a spatial map that
represents the similarities and differences between samples in terms
of their sensory characteristics (Lawless et al., 1995). In this map,
the distance between a pair of samples is related to their degree of
difference, which means that two samples which are represented
close to each other are similar, whereas two samples which are repre-
sented far from each other correspond to dissimilar products. Differ-
ent approaches have been proposed for analyzing data and to get a
sample map.
The most common approach for analyzing data from sorting tasks
is Multidimensional Scaling (Lawless et al., 1995). When using this
approach a similarity matrix is created by counting the number of
times that each pair of samples is sorted within the same group, as
shown in Fig. 2. Non-metric or metric Multidimensional Scaling (MDS)
is carried out on this similarity matrix in order to get a 2-dimensional
representation of the samples. A typical sample representation from
MDS is shown in Fig. 3.
The main disadvantage of MDS is that information about differ-
ences in the perception of the individual assessors is lost since the
similarity matrix computes the similarity and differences between
samples for the entire group of assessors (Lawless et al., 1995). There-
fore, it is not possible to determine if assessors sorted the samples sim-
ilarly or if they had different perceptions and used different criteria
to group them. Abdi, Valentin, Chollet, and Chrea (2007) proposed
the application of a different statistical technique, called DISTATIS, to
Fig. 1. Example of the response provided of an assessor to a free sorting task with 7
samples of milk custards.
896 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Fig. 2. Example of a similarity matrix for analyzing data from a free sorting task using
Multidimensional Scaling. Each cell indicates the number of times that each pair of
samples was placed together within the same group in the free sorting task.
overcome this limitation. This technique allows the analysis of 3-way
distance tables and takes into account the sample grouping provided
by each assessor. DISTATIS first analyzes the individual co-occurrences
matrices of the participants, providing an optimal representation of
the assessors based on their resemblance. Then, a diagonalization of
the linear combination of individual matrices is performed to provide
a consensus sample representation. Finally, the words used by the
assessors to describe the groups are projected by using barycentric
properties.
Cadoret et al. (2009) proposed the application of FAST for analyz-
ing sorting data. This approach provides an optimal representation of
the samples based on Multiple Correspondence Analysis (MCA), and
an optimal representation of the assessors based on Multiple Factor
Analysis (MFA). In this technique all the assessors have the same im-
portance when constructing the sample map. An example of the sam-
ple representation provided by FAST on data from a sorting task is
shown in Fig. 4.
The main advantage of DISTATIS and FAST is that they provide a
representation of the assessors, which enables the visualization of
individual differences. Besides, by applying these techniques, the
words used by the assessors to describe the samples could easily be
projected into the sample space, which improves interpretation and
provides more actionable results.
Within the novel approaches for sensory characterization of food
products, sorting is one of the most popular approaches.
Fig. 3. Typical sample representation of data from a free sorting task of 7 orange juice
samples (named A to G) with 50 consumers using Multidimensional Scaling (MDS).
2.1.3. Application, advantages and limitations
This methodology has been applied to a wide range of products with
different sensory complexities, including cheese (Lawless et al., 1995),
drinking water (Falahee & MacRae, 1997), beer (Chollet & Valentin,
2001), wine (Gawel, Iland, & Francis, 2001), yogurt (Saint-Eve, Paài
Kora, & Martin, 2004), breakfast cereals (Cartier et al., 2006), olive oil
(Santosa, Abdi, & Guinard, 2010), coffee (Moussaoui & Varela, 2010)
and orange-flavored powdered drinks (Ares et al., 2011a).
Free sorting has several advantages which makes it an interesting
methodology for sensory characterization. Firstly, it corresponds to
natural and common mental activity, being an easy and enjoyable
task for participants (Coxon, 1999). Besides, it does not require exten-
sive training and produces little fatigue and boredom, which makes it
appropriate for both trained assessors and consumers (Bijmolt &
Wedel, 1995). As suggested by Cartier et al. (2006) another advantage
of the methodology is that it does not require the use of scales or
other quantitative systems.
Although sorting tasks can be applied to a large sample set it is im-
portant to take into account that all samples should be presented simul-
taneously in a single session. Thus, when dealing with complex and
fatiguing products, the number of products to be evaluated should be
limited. Furthermore, it should be highlighted that when sorting tasks
are performed by untrained assessors, the descriptions provided could
be difficult to interpret in order to get actionable information.
2.1.4. Modifications to the original methodology
Two modifications to the original methodology have been recently
proposed. Santosa et al. (2010) applied a modified sorting task to in-
vestigate consumer perception of extra virgin olive oils. These authors
asked consumers to perform a two-stage sorting task to encourage
them to further discriminate between samples, after participants fin-
ished sorting samples into groups, they asked them to complete a sec-
ond sorting task, which consisted of sorting samples within each
group according to their similarities and differences. Data was analyzed
using DISTATIS considering the hierarchical nature of data from the first
and the second task.
In order to get information about the structure of the groups formed
by each assessor, Courcoux, Qannari, Taylor, Buck, and Greenhoff
(2012) proposed the application of taxonomic sorting task. In this ap-
proach, after assessors have completed the first sorting task, they are
asked to organize the groups into a hierarchical structure. In this last
step, assessors are instructed to put together the two groups that they
think are most similar. This step is repeated until all groups are reduced
to one, yielding a hierarchical structure similar to that shown in Fig. 5.
Data analysis is performed using a non-metric MDS on an averaged dis-
similarity matrix (Courcoux et al., 2012).
2.2. Flash profiling
2.2.1. Theory and implementation
Dairou and Sieffermann (2002) suggested the use of flash profiling
(FP) for sensory description, developed as a variant of free choice
profiling. It was defined as a combination of FCP with a comparative
evaluation of the products via ranking, based on the simultaneous
presentation of the whole sample set. FP is a flexible method meant
to rapidly profile products according to their most salient sensory
attributes. It has proven to be as satisfactory as conventional profiling
in many applications (Dairou & Sieffermann, 2002).
FP can be done in two sessions, or in one session with two steps. In
practice, coded samples are presented all together. In a first step con-
sumers have to taste them comparatively in order to generate all
descriptors that they consider appropriate to discriminate between
the samples. In a second step, they rank all samples from “low” to
“high” on each selected attribute, where ties are allowed. As in FCP,
each consumer generates his/her own set of attributes; no indication
is given regarding the number of attributes (Dairou & Sieffermann,
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908 897

Fig. 4. Representation of 7 orange juice samples and descriptive terms from a free sorting task with 50 consumers using FAST.

2002; Delarue & Sieffermann, 2004; Lassoued, Delarue, Launay, &
Michon, 2008; Moussaoui & Varela, 2010). Fig. 6 shows an example
of how a ballot of one consumer would look like.
When products are tested blind, and the samples permit (not
easily recognizable by shape, color, etc.), it is usual to include a repeated
blind control within the sample set to examine individual assessor
performance, subsequently checking in the final perceptual map that
sample and blind control are close (Ferrage, Nicod, & Varela, 2010).
Another possibility would be repeating the whole evaluation with
the same assessors, but this is more difficult when the number of con-
sumers is considerable.
The simultaneous comparison of all the samples could allow bet-
ter product discrimination. Furthermore, when the tested products
belong to the same or to similar product categories, flash profiling
can be more discriminating than conventional profiling (Delarue &
Sieffermann, 2004; Mazzucheli & Guinard, 1999).
Flash profile has been carried out with trained panels of 6 to 12 trained
or semi-trained assessors (Albert, Varela, Salvador, Hough, & Fiszman,
2011; Dairou & Sieffermann, 2002; Delarue & Sieffermann, 2004;
Moussaoui & Varela, 2010; Tarea, Cuvelier, & Siefffermann, 2007) and
with consumer panels of 20 to 40 participants (Lassoued et al., 2008;
Moussaoui & Varela, 2010; Veinand, Godefroy, Adam, & Delarue, 2011).
2.2.2. Data analysis
The analysis is based on ranking data, Fig. 7a shows how data is
computed, with ties; an easy way of verifying the ranks are entered
correctly is the sum of ranks, as the sum is the same for all attributes
and all assessors, as it depends only on the number of products.
For data collection, individual matrices are built for each consumer
(products in rows × attributes in columns), the table is structured so
that each consumer has a table with his/her own attributes, were
product rankings are inputted (Fig. 7b).
A Generalized Procrustes Analysis (GPA) is run on all the matrices
in order to obtain the product and attribute configurations. GPA de-
livers a consensus configuration and provides, like in PCA, a product
biplot and an attribute plot. In the case of the attributes, consensus
comes from the usage of the same/similar attributes by different asses-
sors, i.e. “sweet”, “sweetness” and “sugary”, etc. (Moussaoui & Varela,
2010). Hierarchical Cluster Analysis (HCA) can be applied to identify
clusters of attributes that are correlated (Lassoued et al., 2008).
Ferrage et al. (2010) put to discussion the topic of how to check
panel performance in novel profiling methods, particularly because
of the use of consumers, and proposed a method to detect bad per-
formers in a FP test, with the use of a blind control, and HCA analysis
per consumer. Also, Veinand et al. (2011) proposed the use of HCA on
the FP product consensus configuration to check the clustering of the
blind control as a performance measure.
2.2.3. Application, advantages and limitations
Flash profile has been applied to describe different foods, includ-
ing jams (Dairou & Sieffermann, 2002), dairy products (Delarue &
Sieffermann, 2004), commercial apple and pear purees (Tarea et al.,

Fig. 5. Typical hierarchical structure obtained from a taxonomic sorting task of an assessor asked to sort 8 samples.
898 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Fig. 6. Example of a ballot for one assessor describing 7 yogurt samples via FP, ranking 3
attributes from “low” to “high”. “Pete” ranked some samples in the same position when
assessing “sweet flavor” (D and E) and “grainy” (C, E, F), as ties are allowed.
2007), flavor perception of bread odor (Poinot et al., 2007), jellies
(Blancher et al., 2007), bread (Lassoued et al., 2008), wines (Perrin
et al., 2008), hot beverages (Moussaoui & Varela, 2010), lemon iced
teas (Veinand et al., 2011) and fish nuggets (Albert et al., 2011).
Also, it has been successfully applied for non-food products as concert
hall acoustics (Lokki, Pätynen, Kuusinen, Vertanen, & Tervo, 2011).
FP is a rapid sensory mapping tool, easy to comprehend for con-
sumers, suited for applications when a quick response is needed but
also as initial screening tool for a new product set or category and
to study a specific market (Dairou & Sieffermann, 2002; Delarue &
Sieffermann, 2004; Tarea et al., 2007).
A limitation of FP, being a comparative method, is that the number
of samples that can be assessed is limited, and it would depend on the
product category. However, Tarea et al. (2007), reported successfully
describing via FP, 49 samples of apple and pear purees, in one session
Fig. 7. (a) Schematic view of how to collect FP data for one attribute. Tied attributes share the same rank, in the examples D and E were ranked together between the 3rd and the 6th
sample, so they share position 4.5 ((4 + 5) / 2 = 4.5); (b) structure of the data matrix for FP calculations, ranks are inputted for each of the attributes selected for the consumers, note
that the number of attributes usually differ between consumers (n ≠ m ≠ p).
lasting between 2 and 5 h, where assessors could take breaks; it is
noteworthy though, that they were 6 trained, motivated assessors.
Another drawback of FP is that each assessor has his/her own
attribute list, so the semantic interpretation can be complex (Dairou
& Sieffermann, 2002; Veinand et al., 2011). However, it has been
proved that even if this method produces a large amount of varied
attributes, the core attributes for the description of the samples set
are well covered when using FP even with consumers speaking differ-
ent languages, what makes the method especially suited for cross
country comparisons (Moussaoui & Varela, 2010). Within the same
line, consumers often use hedonics or benefit-related terms together
with sensory attributes; this fact can be seen as a limitation because
it complicates the analysis (Veinand et al., 2011), nevertheless, this
information could be interesting to relate product characteristics to
marketable features and consumer preference.
2.3. Projective mapping and Napping®
2.3.1. Theory and implementation
Projective mapping, and its special case Napping®, are projective-
type methods that collect bi-dimensional perceptual maps for each
assessor in a single sensory session. There were originally derived
from psychology and previously used in qualitative market research,
to obtain associations between products (Pagès, 2005; Risvik et al.,
1994). Risvik et al. (1994) proposed to use it with consumers, and
linked the results to trained panel data to explain product description.
In projective mapping, samples are simultaneously presented, to
be positioned by each assessor on a bi-dimensional space as a “table-
cloth” (“nappe” in French, which gave the name to Napping®), or more
often in an A4 or A3 blank paper. Samples are arranged by the partici-
pant according to the differences and similarities between them in
such a way that the more similar they are, the closer would be two
samples on the sheet (Perrin et al., 2008). Assessors are asked to ob-
serve, smell and taste the samples and to position them according to
differences/similarities. The positioning criteria and their importance
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
are chosen individually by each assessor, which makes projective
mapping a flexible and spontaneous procedure. After positioning the
samples, the participants are sometimes asked to write down com-
ments to describe samples or groups of samples, the reasons behind
their placement, method generally known as “Ultra Flash Profiling”
(Pagès, 2003; Perrin et al., 2008). When the exercise is blind, and the
samples allow (not easily recognizable by shape, color, etc.), it is com-
mon practice to add a repeated blind control sample in the set, to
check assessor performance, subsequently checking in the final percep-
tual map that sample and blind control are close (Ferrage et al., 2010).
Fig. 8 shows an example of how the configuration of one of the assessors
would look like. Via multivariate statistical analysis of the data, all
the individual maps are collated in a consensus configuration that,
together with the comments, would determine the sensory profile of
the sample set, in terms of distances and descriptions, as perceived by
the group of assessors.
When working with trained panels projective mapping has been
performed with 9 to 15 assessor, as in traditional descriptive analysis
(Perrin et al., 2008; Risvik et al., 1994), whereas when untrained asses-
sors or consumers complete this type of task the number has been in-
creased to 15–50 (Albert et al., 2011; Ares, Deliza, Barreiro, Giménez,
& Gámbaro, 2010; Nestrud & Lawless, 2008).
2.3.2. Data analysis
For data collection, the coordinates of the location of the products
are measured for each consumer (Fig. 9a); also, the comments given
for each of the samples are counted across consumers, for example,
how many consumers mentioned the term “sweet” for sample 1, how
many for sample 2, etc. The data would be structured in three tables,
with the products in rows and the coordinates (x,y) and the mentioned
attributes in columns. The first table would have the x coordinates of
consumer 1 to consumer n, the second table the y coordinates of con-
sumer 1 to consumer n, and the third table would include the frequency
of mention of all the attributes across consumers per each sample
(Fig. 9b).
Pagès (2005) proposed to analyze Napping® data by Multiple
Factor Analysis (MFA), in which after directly collecting the Euclidean
configuration of each subject, the simultaneous processing of all maps
provides a graphical display of the products (biplots) in which two
products are near if they were perceived similar by the whole panel
of subjects, each one having used and weighted its own criteria.
MFA is an enriched PCA — or Multiple Correspondence Analysis
in the case of categorical variables (Pagès, 2005). It analyzes several
tables of variables differing in number and nature, with the require-
ment that within a table, that the variables must be of the same
Fig. 8. Example of a bi-dimensional map or “nappe” configuration of one of the asses-
sors for 7 coffee based beverages.
899
nature (quantitative or qualitative). MFA integrates different tables
of variables describing the same observations (Albert et al., 2011;
Moussaoui & Varela, 2010). The difference with PCA is that MFA
takes into account individual differences rather than averaging the
data (Nestrud & Lawless, 2008). When comments are added to the
sheet to describe the groups (ultra flash profiling), the qualitative
data is analyzed as another data table that is accounted for as supple-
mentary variables (Perrin et al., 2008). Pagès (2005), Perrin et al.
(2008) and Moussaoui and Varela (2010) present schematic exempli-
fying views of how to collect and treat the Napping® data.
An extension of MFA, Hierarchical Multiple Factor Analysis (HMFA),
can be applied when the data are organized in a hierarchical manner;
it balances the role of each table of data and allows the interpretation
in terms of the different levels of hierarchy (Le Dien & Pagès, 2003).
2.3.3. Application, advantages and limitations
Napping® is based on the global perception of the sample set
differences, and it has been described as a natural, intuitive, holistic
way for consumers to describe products, closer to what happens in
front of the supermarket shelf (Ares et al., 2011a; Carrillo, Varela, &
Fiszman, 2012a). It has been applied to various food products like choc-
olate (Risvik et al., 1994), commercial dried soup samples (Risvik,
McEwan, & Rodbotten, 1997), snack bars (King, Cliff, & Hall, 1998),
ewe milk cheeses (Bárcenas, Pérez Elortondo, & Albisu, 2004), citrus
juices (Nestrud & Lawless, 2008), wines (Perrin & Pagès, 2009), hot bev-
erages (Moussaoui & Varela, 2010), milk desserts (Ares, Deliza, et al.,
2010), fish nuggets (Albert et al., 2011) and powdered drinks (Ares et
al., 2011a). Furthermore, Napping® has also been applied with success
to study other stimuli than sensory, like the influence of packaging
info and nutritional claims on consumer perception (Carrillo, Varela,
& Fiszman, 2012a; Carrillo, Varela, & Fiszman, 2012b). The method
can be quite good to understand the relation to consumer hedonic per-
ception and uncover drivers of liking, Ares, Varela, Rado, and Giménez
(2011b) even proposed it as a method to identifying the consumers'
ideal product or flavor space.
Napping has proved to be quite easy and understandable for con-
sumers to perform, Risvik et al. (1997) even suggested the possibility
to use it with children, for the possibility of turning it into a game.
However, some authors report that other novel techniques could be
easier to understand than Napping® to naïve consumers (Ares et al.,
2011a).
The greatest limitation of napping, like in other comparative tech-
niques, is the number of products that can be tested at the same time,
that would much depend on the sensory characteristics of the prod-
uct, but generally with a maximum of 12 (Pagès, 2005). Other limita-
tion is the reproducibility, validity or robustness of the methods, like
most novel techniques, has not been studied in detail so far. Ferrage
et al. (2010) put to discussion the topic of panel performance in
novel profiling methods, particularly with the use of non-trained as-
sessors, and proposed a method to detect bad performers in a nap-
ping test, with the use of a blind control; they stated that Structured
Napping was quite a robust method, even when bad performers
were present in the set. This could be owed to the pre-set dimensions,
that is not always possible to use or relevant for all categories of prod-
ucts or sample sets. There is still much to do in this area and one of
the topics that we expect to appear more often in scientific research
about Napping® and other novel methods of sensory characterization.
2.3.4. Modifications to the original methodology
Napping® is still a developing method as its flexibility allows some
changes in implementation to suit different objectives or depending on
the complexity of the products. Pagès, Cadoret, and Lê (2010) proposed
the sorted Napping® procedure, combining napping with a categoriza-
tion task: after the Napping® exercise panelists are asked to identify ex-
plicit groups by circling products on the sheet. Also, in its initial paper,
Pagès (2003) suggested as a good idea to perform napping by modality
900 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Fig. 9. (a) Schematic view of how to collect Napping® data for 7 coffee based beverages; (b) structure of the data matrix: as an example, x1 and y1 would be the coordinates of P2
for consumer 1, x2 and y2 are the coordinates of P2 for consumer 2, etc. Attributes were recorded with the frequency of mention per product; “bitter” was mentioned 21 times for
P2, “watery” 15 times, “sweet” 3 times, etc.
(appearance, odor, flavor, texture), which was later on implemented
by Pfeiffer and Gilbert (2008) and presented in Sensometrics2008.
Afterwards it has also been performed by Dehlholm, Brockhoff,
Meinert, Aaslyng, and Bredi (2012) with success; this variation is
known as Partial Napping. Finally, Ferrage et al. (2010) suggested in
their communication in Eurosense2010 the possibility of performing
Structured Napping, where pre-named axis would be given to the pan-
elists, signaling the 2 main directions of the sensory flavor space (x and
y axes of the “nappe”), this variant has not been further investigated so
far.
2.4. Check-all-that-apply questions
2.4.1. Theory and implementation
Check-all-that-apply (CATA) questions are multiple choice ques-
tions which are commonly used in marketing research in order to re-
duce response burden (Rasinski, Mingay, & Bradburn, 1994). These
questions consist of a list of words or phrases from which respondents
should select those they consider that apply to answer a certain ques-
tion. The main advantage of this type of question is that it allows mul-
tiple options to be selected, instead of limiting respondents to select
only one answer or forcing consumers to focus their attention and eval-
uate specific attributes (Smyth, Dillman, Christian, & Stern, 2006).
Adams et al. (2007) proposed the application of CATA questions as
a simple method to gather information about consumers' perception
of the sensory characteristics of food products. In this approach, prod-
ucts are presented to consumers in monadic sequence, following a
balanced rotation order. Consumers are asked to try the products
and to answer a CATA question by selecting all the terms that they
consider appropriate to describe each of the samples. There are usual-
ly no constraints on the number of attributes that could be selected by
the consumers. The list of words or phrases in the CATA question
could be exclusively related to the sensory characteristics of the prod-
uct (Fig. 10a) or also include terms related to non-sensory character-
istics, such as usage occasions, product positioning and emotions
(Fig. 10b).
The selection of the list of words or phrases included in the CATA
question is one of the main challenges of the methodology. Sensory
terms could correspond to the descriptors used by trained assessors
panels to characterize the products or could be selected considering
results from previous focus groups or quantitative consumer studies.
As this methodology is mainly used with consumers, the number of
assessors necessary to perform a sensory characterization using CATA
questions ranges from 50 to 100 (Adams et al., 2007; Ares, Barreiro,
Deliza, Giménez, & Gámbaro, 2010; Dooley, Lee, & Meullenet, 2010).
2.4.2. Data analysis
The first step when analyzing data from CATA questions is deter-
mining if consumers detected significant differences between sam-
ples for each of the terms of the CATA question. This analysis is
performed using Cochran's Q test (Parente, Manzoni, & Ares, 2011),
which is a nonparametric statistical test used in the analysis of
two-way randomized block designs to check whether k treatments
have identical effects, when the response variable is binary. For
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Fig. 10. Examples of check-all-that-apply (CATA) questions including: (a) sensory and
(b) non-sensory terms.
each term of the CATA question a data matrix is created containing
samples in columns, consumers in rows. In this matrix each cell indi-
cates if the term was checked or not (1/0 respectively) by each con-
sumer to describe each sample (Fig. 11).
Then, in order to get a bi-dimensional representation of the sam-
ples Multiple Correspondence Analysis (MCA) is used on a matrix
containing the number of consumers who checked each term from
the CATA question to describe each sample (Fig. 12). This analysis
provides a sensory map of the samples, which enables to determine
the similarities and differences between the samples, as well as the sen-
sory attributes that characterize their sensory attributes (Ares, Barreiro,
et al., 2010; Ares, Deliza, et al., 2010; Ares, Giménez, Barreiro and
Gámbaro, 2010).
2.4.3. Application, advantages and limitations
CATA questions have been used for the sensory characterization of
several food products: snacks (Adams et al., 2007), strawberry cultivars
(Lado, Vicente, Manzzioni, & Ares, 2010), ice-cream (Dooley et al.,
2010), milk desserts (Ares, Barreiro, et al., 2010), orange-flavored
powdered drinks (Ares et al., 2011a, 2011b), and citrus-flavored sodas
(Plaehn, 2012).
Some studies have compared the sensory maps generated by
CATA questions with those provided by classic descriptive analysis
(QDA) with a trained assessor panel, reporting similar results (Ares,
Barreiro, et al., 2010; Bruzzone et al., 2012; Dooley et al., 2010). Con-
sidering these results and the fact that checking terms from a list does
not require much effort for consumers, CATA questions have been
reported to be a quick, simple and easy method to gather information
Fig. 11. Example of the data matrix used for analyzing data from check-all-that-apply
(CATA) questions using Cochran's Q test.
901
Fig. 12. Example of the frequency table used for analyzing data from a term of
check-all-that-apply (CATA) questions using correspondence analysis.
about consumer perception of the sensory characteristics of food
products. Moreover, Adams et al. (2007) have reported that CATA
questions have a smaller influence on liking scores than
just-about-right or intensity questions.
However, it is important to take into account that despite the fact
that frequency of mention of the terms from CATA questions have
been reported to be closely related to attribute intensity, they do
not provide quantitative information since consumers only evaluate
if a term is appropriate or not to describe the product. When used
in marketing research, Sudman and Bradburn (1982) indicated that
respondents may not select a term in a check-all question due to
three main reasons: because they consider that the term does not
apply, because they are neutral or undecided about it, or because
they did not pay attention to it. Therefore, if consumers do not select
a term in a CATA question it cannot be concluded that they consider
that it does not apply to the product. Moreover, it has been reported
that respondents do not usually involve in a deep processing and give
quick answers, tending to select the terms that appear near the top
of the list rather than those that appear near the bottom of the list
(Krosnick, 1999). This effect has been also reported for the application
of CATA questions to sensory characterization of food products.
Castura (2009) reported that term position strongly affect results,
which suggests that fixed choice order CATA ballots skew results. For
this reason, rotation of terms within a CATA question is recommended
for getting valid results. However, further research is needed to evaluate
the influence of the number and type of terms, as well as the design of
the questionnaire, in results from sensory characterization provided
by this methodology.
Another limitation of CATA questions is that they provide qualita-
tive data and therefore might have smaller discriminative capacity than
ranking tasks or intensity scales (Dooley et al., 2010). Moreover, CATA
questions require a relatively large number of consumers, not being
recommended with trained assessors. However, frequency-based tech-
niques have been used to characterize the aroma of complex products
such as wine (Campo, Ballester, Langlois, Dacremont, & Valentin, 2010;
Campo, Do, Ferreira, & Valentin, 2008; Le Fur, Mercurio, Moio, Blanquet,
& Meunier, 2003). Campo et al. (2010) have recommended that this ap-
proach could be a practical alternative to conventional descriptive analy-
sis for characterizing products with complex aroma. Also, Nicod et al.
(2010) recommended the application of CATA questions with trained as-
sessors for the evaluation of complex products. In particular, they
suggested its use for attributes present in the sample in low concentra-
tions. These attributes cannot be easily discriminated via scaling since
their perception is more related to presence/absence than to different
intensities.
Furthermore, due to the nature of the response provided by partici-
pants, if the products are very similar the same terms will be selected
for all the evaluated samples. In these cases frequency data from CATA
902 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
questions would not be able to detect significant differences between
samples. Thus, this approach is not recommended when evaluating
products with subtle differences in their sensory characteristics.
2.5. Other methodologies
Several other less common methodologies have been used for sen-
sory characterization of food products. Despite most of these method-
ologies have been applied to a limited number of products, they have
been reported to provide valid results.
2.5.1. Intensity scales
One of the main disadvantages of QDA and other classical descrip-
tive methods is the time and resources needed to train the assessor
panel. Therefore, some authors have stated that the training period
could be omitted and have performed sensory characterization with
the use of consumers (Husson, Le Dien, & Pagès, 2001; Worch, Lê, &
Punter, 2010). In this approach consumers are asked to rate the inten-
sity of a fixed set of sensory attributes using intensity scales, as it is
commonly done with trained assessors in descriptive analysis. The
main difference with the traditional approach is that descriptors are
provided to consumers by the researcher and that no training in attri-
bute recognition or quantification is performed.
Despite this approach has been traditionally not recommended
(Lawless & Heymann, 2010; Stone & Sidel, 2004), recent studies
have reported that results from sensory characterization performed
by 50–100 consumers with intensity scales are similar to those pro-
vided by trained assessor panels (Husson et al., 2001; Worch et al.,
2010). These authors concluded that consumers and trained assessors
panel provided similar results in terms of discrimination, consensus,
and reproducibility and that the product spaces obtained from both
panels were similar.
However, Ares, Bruzzone, & Giménez (2011) compared global and in-
dividual performance of a consumer and trained assessor panels for tex-
ture evaluation of milk desserts, concluding that both showed similar
discriminative capacity and reproducibility and were able to detect the
same differences in the texture of the evaluated milk desserts. On the
other hand, consumer agreement was low and the majority of consumers
were not able to give scores that significantly discriminated among sam-
ples. Thus, the lack of consensus in the consumer panel and the high var-
iability in their evaluations were compensated by the large sample size.
For these reasons, sensory characterization with consumers using intensi-
ty scales would not be recommended, except for specific situations in
which information about the intensity of sensory attributes is needed
and the cost and time involved in the selection and training of the asses-
sors might be higher than those needed to perform a study with 50–150
consumers. In particular, the evaluation of sensory attributes using inten-
sity scales by consumers might be a good option in specific applications
when food companies do not have a trained panel or when the product
is not evaluated on a regular basis. When, information about the intensity
of sensory attributes is not needed other sensory characterization meth-
odologies are recommended.
2.5.2. Open-ended questions
In many methodologies assessors' descriptions have been consid-
ered as supplementary information in order to better understand
the similarities and differences between products (Bécue-Bertaut,
Álvarez-Esteban, & Pagès, 2008; Lawless et al., 1995; Pagès, 2005).
However, sensory characterization has also been performed using ex-
clusively consumer descriptions of products, as suggested by ten Kleij
and Musters (2003). In this methodology open-ended questions are
used to ask assessors to provide a description of the sensory charac-
teristics of a set of products, which aims at understanding the main
characteristics that determine consumer perception of the products
and especially what motivates their liking scores.
Product descriptions have been gathered using three main ap-
proaches. In the original application of the methodology ten Kleij
and Musters (2003) allowed consumers to voluntarily write down
remarks after their overall liking evaluations. Alternatively, Ares,
Giménez, et al. (2010) asked consumers to provide up to four words
to describe the samples after their overall liking evaluation, as part of
the task they had to complete. More recently Symoneaux, Galmarini,
and Mehinagic (2012) gave consumers the option to freely state
what they liked and/or disliked about the evaluated products. All
these options enabled consumers to provide a description about the
sensory characteristics of food products.
Considering that this methodology is applicable to consumer studies,
the number of assessors necessary to perform a sensory characterization
using open-ended questions ranges from 50 to 100 (Ares, Giménez, et al.,
2010; Symoneaux et al., 2012; ten Kleij & Musters, 2003).
Consumer responses to open-ended questions are not subjected to
restrictions from the researchers and therefore contain rich informa-
tion that could underscore and complement quantitative findings
from trained assessor panels (ten Kleij & Musters, 2003). However,
due to the inherent complexity of textual data, data analysis is often
difficult, labor-intensive and time-consuming.
Consumers answer open-ended questions in their own style,
without any specific guidance, even with typing, orthographic and
grammatical mistakes, which makes it necessary to transform the
data into accurate sensory terms (Symoneaux et al., 2012). Analysis
of text data consists of the following stages: removing mistakes, elim-
ination of connectors and auxiliary terms, identification of phrases
and terms which make them up, regrouping synonyms, managing
ambiguous words, and marking terms of interest for the researcher
(Rostaing, Ziegelbaum, Boutin, & Rogeaux, 1998). The first step of
the analysis usually consists of deleting stopwords, auxiliary terms
and other irrelevant words. Then, words with similar meaning are
grouped into the same category according to word synonymy as
determined by a dictionary and the personal interpretation of the re-
searchers. This classification is usually performed consensually by
three researchers (Modell, 2005). Categories mentioned by more
than 5% or 10% of the consumers are selected and frequency of men-
tion of each category is determined by counting the number of partic-
ipants that used each category to describe each product. A frequency
table is constructed and analyzed using Chi-square test and corre-
spondence analysis. Global Chi-square test could be used to deter-
mine significant differences in the description of the evaluated
samples by studying the independence between rows and columns
(Ares, Giménez, et al., 2010). Moreover, a Chi-square per cell test
can be used to identify significant differences between samples for
each of the sensory characteristics used by consumers to describe
the evaluated products (Symoneaux et al., 2012). Finally, correspon-
dence analysis can be used to get a 2-dimensional representation of
the samples and the attributes (Ares, Giménez, et al., 2010; ten Kleij
& Musters, 2003).
This methodology has been used in a limited number of food
products: mayonnaise (ten Kleij & Musters, 2003), milk desserts
(Ares, Giménez, et al., 2010), and apples (Symoneaux et al., 2012).
In these studies, sample maps gathered from open-ended questions
have been reported to be similar to those obtained from classic de-
scriptive analysis (QDA) with trained assessor panels.
It is important to take into account that despite its simplicity and
ease of use for consumers, the descriptions provided by consumers
are usually vague and difficult to interpret, which makes data analysis
tedious and difficult. Moreover, the information provided by this
methodology is not as precise as the information provided by descrip-
tive analysis or other methodologies, particularly in those cases in
which differences between the products are small.
Open-ended questions can be considered as complementary to
traditional descriptive approach with trained assessors. Consumers'
responses to open-ended questions could be used to get an insight
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
on the vocabulary used by consumers to describe the products, which
could be interesting when designing communication strategies (Ares,
Giménez, et al., 2010; Symoneaux et al., 2012).
A particular case in which this methodology is recommended is
when working with experts in the particular product under study,
since they are usually familiar with providing a verbal description of
products. In this sense, Thuillier (2007) proposed a variation of this
method, called Pivot© profile to characterize the sensory properties
of champagne with wine experts. The method is based on the free de-
scription of the differences between samples and a reference product,
which is called “pivot”. Assessors are asked to describe the sensory
attributes that they perceive as less intense in the product compared
to the pivot (e.g. less bitter), as well as those that they perceive as
more intense (more acid). They are asked to use only descriptive
words and to avoid writing complete sentences. An example of the
evaluation sheet is shown in Fig. 13.
2.5.3. Preferred attribute elicitation method
Grygorczyk et al. (in press) proposed the preferred attribute elicita-
tion method as a novel approach to identify the key attributes which
drive consumer liking. In this method, consumers are asked to agree
on a set of attributes to describe a group of samples within a product
category, and to rank them according to how important they consider
they are in determining their liking of the product category.
Grygorczyk et al. (in press) presented a set of commercial vanilla
yogurts to consumers and asked them to rate their texture liking
and to write down the sensory attributes they liked and disliked
about the samples. Then, through a round-table discussion with a
moderator consumers were asked to group the attributes, based on
their similarity, to select anchors for their evaluation, and to rank
the groups according to their importance in driving their liking.
After a short break, consumers were asked to individually rate prod-
ucts on 7-point scales for each attribute.
Intensity data was analyzed using GPA as in free choice profiling.
When characterizing yogurt texture, results from preferred attribute elic-
itation method with 42 consumers were similar to those provided by con-
ventional descriptive profiling with 10 trained assessors (Grygorczyk et
al., in press). Regarding attribute importance, consumers consistently
ranked texture and flavor as very important to their liking, whereas ap-
pearance attributes were ranked as the least important.
According to the authors, the advantage of this method is that it
directly identifies the most relevant attributes for consumers, provid-
ing product developers a smaller set of terms to be considered during
product optimization.
2.5.4. Polarized sensory positioning
Polarized sensory positioning is based on the comparison of food
samples to a fixed set of reference products, or poles. This methodology
Fig. 13. Example of an evaluation sheet used in pivot profile.
903
was developed by Teillet et al. (2010) to explore the sensory character-
istics of water. Despite the fact that these authors applied PSP with 15
trained assessors, this methodology could be applied with semi-trained
assessors or naive consumers.
Assessors are asked to evaluate the degree of similarity of the sam-
ples to a set of reference products. Teillet et al. (2010) selected three
poles that represented three typical profiles of mineral water. When
evaluating a sample assessors should quantify the overall difference
between it and each of the references using unstructured scales rang-
ing from “exactly the same taste” to “totally different taste”, as shown
in Fig. 14. A description phase should be performed in order to get in-
formation about the sensory characteristics responsible for the simi-
larities and differences between products.
Data analysis could be performed using MDS or PCA. In the first
approach assessors ratings are considered as a measure of the dis-
tance from each pole (Fig. 15). Ratings are averaged by sample and an-
alyzed using Multidimensional Scaling unfolding techniques (Busing,
Groenen, & Heiser, 2005) on the samples by poles matrix, to get a
two-dimensional representation of the samples. In the second approach
the poles are considered as descriptors, data is analyzed by calculating
average scores, and sample representation is obtained by Principal
Component Analysis (Teillet et al., 2010).
Polarized sensory positioning is an easy and quick methodology
which could be performed with trained and untrained assessors. Its
main advantage is that it enables to compare all products with fixed
references, even if they are not evaluated in the same session. Howev-
er, it is important to note that research is necessary to determine how
reference samples should be selected, and particularly how many
samples are necessary and which their characteristics should be.
2.5.5. Paired comparison
Paired comparisons are one of the most common methodologies in
sensory science (Lawless & Heymann, 2010). They are usually used to
determine if two samples are perceived as equal or different in a specific
sensory characteristic.
Poirson, Petiot, and Richard (2010) proposed the application of
paired-comparison tasks for gathering information about consumers'
perceptual space of diesel motor sounds. These authors asked consumers
to perform a series of paired comparison tasks according to a list of attri-
butes. For each attribute a comparison matrix was created, which con-
tained all samples in rows and columns. Each intersection of columns
and rows corresponds to a paired comparison (Fig. 16), which has to be
filled up using a 7-point scale (□□, □, □~, =, □~, □, □□). Assessors
are asked to complete some of the comparisons of each matrix by com-
paring each pair of samples and to assess their difference by using the
scale. The authors worked with two different panels: 10 experts and 30
naive consumers.
Data is analyzed to determine the discriminative power of the at-
tributes and the consensus between panelists by using least squares
logarithmic regression (LSLR).
Poirson et al. (2010) reported that paired comparisons provided
better agreement between consumers and were more discriminating
than ratings tasks. The results of these two tests with the naives were
then compared with the conventional sensory profile of the experts
using Generalized Procrustes Analysis. Moreover, good consensus
was found between results from paired comparison task and those
from descriptive analysis with a trained assessor panel.
2.6. Comparison of the methodologies
Several studies have focused on the comparison of the sensory
characterization provided by conventional descriptive analysis and
novel methodologies in a wide range of food products with different
complexity, ranging from simple products such as mineral water
(Teillet et al., 2010) to complex products as wine (Perrin et al.,
2008) or fish nuggets (Albert et al., 2011). Most studies have reported
904 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Fig. 14. Example of an evaluation sheet used in polarized sensory positioning to compare one sample with three reference products (R1, R2 and R3).
that, compared to results provided by conventional descriptive analysis
with trained assessors, novel methodologies provide similar informa-
tion on the main sensory characteristics responsible for differences
between products, as well as similar sensory maps (Albert et al., 2011;
Ares, Barreiro, et al., 2010; Ares, Giménez, et al., 2010; Ares et al.,
2011; Bruzzone et al., 2012; Cartier et al., 2006; Chollet et al., 2011;
Dairou & Sieffermann, 2002; Delarue & Sieffermann, 2004; Dooley et
al., 2010; Guàrdia, Aguiar, Claret, Arnau, & Guerrero, 2010; Jack &
Piggott, 1991; Moussaoui & Varela, 2010; Risvik et al., 1994, 1997;
Symoneaux et al., 2012; Teillet et al., 2010).
However, it is important to stress that the information provided
by descriptive analysis is clearly different from that gathered from
novel methodologies. Descriptive analysis provides quantitative in-
formation about the intensity of specific sensory attributes, enabling
to identify significant differences between samples in each of the
evaluated attributes. On the other hand, it is not possible to gather
this information using novel methodologies. From a statistical point
of view, descriptive analysis is more robust than most novel methodol-
ogies, which makes it possible to identify small and subtle differences
between products (Albert et al., 2011). Furthermore, considering asses-
sors' training, descriptive analysis is more appropriate for comparing
different sample sets or for comparing samples evaluated at different
moments in time (Moussaoui & Varela 2010).
Another disadvantage of many novel methodologies is related to
the interpretation of the sensory terms provided by assessors. Analy-
sis of assessor descriptions in free profiling, flash profile, open-ended
questions, or holistic methodologies is in general a time-consuming,
labor-intensive and difficult task. Due to the heterogeneity of con-
sumers' descriptions, the large number of terms used and the lack
Fig. 15. Example of the data matrix used for analyzing data from polarized sensory po-
sitioning using multidimensional scaling. Each couple of columns R1, R2, and R3 repre-
sent the degree of difference between a sample and each of the references (R1, R2 and
R3 respectively) for each of the assessors.
of definitions and evaluation procedures information about specific
sensory attributes could be difficult to interpret. In particular, the in-
terpretation of consumers' vocabulary can be difficult for complex
multiparameter sensations as for example “creaminess”, when more
than one modality is involved and it is not known if they are referring
to flavor, texture or aroma (Moussaoui & Varela, 2010). Therefore, re-
sults from descriptive analysis are usually more actionable for prod-
uct developers than those from novel methodologies; being the
latter particularly useful when the objective is to identify the most
salient attributes and the most relevant characteristics responsible
for the similarities and differences between products.
Added to the vocabulary complexity, consumers sometimes use
hedonics or benefit-related terms which in principle can be seen as a
limitation because it complicates the analysis (Veinand et al., 2011),
however, this information could be interesting to relate product charac-
teristics to marketable features and consumer preference.
Despite the fact that descriptive analysis provides more accurate and
reliable information in most cases, some clear advantages of novel meth-
odologies could be mentioned. In the first place, the time needed for the
implementation of novel methodologies for sensory characterization of a
set of products is considerably shorter than for descriptive analysis,
which makes the novel approach an interesting alternative, particularly
for those working in the industry. Another advantage of novel methodol-
ogies is that they do not require consensus from the panel, which could
potentially lead to some loss of information due to the fact that if the per-
ception of the minority of the assessors differs from that of the majority,
it is not taken into account (Albert et al., 2011). The lack of need for con-
sensus between panelists allows a diversity of points of views, which
could provide richer information (Dairou & Sieffermann, 2002).
Fig. 16. Example of the evaluation sheet used by consumers in the paired comparison
methodology.
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
In a few cases novel methodologies have been reported to provide
better information than descriptive analysis. For example, Delarue
and Sieffermann (2004) stated that when working with similar prod-
ucts flash profile was more discriminating than descriptive analysis.
Similarly, Albert et al. (2011) reported that flash profile performed
with semi-trained assessors provided a more detailed description of
the sensory characteristics of fish nuggets than descriptive analysis.
Furthermore the lack of consensus could also be advantageous in
the case of very heterogeneous products where it is very difficult to
reach.
Novel methodologies clearly differ in the way in which they gath-
er information about the sensory characteristics of food products,
which leads to differences in the information they provide about
products and the applications for which they are recommended
(Blancher et al., 2007). Holistic methodologies, such as sorting and
projective mapping, are based on assessors' global perception of the
products, which might enable to identify the main attributes respon-
sible for differences in how they perceive the samples. In free sorting
and projective mapping assessors focus their attention on the global
perception of the products, which enables to identify the most salient
sensory characteristics of the products. On the other hand, when
using methods based on the evaluation of specific attributes assessors'
perception is focused on specific features. This leads to differences in
the information provided by similarity based methodologies and
those that rely on the evaluation of specific attributes. The Focusing
on attributes can be of use when specific or more detailed information
about the product is needed, however, the evaluation would be more
artificial than that of holistic approaches. Methods that are not based
on attributes have the extra advantage that can be used for
cross-country studies with consumers who speak different languages,
without translation problems.
Many studies have reported that information provided by different
methodologies is similar. When comparing CATA questions and projec-
tive mapping for sensory characterization of milk desserts Ares, Deliza,
et al. (2010) concluded that both methodologies provided similar infor-
mation. Similarly, when evaluating powdered orange-flavored drinks
Ares et al. (2011a) reported that CATA questions, sorting, projective
mapping and intensity scales were equivalent.
On the other hand, some authors have reported that similarity-based
methods are less discriminative than those from methodologies
based on the evaluation of specific sensory attributes, particularly
when small differences between samples are considered. For example,
Veinand et al. (2011) compared three methodologies (free choice pro-
filing, flash profile and projective mapping) for consumer profiling of
lemon iced teas and concluded that flash profile showed the highest
discriminative ability, whereas projective mapping showed the lowest.
Albert et al. (2011) reported that flash profile provided more detailed
information about the sensory characteristics of fish nuggets than
projective mapping due to the fact that the latter was based on asses-
sors' global perception of the products. Moussaoui and Varela (2010)
reported that flash profile and free choice profile provided richer vocab-
ularies and more accurate sample maps than similarity-based method-
ologies such as projective mapping and free sorting when working
with hot beverages. Moreover, these authors reported that untrained
assessors were more repeatable when working with flash profile
compared to projective mapping or free sorting. Nestrud and Lawless
(2010) compared projective mapping and free sorting and concluded
that, despite the sensory maps provided by both methodologies were
similar for apples and for cheese, the identification of samples with sim-
ilar sensory characteristics was easier to interpret for projective mapping.
When comparing novel methodologies it is also important to take
into account practical issues since they clearly differ in the difficulty
that assessors encounter when completing the tasks. Holistic meth-
odologies can be considered more intuitive and less rational than
other methodologies based on the evaluation of specific sensory attri-
butes. However, Ares et al. (2011a) reported that although consumers
905
were able to understand projective mapping and sorting tasks, they
found them much more difficult than CATA questions or intensity
scales. In agreement with this result, Veinand et al. (2011) reported
that projective mapping was more difficult to perform with con-
sumers than flash profile. These authors stated that when performing
a projective mapping task consumers found it difficult to use the
sheet of paper to locate the samples according to their similarities
and differences. Besides, Ares, Deliza, et al. (2010) reported that, in
order to assure that consumers understood the task, it was necessary
to provide further explanations when working with projective map-
ping compared to CATA questions.
Regarding the time needed by assessors to complete the task, in-
tensity scales, CATA questions, open-ended questions and pivot pro-
file are usually less time-consuming than projective mapping, free
choice profiling, flash profile, and polarized sensory positioning. Free
choice profiling and flash profile require two separate sessions, one
for descriptor generation and another for evaluating the sample set.
Meanwhile, the rest of the methodologies could be carried out in a
single session. Projective mapping and sorting tasks have been reported
to be more time-consuming than CATA or open-ended questions.
According to Ares, Deliza, et al. (2010), consumers needed between 5
and 15 min to complete a CATA question for sensory characterization
of 8 milk desserts, whereas they needed between 18 and 25 min to
complete a projective mapping task with the same samples.
Thus, holistic methodologies such as projective mapping and free
sorting seem to be more difficult and time-consuming for consumers.
Considering that trained assessors with previous experience with
sensory evaluation techniques could more easily understand these
methodologies, Veinand et al. (2011) recommended performing pro-
jective mapping with expert panelists.
Another disadvantage of projective mapping when using paper
ballots is that measuring the products' coordinates in the sheet of
each assessor is tedious and tiresome for the researchers, particularly
when a large number of consumers are used (Veinand et al., 2011).
The methods also differ in the number of samples that could be in-
cluded within a set in a single session. Free choice profiling, flash pro-
file, free sorting and projective mapping require that all products
are evaluated by assessors simultaneously in the same session, since
comparisons between them are made. Thus, in order to avoid fatigue
and adaptation, the number of samples to be evaluated in a single
session is limited when compared to other methodologies such as in-
tensity scales, CATA questions or polarized sensory positioning. For
this reason, it could be complicated to use the former methodologies
them when working with products that require careful temperature
control or that have intense and persistent sensory characteristics. In
particular, polarized sensory positioning requires the use of a fixed
reference product, which makes it appropriate for comparing products
over time or when dealing with evaluations performed on different
sessions. However, the criteria for the selection of stable and easily
available reference products should be carefully taken into account;
also, the fact of having to compare with a reference sample makes it
more tiresome for the panelists, as the technique requires repeated tast-
ing of various samples.
3. Conclusions and recommendations
A summary of the characteristics of the novel methodologies for
sensory characterization reviewed in the present article is provided
in Table 1.
Novel methodologies consist of valid, reliable, simple and quick al-
ternatives for sensory characterization of food products. They have
been reported to provide similar information to classical descriptive
analysis performed with trained assessor panels. However, it is im-
portant to highlight that they could not be considered a replacement
for classic descriptive analysis since it is always more accurate due to
the fact that assessors are extensively trained in the identification and
906 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

Table 1
Summary of the characteristics of the methodologies reviewed in the present article.

Method Type of evaluation Vocabulary Statistical method Limitations

Sorting Classification of samples based on their similarities Elicited by the assessors or MDS, DISTATIS or All samples should be presented
and differences provided by the researcher FAST simultaneously
(MCA and MFA)
Flash profiling Ranking of samples on a set of selected attributes Elicited by the assessors GPA Two sessions are required
All samples should be presented
simulteaneously
Projective mapping Generating samples on a two-dimensional map according to Elicited by the assessors MFA All samples should be presented
or Napping® their similarities and differences simulteaneously
It could be difficult to understand
for naïve consumers
Check-all-that-apply Selection of terms from a list that are appropriate to Provided by the researcher Cochran Q test, The design of the attribute list
(CATA) questions describe the samples MCA, MFA could strongly affect the responses
Not recommended for evaluating
very similar samples
Intensity scales Rating the intensity of a set of attributes using scales Provided by the researcher ANOVA, PCA Lack of consensus in consumers'
responses
Open-ended questions Verbal description of the samples Elicited by the assessors Content analysis, Difficulties for analyzing verbal
Chi-square and MCA descriptions
Preferred attribute Ranking of attributes according to their importance and Elicited by the assessors GPA A round-table discussion is necessary
elicitation rating of products using structured scales All samples should be presented
simultaneously
Polarized sensory Evaluation of global differences between samples and a Not gathered in the original MDS or PCA Stable and readily-available refer-
positioning set of fixed references method ences are needed
It could be elicited by the Selection of the references could
assessors strongly affect the results
Paired comparison Paired comparisons between samples in a set of attributes Provided by the researcher LSLR Complicated experimental design

quantification of sensory attributes. Therefore, descriptive analysis is
more appropriate when the objective of the sensory characterization
is to identify small differences between products or to detect differ-
ences in the intensity of specific sensory attributes, as it happens in
many cases during the optimization step of the development of new
products.
On the other hand, when non-detailed information about the sen-
sory characteristics of food products is required, novel methodologies
offer a good and quick alternative. They could be considered a valu-
able alternative to get information about the sensory characteristics
of food products for companies which do not have enough time or re-
sources to train a panel for evaluating a specific product, which is
common in many small companies or developing countries. In these
situations the cost and time required for the selecting and training
sensory assessors might be higher than those needed to perform a con-
sumer study. Novel methodologies are also useful when conducting
preliminary studies for getting information about the sensory charac-
teristics of food products or when performing screening tests for the se-
lection of products or conditions for the design of a larger experiment.
Moreover, sensory characterization with consumers could be use-
ful for uncovering consumer perception of food products, with their
own vocabulary, which could provide valuable information during
new food product development or when designing marketing or
communication campaigns. In this context, holistic methodologies,
free choice profiling and flash profile enable the identification of
consumers' vocabulary to describe the sensory characteristics of the
products. On the other hand, CATA questions and intensity scales
rely on previous studies to identify consumers' relevant terms.
The selection of a novel methodology for a particular application
depends on the type of assessors to be considered, practical issues
and the specific objectives of the studies. However, when working
with consumers it would be generally easier to work with simple
methodologies such as CATA questions, open-ended questions or
pivot profile. On the other hand, when a trained assessor panel is
available and quick information about the sensory characteristics of
food products is needed, the recommended approach would be to
apply flash profile, sorting, projective mapping or polarized sensory
positioning due to their higher complexity. Holistic methods based
on global similarity, such as sorting and projective mapping seem
more appropriate when summarized sensory information is needed;
also, they can be an interesting approach when analyzing the percep-
tion of external cues like packaging information, as holistic techniques
allow a more realistic setting, closer to what a consumer would do
when buying. Polarized sensory positioning or pivot profile is a good
option when the aim of the study is to compare new products with
known or reference products or when the sensory characteristics of
samples evaluated over time are to be compared.
Finally, it is important to take into account that most of the novel
methodologies for sensory characterization have been used for a rel-
atively short period of time and have been applied in a limited number
of applications. For this reason, further research on the applicability,
reliability and reproducibility of new approaches for sensory character-
ization is still strongly needed, particularly when dealing with complex
products.
References
Abdi, H., Valentin, D., Chollet, S., Chrea, C., Abdi, H., Valentin, D., et al. (2007). Analyzing
assessors and products in sorting tasks: DISTATIS, theory and applications. Food
Quality and Preference, 18, 627–640 (Food Quality and Preference, 20, 410–417).
Adams, J., Williams, A., Lancaster, B., & Foley, M. (2007). Advantages and uses of
check-all-that-apply response compared to traditional scaling of attributes for
salty snacks. 7th Pangborn Sensory Science Symposium. Minneapolis, USA, 12–16
August, 2007.
Albert, A., Varela, P., Salvador, A., Hough, G., & Fiszman, S. (2011). Overcoming the is-
sues in the sensory description of hot served food with a complex texture. Applica-
tion of QDA®, flash profiling and projective mapping using panels with different
degrees of training. Food Quality and Preference, 22, 463–473.
Ares, G., Barreiro, C., Deliza, R., Giménez, A., & Gámbaro, A. (2010). Application of
a check-all-that-apply question to the development of chocolate milk desserts.
Journal of Sensory Studies, 25, 67–86.
Ares, G., Bruzzone, F., & Giménez, A. (2011). Is a consumer panel able to reliably eval-
uate the texture of dairy desserts using unstructured intensity scales? Evaluation
of global and individual performance. Journal of Sensory Studies, 26, 363–370.
Ares, G., Deliza, R., Barreiro, C., Giménez, A., & Gámbaro, A. (2010). Comparison of
two sensory profiling techniques based on consumer perception. Food Quality
and Preference, 21, 417–426.
Ares, G., Giménez, A., Barreiro, C., & Gámbaro, A. (2010). Use of an open-ended ques-
tion to identify drivers of liking of milk desserts. Comparison with preference map-
ping techniques. Food Quality and Preference, 21, 286–294.
 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Ares, G., Varela, P., Rado, G., & Gimenez, A. (2011a). Are consumer profiling techniques
equivalent for some product categories? The case of orange-flavoured powdered
drinks. International Journal of Food Science and Technology, 46, 1600–1608.
Ares, G., Varela, P., Rado, G., & Giménez, A. (2011b). Identifying ideal products using
three different consumer profiling methodologies. Comparison with external pref-
erence mapping. Food Quality and Preference, 22, 581–591.
Bárcenas, P., Pérez Elortondo, F. J., & Albisu, M. (2004). Projective mapping in sensory
analysis of ewes milk cheeses: A study on consumers and trained panel perfor-
mance. Food Research International, 37, 723–729.
Bécue-Bertaut, M., Álvarez-Esteban, R., & Pagès, J. (2008). Rating of products through
scores and free-text assertions: Comparing and combining both. Food Quality and
Preference, 19, 122–134.
Bijmolt, T., & Wedel, M. (1995). The effects of alternative methods of collecting similar-
ity data for multidimensional scaling. International Journal of Research in Marketing,
12, 363–371.
Black, M. B. (1963). On formal ethnographic procedures. American Anthropologist, 65,
1347–1351.
Blancher, G., Chollet, S., Kesteloot, R., Nguyen, D., Cuvelier, G., & Sieffermann, J. -M.
(2007). French and Vietnamese: How do they describe texture characteristics of
the same food? A case study with jellies. Food Quality and Preference, 18, 560–575.
Brandt, M. A., Skinner, E. Z., & Coleman, J. A. (1963). Texture profile method. Journal of
Food Science, 28, 404–409.
Bruzzone, F., Ares, G., & Giménez, A. (2012). Consumers' texture perception of milk
desserts II- Comparison with trained assessors' data. Journal of Texture Studies,
43, 214–226.
Busing, F. M. T. A., Groenen, P. J. F., & Heiser, W. J. (2005). Avoiding degeneracy in
multidimensional unfolding by penalizing on the coefficient of variation. Psychometrika,
70, 49–76.
Cadoret, M., Lê, S., & Pagès, J. (2009). A factorial approach for sorting task data (FAST).
Food Quality and Preference, 20, 410–417.
Cairncross, S. E., & Sjöstrom, L. B. (1950). Flavour profiles: A new approach to flavour
problems. Food Technology, 4, 308–311.
Campo, E., Ballester, J., Langlois, J., Dacremont, C., & Valentin, D. (2010). Comparison of
conventional descriptive analysis and a citation frequency-based descriptive meth-
od for odor profiling: An application to Burgundy Pinot noir wines. Food Quality
and Preference, 21, 44–55.
Campo, E., Do, B. V., Ferreira, V., & Valentin, D. (2008). Aroma properties of young Spanish
white wines: A study using sorting task, list of terms and frequency of citation.
Australian Journal of Grape and Wine Research, 14, 104–115.
Carrillo, E., Varela, P., & Fiszman, S. (2012a). Packaging information as a modulator of
consumers' perception of enriched and reduced-calorie biscuits in tasting and
non-tasting tests. Food Quality and Preference, 25, 105–115.
Carrillo, E., Varela, P., & Fiszman, S. (2012b). Effects of food package information and
sensory characteristics on the perception of healthiness and the acceptability of
enriched biscuits. Food Research International, 48, 209–216.
Cartier, R., Rytz, A., Lecomte, A., Poblete, E., Krystlik, J., Belin, E., et al. (2006). Sorting
procedure as an alternative to quantitative descriptive analysis to obtain a product
sensory map. Food Quality and Preference, 17, 562–571.
Castura, J. C. (2009). Do panellists donkey vote in sensory choose-all-that-apply questions?
8th Pangborn Sensory Science Symposium, July 26–30, Florence.
Chollet, S., & Valentin, D. (2001). Impact of training on beer flavour perception and
description: Are trained and untrained subjects really different. Journal of Sensory
Studies, 16, 601–618.
Chollet, S., Lelièvre, Abdi, H., & Valentin, D. (2011). Sort and beer: Everything you
wanted to know about the sorting task but did not dare to ask. Food Quality and
Preference, 22, 507–520.
Courcoux, P., Qannari, E. M., Taylor, Y., Buck, D., & Greenhoff, K. (2012). Taxonomic free
sorting. Food Quality and Preference, 23, 30–35.
Coxon, A. P. M. (1999). Sorting data: Collection and analysis. Thousand Oaks, CA: Sage.
Coxon, A. P. M., Davies, P. M., & Jones, C. L. (1986). Images of social stratification. London: Sage.
Dairou, V., & Sieffermann, J. -M. (2002). A comparison of 14 jams characterized by con-
ventional profile and a quick original method, flash profile. Journal of Food Science,
67, 826–834.
Dehlholm, C., Brockhoff, P., Meinert, L., Aaslyng, M., & Bredi, W. (2012, December). Sort
and beer: Everything you wanted to know about the sorting task but did not dare
to ask. Food Quality and Preference, 26(2), 267–277.
Delarue, J., & Sieffermann, J. -M. (2004). Sensory mapping using Flash profile. Compar-
ison with a conventional descriptive method for the evaluation of the flavour of
fruit dairy products. Food Quality and Preference, 15, 383–392.
Dooley, L., Lee, Y. -S., & Meullenet, J. -F. (2010). The application of check-all-that-apply
(CATA) consumer profiling to preference mapping of vanilla ice cream and its com-
parison to classical external preference mapping. Food Quality and Preference, 21,
394–401.
Falahee, M., & MacRae, A. W. (1997). Perceptual variation among drinking waters: The
reliability of sorting and ranking data for multidimensional scaling. Food Quality
and Preference, 8, 389–394.
Ferrage, A., Nicod, L., & Varela, P. (2010). How does assessors' performance influence
the outcome of alternative sensory methodologies? Eurosense 2010 — A sense of
quality. Vitoria-Gasteiz, Spain, 5–8 September, 2010.
Gacula, M. C. (1997). Descriptive sensory analysis in practice. Trumbull, CT: Food and
Nutrition Press.
Gawel, R., Iland, P. G., & Francis, I. L. (2001). Characterizing the astringency of red wine:
A case study. Food Quality and Preference, 12, 83–94.
Gower, J. C. (1975). Generalised procrustes analysis. Psychometrika, 40, 33–50.
Grygorczyk, A., Lesschaeve, I., Corredig, M., & Duizer, L. (in press). Extraction of
consumer texture preferences for yogurt: Comparison of the preferred attribute
907
elicitation method to conventional profiling. Food Quality and Preference, http://
dx.doi.org/10.1016/j.foodqual.2012.02.017.
Guàrdia, M. D., Aguiar, A. P. S., Claret, A., Arnau, J., & Guerrero, L. (2010). Sensory charac-
terization of dry-cured ham using free-choice profiling. Food Quality and Preference,
21, 148–155.
Husson, F., Le Dien, S., & Pagès, J. (2001). Which value can be granted to sensory pro-
files given by consumers? Methodology and results. Food Quality and Preference,
12, 291–296.
Jack, F. R., & Piggott, J. R. (1991). Free choice profiling in consumer research. Food Quality
and Preference, 3, 129–134.
Kelly, G. A. (1955). The psychology of personal constructs. New York: Norton.
King, M. C., Cliff, M. A., & Hall, J. W. (1998). Comparison of projective mapping and sorting
data collection and multivariate methodologies for identification of similarity-of-use
of snack bars. Journal of Sensory Studies, 13, 347–358.
Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567.
Lado, J., Vicente, E., Manzzioni, A., & Ares, G. (2010). Application of a check-all-that-apply
question for the evaluation of strawberry cultivars from a breeding program. Journal
of the Science of Food and Agriculture, 90, 2268–2275.
Lassoued, N., Delarue, J., Launay, B., & Michon, C. (2008). Baked product texture: Corre-
lations between instrumental and sensory characterization using flash profile.
Journal of Cereal Science, 48, 133–143.
Lawless, H. T., & Heymann, H. (2010). Sensory evaluation of food. Principles and prac-
tices. (2nd edition). New York: Springer.
Lawless, H. T., Sheng, N., & Knoops, S. S. C. P. (1995). Multidimensional scaling of
sorting data applied to cheese perception. Food Quality and Preference, 6, 91–98.
Le Dien, S., & Pagès, J. (2003). Hierarchical multiple factor analysis: Application to the
comparison of sensory profiles. Food Quality and Preference, 14, 397–403.
Le Fur, Y., Mercurio, V., Moio, L., Blanquet, J., & Meunier, J. M. (2003). A new approach to
examine the relationships between sensory and gas chromatography–olfactometry
data using generalized procrustes analysis applied to Six French Chardonnay wines.
Journal of Agriculture and Food Chemistry, 51, 443–452.
Lelièvre, M., Chollet, S., Abdi, H., & Valentin, D. (2008). What is the validity of the
sorting task for describing beers? A study using trained and untrained assessors.
Food Quality and Preference, 19, 697–703.
Lokki, T., Pätynen, J., Kuusinen, A., Vertanen, H., & Tervo, S. (2011). Concert hall acous-
tics assessment with individually elicited attributes. Journal of the Acoustical Society
of America, 130, 835–849.
Mazzucheli, R., & Guinard, J. X. (1999). Comparison of monadic and simultaneous sample
presentation modes in a descriptive analysis of milk chocolate. Journal of Sensory
Studies, 14, 235–248.
Meilgaard, M. C., Civille, G. V., & Carr, B. T. (1991). Sensory evaluation techniques (2nd
edition). Boca Raton, FL: CRC Press.
Miller, D. M., Wiley, D. E., & Wolfe, R. G. (1986). Categorization methodology: An ap-
proach to the collection and analyses of certain classes of qualitative information.
Multivariate Behavioral Research, 21, 135–167.
Modell, S. (2005). Triangulation between case study and survey methods in manage-
ment accounting research: An assessment of validity implications. Management
Accounting Research, 16, 231–254.
Moussaoui, K. A., & Varela, P. (2010). Exploring consumer product profiling techniques
and their linkage to a quantitative descriptive analysis. Food Quality and Preference,
21, 1088–1099.
Murray, J. M., Delahunty, C. M., & Baxter, I. A. (2001). Descriptive sensory analysis: Past,
present and future. Food Research International, 34, 461–471.
Nestrud, M., & Lawless, H. (2008). Perceptual mapping of citrus juices using projective
mapping and profiling data from culinary professionals and consumers. Food Quality
and Preference, 19, 431–438.
Nestrud, M. A., & Lawless, H. T. (2010). Perceptual mapping of apples and cheeses using
projective mapping and sorting. Journal of Sensory Studies, 25, 390–405.
Nicod, L., Varela, P., & Ferrage, A. (2010). Could too much training ever be a limitation
in products' characterization? Oral presentation. Eurosense 2010 — A sense of quality.
Vitoria-Gasteiz, Spain, 5–8 September, 2010.
Pagès, J. (2003). Recueil direct de distances sensorielles: Application à l'évaluation de
dix vins blancs du Val-de-Loire. Sciences des Aliments, 23, 679–688.
Pagès, J. (2005). Collection and analysis of perceived product inter-distances using
multiple factor analysis: Application to the study of 10 white wines from the
Loire Valley. Food Quality and Preference, 16, 642–649.
Pagès, J., Cadoret, M., & Lê, S. (2010). The sorted napping: A new holistic approach in
sensory evaluation. Journal of Sensory Studies, 25, 637–658.
Parente, M. E., Manzoni, A. V., & Ares, G. (2011). External preference mapping of com-
mercial antiaging creams based on consumers' responses to a check-all-that-apply
question. Journal of Sensory Studies, 26, 158–166.
Perrin, L., & Pagès, J. (2009). Construction of a product space from the ultra-flash pro-
filing method: Application to 10 red wines from the Loire Valley. Journal of Sensory
Studies, 24, 373–395.
Perrin, L., Symoneaux, R., Maître, I., Asselin, C., Jourjon, F., & Pagès, J. (2008). Compari-
son of three sensory methods for use with the Napping® procedure: Case of ten
wines from Loire Valley. Food Quality and Preference, 19, 1–11.
Pfeiffer, J., & Gilbert, C. (2008). Napping by modality: A happy medium between analytic
and holistic approaches. Oral communication. Sensometrics 2008.
Plaehn, D. (2012). CATA penalty/reward. Food Quality and Preference, 24, 141–152.
Poinot, P., Grua-Priol, J., Arvisenet, G., Rannou, C., Semenou, M., Le Bail, A., et al. (2007).
Optimisation of HS-SPME to study representativeness of partially baked bread
odorant extracts. Food Research International, 40, 1170–1184.
Poirson, E., Petiot, J. -F., & Richard, F. (2010). A method for perceptual evaluation of
products by naive subjects: Application to car engine sounds. International Journal
of Industrial Ergonomics, 40, 504–516.
908 P. Varela, G. Ares / Food Research International 48 (2012) 893–908
Popper, P., & Heymann, H. (1996). Analyzing differences among products and panelists
by multidimensional scaling. In T. Naes, & E. Risvik (Eds.), Multivariate analysis of
data in sensory science (pp. 159–184). Amsterdam: Elsevier.
Rasinski, K. A., Mingay, D., & Bradburn, N. M. (1994). Do respondents really mark all
that apply on self-administered questions? Public Opinion Quarterly, 58, 400–408.
Risvik, E., McEvan, J. A., Colwill, J. S., Rogers, R., & Lyon, D. H. (1994). Projective mapping:
A tool for sensory analysis and consumer research. Food Quality and Preference, 5,
263–269.
Risvik, E., McEwan, J. A., & Rodbotten, M. (1997). Evaluation of sensory profiling and
projective mapping data. Food Quality and Preference, 8, 63–71.
Rostaing, H., Ziegelbaum, H., Boutin, E., & Rogeaux, M. (1998). Analyse de commentaires
libres par la techniques des réseaux de segments. Fourth International Conference on
the Statistical Analysis of Textual Data, JADT'98.
Saint-Eve, A., Paài Kora, E., & Martin, N. (2004). Impact of the olfactory quality and
chemical complexity of the flavouring agent on the texture of low fat stirred
yogurts assessed by three different sensory methodologies. Food Quality and
Preference, 15, 655–668.
Santosa, M., Abdi, H., & Guinard, J. -X. (2010). A modified sorting task to investigate
consumer perceptions of extra virgin olive oils. Food Quality and Preference, 21,
881–892.
Schiffman, S. S., Reynolds, M. L., & Young, F. W. (1981). Introduction to multidimensional
scaling. New York: Academic Press.
Smyth, J. D., Dillman, D. A., Christian, L. M., & Stern, M. J. (2006). Comparing check-all and
forced-choice question formats in web surveys. Public Opinion Quarterly, 70, 66–77.
Stampanoni, C. R. (1993b). Quantitative flavour profiling: An effective tool in flavour
perception. Food and Marketing Technology.
Stampanoni, C. R. (1993a). The quantitative profiling technique. Perfumer Flavourist, 18,
19–24.
Stone, H., & Sidel, J. L. (2004). Sensory evaluation practices. London: Elsevier Academic Press.
Stone, H., Sidel, J. L., Oliver, S., Woolsey, A., & Singleton, R. C. (1974). Sensory evaluation
by quantitative descriptive analysis. Food Technology, 28(11), 24–33.
Sudman, S., & Bradburn, M. B. (1982). Asking questions. San Francisco: Jossey-Bass.
Symoneaux, R., Galmarini, M. V., & Mehinagic, E. (2012). Comment analysis of
consumer's likes and dislikes as an alternative tool to preference mapping. A case
study on apples. Food Quality and Preference, 24, 59–66.
Tarea, S., Cuvelier, G., & Siefffermann, J. -M. (2007). Sensory evaluation of the texture of
49 commercial apple and pear purees. Journal of Food Quality, 30, 1121–1131.
Teillet, E., Schlich, P., Urbano, C., Cordelle, S., & Guichard, E. (2010). Sensory methodol-
ogies and the taste of water. Food Quality and Preference, 21, 967–976.
ten Kleij, F., & Musters, P. A. D. (2003). Text analysis of open-ended survey responses:
A complementary method to preference mapping. Food Quality and Preference, 14,
43–52.
Thomson, D., & Mcewan, J. (1988). An application of the repertory grid method to in-
vestigate consumer perceptions of foods. Appetite, 10, 181–193.
Thuillier, B. (2007). Rôle du CO2 dans l'Appréciation Organoleptique des Champagnes—
Expérimentation et Apports Méthodologiques. Thèse de l'URCA, Reims, France. Avail-
able at http://www.abt-sensory-analysis.com/docs/methode_profil_pivot.pdf.
Veinand, B., Godefroy, C., Adam, C., & Delarue, J. (2011). Highlight of important product
characteristics for consumers. Comparison of three sensory descriptive methods
performed by consumers. Food Quality and Preference, 22, 474–485.
Williams, A., & Arnold, G. (1985). A comparison of the aromas of 6 coffees characterized
by conventional profiling, free-choice profiling and similarity scaling methods.
Journal of the Science of Food and Agriculture, 36, 204–214.
Worch, T., Lê, S., & Punter, P. (2010). How reliable are the consumers? Comparison of
sensory profiles from consumers and experts. Food Quality and Preference, 21,
309–318.