Experiences with the reliability and Validity of the Kano-Method: Comparison to Alternate Forms of Classification of Product Requirements Dr.

Elmar Sauerwein Department of Management/University of Innsbruck Universitsstrae 15 A-6020 Innbruck/Austria Tel. ++43/512/507-7188 or +43/664/3357775 Fax: ++43/512/507-2968 Elmar.sauerwein@uibk.ac.at

ABSTRACT: Reliability and validity of the Kano-Model have not yet been tested thouroughly. This paper tries to examine the reliability of test-retest, alternate forms and stability of interpretation. Furthermore concurrent, predictive and convergent validity were tested. Other methods of classification were tested, too.The results are supportive for the Kano model. Introduction: Kanos model of customer satisfaction Methodology

1.0 1.1

In his model, Kano [1984] distinguishes between three types of product requirements which influence customer satisfaction in different ways when met: Must-be requirements: If these requirements are not fulfilled, the customer will be extremely dissatisfied. On the other hand, as the customer takes these requirements for granted, their fulfillment will not increase his satisfaction. The must-be requirements are basic criteria of a product. Fulfilling the must-be requirements will only lead to a state of "not dissatisfied". One-dimensional requirements: With regard to these requirements, customer satisfaction is proportional to the level of fulfillment - the higher the level of fulfillment, the higher the customers satisfaction and vice versa. Attractive requirements: These requirements are the product criteria which have the greatest influence on how satisfied a customer will be with a given product. Attractive requirements are neither explicitly expressed nor expected by the customer. Fulfilling these requirements leads to
1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

more than proportional satisfaction. If they are not met, however, there is no feeling of dissatisfaction.
Customer satisfied

One-dimensional requirements Attractive requirements

- not expressed - customer tailored - cause delight - articulated - specified - measurable - technical

Requirement not fulfilled

Requirement fulfilled

Must-be requirements
- implied - self-evident - not expressed - obvious

Customer dissatisfied

Figure 1: Kanos model of customer satisfaction [Berger, 1993]

For each product feature a pair of questions is formulated to which the customer can answer in one of five different ways [Kano, 1984]. The first question concerns the reaction of the customer if the product has that feature (functional form of the question), the second concerns his reaction if the product does not have that feature (dysfunctional form of the question).
Functional form of the question 1. I like it that way 2. It must be that way 3. I am neutral 4. I can live with it that way 5. I dislike it that whay

If the edges of your skis grip well on hard snow, how do you feel?

If the edges of your skis do not grip well on hard snow, how do you feel?

1. I like it that way 2. It must be that way 3. I am neutral 4. I can live with it that way 5. I dislike it that whay

Dysfunctional form of the question

Figure 2: Functional and dysfunctional question in the Kano questionnaire By combining the two answers in the following evaluation table, the product features can be classified [Sauerwein et.al. 1996]:
1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Customer requirements
1. like

Dysfunctional (negative) question

2. must be 3. neutral 4. live with 5. dislike

1. like

Q R R R R

A I I I R

A I I I R

A I I I R

O M M M Q

2. must-be

Functional (positive) question

3. neutral

4. live with

5. dislike

Customer requirement is ... A: Attractive M: Must-be R: Reverse O: One-dimensional Q: Questionable I: Indifferent

Figure 3: Kano evaluation table If the customer answers, for example, "I like it that way," as regards "If the edges of your skis grip well on hard snow, how do you feel?" - the functional form of the question, and answers "I am neutral," or "I can live with it that way," as regards "If the edges of your skis dont grip well on hard snow, how do you feel?" - the dysfunctional form of the question, the combination of the questions in the evaluation table produces category A, indicating that edge grip is an attractive customer requirement from the customers viewpoint. If combining the answers yields category I, this means that the customer is indifferent to this product feature. He does not care whether it is present or not. He is, however, not willing to spend more on this feature. Category Q stands for questionable result. Normally, the answers do not fall into this category. Questionable scores signify that the question was phrased incorrectly, or that the person interviewed misunderstood the question or crossed out a wrong answer by mistake. If looking up the answer in the evaluation table yields category R, this product feature is not only not wanted by the customer but he even expects the reverse. For instance, when offering holiday tours it might well be that a specific customer segment wants pre-planned events every day, while another would dislike it [Berger, 1993]. In addition to the Kano questionnaire, it is helpful to have the customer rank the individual product criteria of the current product and to determine the relative importance of the individual product criteria (self-stated-importance). This will help to establish the priorities for product development and make improvements wherever necessary.
1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

1.2.

Interpretation of Results

Mode rule (Most frequent answer)

Product requirement Edge grip Ease of turn Service

Total

Category

7 10.4 63.8

32.3 45.1 21.6

49.3 30.5 2.9.

9.5 11.5 8.5

0.3 1.2 0.7

1.5 1.2 2.5

100% 100% 100%

M O A

Figure 4: Table of results (Data from the ski study, n=1525 [Sauerwein et.al. 1996]) The easiest method is evaluation and interpretation according to the frequency of answers. Thus, edge grip would be a must-be requirement (49.3%), ease of turn a one-dimensional requirement (45.1%) and service of edges and base an attractive requirement (63.8%). Evaluation rule (O+A+M) >< (I+Q+R) When the percentages of Indifferent and e.g. Attractives are quite close (this is quite often the case with Attractives), one may use the follwing rule: If (M+A+O) > (I+Q+R), then Maximum of M or A or O If (M+A+O) < (I+Q+R), then Max (I,Q,R). For example, if 18 answers classify a requirement as One-Dimensional, 19 as Attractive, 18 as Must-Be, 20 as indifferent, 2 as Reverse and 3 as Questionable, then the mode rule would classify the requirement as Indifferent, although 57 out of 82 people say they need it in one or the other way. Applying this rule would classify the requirement as Attractive, because the sum of 18+18+19 is greater than 20+2+3. Evaluation rule M>O>A>I If the individual product requirements cannot be unambiguously assigned to the various categories, the evaluation rule "M>O>A>I" is very useful. When making decisions about product developments, primarily those features have to be taken into consideration which have the greatest influence on the perceived product quality. First those requirements have to be fulfilled

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

which cause dissatisfaction if not met. When deciding which attractive requirements should be satisfied, the decisive factor is how important they are for the customer. Customer satisfaction coefficient (CS coefficient) The customer satisfaction coefficient states whether satisfaction can be increased by meeting a product requirement, or whether fulfilling this product requirement merely prevents the customer from being dissatisfied [Berger et al., 1993]. The CS-coefficient is indicative of how strongly a product feature may influence satisfaction or, in case of its non-fulfillment customer dissatisfaction. Extent of satisfaction: A+O A+O+M+I Extent of dissatisfaction: O+M (A+O+M+I) x (-1) Product requirement Edge grip Ease of turn Service A 7 O M I Total Category M O A A+O O+M A+O+M+I A+O+M+I 0.40 0.57 0.89 -0.83 -0.78 -0.25

33 50 10 100% 3 9 100%

11 46 31 12 100% 66 22

Figure 5: Table of CS-Coefficients (Data from the original ski study, n=1525 [Sauerwein et.al. 1996])

Category and Total Strength Category and Total Strength are another rules for interpreting close results [Lee/Newcomb 1996] Category strength = Most frequent answer secondmost frequent answer This formula shows us, wether the categorization is clear or not. In figure 6 we see, that there is a clear categorizaton of edge grip (14,6 %), whereas the category strength of Integrated Antitheft device is very small (0,2 %). The thumb rule is, when category strength is greater than 5 %, the requirement is considered clearly classified.

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Total Strength = M+O+A This formula shows us, wether more than 50 % of a sample regard the requirement as important or not. Both rules allow the prioritization of the fulfillment of requirements.

Item Edge grip

R 1,2

Q 1,2

Total

Strength Categ. O

10,4 45,1 30,5 11,5

100% Cat = 14,6 Tot = 86,1

Regular up-to-date information

40,3 15,7

4,0

35,9

3,2

1,1

100%

Cat = 4,4 Tot = 89

Mixed

Integrated Anti-Theft 40,2 14,5 device

2,2

40,4

1,8

0,8

100%

Cat = 0,2 Tot = 57

Mixed

Figure 6: Example for Category and Total Strength

2.0 2.1.

Reliability of the Kano Model Test-Retest-Reliability

A questionnaire concerning product features of skis was filled out twice by 120 Business Students with an intervall of two weeks between test and retest. Correlation coefficients were calculated for the two sets of responses. The percentage of correspondence was calculated, too [Sauerwein, E 1998]. Reliability of Test-Retest is only moderate according to classic test theory. (see figure 7). Coefficients of correlation range from .581 to .771 for the Kano categories. Coefficients of greater than 0.5 are considered moderate, coefficients of greater than 0.7 are considered good.

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Product requirement Edge grip on hard pists Great ease of turn Good powder snow features Very light skis Scratch-resistant surface Design of skis and boots Integrated anti-theft device Free service on edges and base Trade-in offer for old skis Regular up-to-date information

functional .634 .530 .433 .661 .523 .456 .540 .607 .448 .536

Dysfunct. P .736 .619/ .551 .631 .717 .585 .546 .263 .496 .365

Kano category .708 .606 .581 .771 .616 .692 .661 .595 .631 .660

Figure 7: Test-Retest Correlation coefficients of functional, dysfunctional questions (Pearsons) and for Kano categories (contingency coefficients) (all correlations highly significant p=0.01) 2.2. Stability of Interpretation

But although the classifications referring to the individual answers have only moderate reliability, stability of interpretation for a requirement as a whole is extraordinary stable, when all answers a cumulated: There were three product features where percentages varied substantially fromt est to retest. But only one product feature changed its categorization: Very light skis was classified as Indifferent in the first run and changed to Attractive in the second run. (see figure 8).

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Great ease of turn Category Attractive Must-be One-dim. Indifferent Reversal Quest. Sum 100,00 Test 18,35 28,44 34,86 17,43 0,92 0,91 100,00 Retest 21,82 10,91 47,27 19,09

Good powder snow features Test 37,38 2,80 30,84 28,97 Retest 35,45 2,73 20,91 40,00 0,91

Very light skis

Regular up-todate information

Test 22,22 11,11 25,00 37,04 4,63

Retest 36,36 6,36 24,55 30,00 1,82 0,91

Test 35,5% 1,8% 3,6% 57,3% 1,8%

Retest 42,7% 0,9% 7,3% 46,4% 1,8% ,9%

100,00

100,00

100,00

100,00

100,00

100,00

Great ease of turn Rules Mode Total Str. Cat. Str Test M 81,65 6,42 Retest M 80,00 25,45

Good powder snow features Test A 71,03 6,54 A Retest I 59,09 4,55 I 0,39

Very light skis

Regular up-todate information

Test I 58,33 14,81 I 0,35

Retest A 67,27 6,36 A 0,44

Test I 40,9 21,8 I 0,40

Retest I 50,9 3,7 I 0,51

A+O/ A+O+M+I -(M+O/ (A+O+M+I) O+A+M I+R+Q

0,47

0,33

0,40

-0,64

-0,59

-0,34

-0,24

-0,38

-0,32

-0,06

-0,08

81,65 18,35

80,00 20,00

71,03 28,97

59,09 40,91

58,33 41,67

67,27 32,73

40,9 59,1

50,9 49,1

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Category

Figure 8: Comparison of Interpretation between test and retest This is a very positive result for the practicioner, because it ensures him to get trustworthy results despite the only moderate test-retest-reliability. Alternative form reliability was tested too, using answering combinations of functional and dysfunctional questions. This form could not deliver better reliability results. Classification by using the means (mode, median, arithmetic mean) of functional and dysfunctional question was also tested, but could not deliver better results. The conclusion is, there is currently no alternative to the conventional well-known form of the KanoMethod. 2.3. Alternate forms reliability

In order to evade the reliability problems by combining functional and dysfunctional questions an alternate form was created by combining the two questions in one. 110 Students ahd to anser the conventional form of the Kano Questionnaire and the combined form simultaneously. Please categorize the Product requirements considering the following categories: A (ttractive): I do not expect the fulfillment and I will not be dissatisfied, when it is not fulfilled. But I would be very satisfied, when the requirement is fulfilled. O (One-Dimensional): The better the requirement is fulfilled, the more satisfied I am and vice versa. M (Must Be): I expect it and will not be satisfied, when it is fulfilled, but will be very dissatisfied,, when it is not fulfilled. I (ndifferent): I have no use for it, I am neutral R (eversal): I expect the opposite. Figure 9: Alternate form of the Kano questionaire The procentual correspondence between the two forms is not good:

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Requirement Edge grip on hard pists Great ease of turn Good powder snow features Very light skis Scratch-resistant surface

% 41,0 34,4 47,5 42,6 52,5

Requirement Design of skis and boots Integrated anti-theft device Free service on edges and base Trade-in offer for old skis Regular up-to-date information

% 67,2 59,0 67,2 59,0 65,6

Figure 10: Correspondence of categorization between conventional and alternate form Astonishingly the stability of interpretation is very good: Only one requirement (ease of turn) changed its categorization from One Dimensional to Must be. This fact should encourage further research in this direction. 3.0 Validity of the Kano Model

3.1. Content Validity Validity was tested using a survey with 1450 ski customers in Austria and South Tyrol. Kano states, that there is a hierarchy of product classifications (concerning its importance for the customer) Indifferent -Attractive - One-Dimensional - Must-Be. This could be strongly confirmed: Importance ratings of people, who classified a certain feature as Must Be were compared to importance ratings of people, who classified the feature as One-Dimenional, Attractive, Indifferent, respectivily. Must-Bes were rated most important, One-Dimensionals were of second importance, Attractives of third and Indifferents of least importance to the customers (see fig. 11). The significance of the differences was calculated by comparing the means using a one-factorial analysis of variance and the differences were almost all significant. Product feature Edge grip on hard pists Great ease of turn Good powder snow features I 5,778 5,634 3,273
th

A 6,276 5,839 4,440

O 6,452 6,361 4,943

M 6,617 6,385 5,306

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Very light skis Scratch-resistant surface Design of skis and boots

3,784 3,031 2,668

4,883 4,310 4,073

5,649 5,239 4,97

5,378 5,182 5,393

Figure 11: Arithmetic Means of importance (1= very unimportant 7 very important, n=1450) 3.2. Concurrent Validity Product feature Edge grip on hard pists Great ease of turn Good powder snow features Very light skis Scratch-resistant surface Design of skis and boots 5x M Conf. Conf. Conf. Conf. Conf. Conf. 2x Conf. Conf. Conf. Conf. 4x Conf. O A

Figure 12: Confirmation of Kano curve shapes Concurrent validity could be supported through nonlinear regression analysis with the satisfaction with a requirement as independent variable and overall satisfaction as dependent variable. The SPSS-procedure Curve-Fit [Bhl, A./Zfel; P., 1996] was used to determine the nonlinear equation with the highest coefficient of determination in order to find the highest possible explanation for the relationship between overall satisfaction and the satisfaction with a certain product feature. All respondents and each group of respondents separately (Atttractives, MustBes, etc.) were regressed with overall satisfaction. Thereupon the curve shapes of this equation were compared to the Kano curve shapes. The impact of Attractives on overall satisfaction is disproportionate compared to OneDimensionals and Must-Bes in 4 cases out of 6. One-Dimensionals could only be confirmed 2x having a quite straight proportional impact and Must Bes have a less than proportional impact on overall satisfaction in 5 cases. (see fig. 12). Therefore, in 61,1 percent of the equations the Kano curve shapes could be confirmed. As an example see figure 13.

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Ge s a m t/P O W /R s q. = 0 ,4 2 4

Attr ./Q UA/R s q. = 0 ,6 2 8

4 ,5 4 4 3 ,5 3 3 2 ,5 2 2 1 ,5 1 1 0 ,5 0 1 2 3 4 5 0

M -B/P O W /R s q. = 0 ,4 0 9

O -D/CUB/R s q. = 0 ,4 4 3

I n d . / S / R sq . = 0 , 6 0 4

5 4 3 2 1

0
0 1 2 3 4 5

figure 13: Curve shapes concerning the requirement Very light skis 3.3. Predictive Validity Customers, who classified a requirement as Attractives have a significant higher repurchase rate in comparison to customers classifying the requirement as Must-Bes or One-Dimensionals concerning four product features. So, Attractives can be decisive for the purchase of a product, but the evidence is not very strong. Product feature Edge grip on hard pists Great ease of turn Good powder snow features Very light skis Scratch-resistant surface Design of skis and boots Mean A 73,9 63,9 77,0 68,8 73,5 70,0 71,2 O 62,2 66,6 66,4 64,3 67,3 67,3 65,7 M 70,3 66,3 67,7 69,2 56,4 56,3 64,4

Figure 14: Percentages of repurchase

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

4.0

Implications for further research

Evidence for the reliability and validity is confirmed, but only moderate. Further studies have to be conducted to confirm the usability of the Kano method. Especially the Kano method is to be compared with other methods of classifying product features.

References Berger, C.; Blauth, R.; Boger, D.; Bolster, C.; Burchill, G.; DuMouchel, W.; Pouliot, F.; Richter, R.; Rubinoff, A.; Shen, D.; Timko, M.; Walden, D. [1993]. Kanos Methods for Understanding Customer-defined Quality, Center of Quality Management Journal, vol. 2, no. 4, pp. 3-36, Fall Bhl, A.; Zfel, P. [1993]. SPSS for Windows Version 6.1, 2. ed., McGraw-Hill, New York Kano, N.; Seraku, N.; Takahashi, F.; Tsuji, S. [1993]. Attractive Quality and Must-Be Quality, Quality (Hinshitsu): The Journal of the Japanese Society for Quality Control, vol. 14, no. 2, pp. 39-48, April Lee, M.C:; Newcomb, J. [1996]. Applying the Kano Methodology in Managing NASAs Science Research Program, Center of Quality Management Journal, vol. 5, no. 3, pp. 13-20 Sauerwein, E.; Bailom, F.; Hinterhuber, Hans H.; Matzler, K. [1996]. Kanos Model: How to delight Your Customers, Preprints Volume I of the IX. International Working Seminar on Production Economics, Innsbruck/Igls, Feb 16th - 20th, ed.: Institute of Technology, Linkping, 1996, Igls/Innsbruck, pp. 313 327 Sauerwein, E [1998], Reliabilit und Validitt des Kano-Modells zur Klassifizierung von Produkteigenschaften (Reliability and Validity of the Kano Modell for categorization ot product requirements, in German), Innsbruck, 1998 Biography: Dr. Elmar Sauerwein is Assistant Professor at the Department of Management/University of Innsbruck/Austria. His major research areas: Quality Management, Customer Satisfaction, Risk Management, Health and Hospital Management. His professional experience before is current assignment is: Public Relations Affairs at the Tyrolean Peoples Party (Conservative Party), Consultant in computer affairs and statistical analysis for market research Foundation member and sales manager of Nova Computer & Management
1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA

Currently he holds the following positions: Deputy representative of the Alpine Society for further education in health service Member of the board of the Society of Strategic Management at the University of Innsbruck Organizational Director of the Post-Graduate-Course "International Hospital Management"/University of Innsbruck

1999 QFD Institute, Elmar Sauerwein; Transactions of the 11 th Symposium on QFD, June 12-18, 1999, Novi Michigan, USA