The current issue and full text archive of this journal is available at www.emeraldinsight.com/1468-4527.

htm

OIR 34,3

420
Refereed article received 27 June 2009 Approved for publication 23 November 2009

The relative importance of usability and functionality factors for online auction and shopping web sites
Fethi Calisir, A. Elvan Bayraktaroglu, Cigdem Altin Gumussoy, Y. lker Topcu and Tezcan Mutlu I
Department of Industrial Engineering, Istanbul Technical University, Istanbul, Turkey
Abstract
Purpose The purpose of this study is to determine the relative importance of the usability and functionality factors for online auction and shopping web sites from the perspective of young consumers. Design/methodology/approach The Analytical Network Process (ANP) was used to analyse the relative importance of the usability and functionality factors. Findings The ndings show that customers of online auction and shopping web sites give higher priority to usability and usability factors than to functionality, with navigation and interaction being the factors of highest relative importance. Practical implications First, customers assign higher priority to usability factors. Thus, early in the design phase, designers should attach more importance to navigation- and interaction-related aspects of their web sites. Second, designers of web sites should include basic and advanced search options to increase the number of the web site users. Originality/value The study is the rst to examine the relative importance of the usability and functionality factors of online auction and shopping web sites. Furthermore, the study lls this gap by identifying the complex interrelationships between various features of usability and functionality, and their impact on each other, by examining two Turkish online-auction and shopping web sites. Keywords Electronic commerce, Turkey, Internet, User studies, Youth Paper type Research paper

Online Information Review Vol. 34 No. 3, 2010 pp. 420-439 q Emerald Group Publishing Limited 1468-4527 DOI 10.1108/14684521011037025

Introduction E-commerce can be described as the purchase and sale of products and/or services through electronic media such as the internet (Wen et al., 2001). Online auction and shopping web sites are a popular e-commerce application. An auction is a market institution with an explicit set of rules around offering to sell and bidding to buy (McAfee and McMillan, 1987). An online auction is where the negotiation between buyers and sellers is conducted via an online marketplace open to all participants (Klein and Quelch, 1997). Online auction sites may also serve as shopping web sites where products may be bought at a xed price. Online auction and shopping web sites offer many advantages to buyers as well as sellers. First, geographically dispersed buyers and sellers can reach each other through these sites. Second, for sellers, auctions may be an alternative and less expensive distribution channel to sell difcult products such as perishable products (McCart et al., 2009). Additionally, online auction and

shopping web sites offer customers some benets like product promotion, cost saving, timely information, shortened remittance time, information consistency, better customer service, better customer relationship, customization of products, competitive advantages, convenience of doing business (Wen et al., 2001, p.5). In e-commerce, the web site becomes the storefront and interaction between sellers and buyers. It replaces the face-to-face communication that takes place between the two parties in traditional commerce (Park and Kim, 2003). Bontis and De Castro (2000) point out that the design of a web site is the online consumers gateway to the organization (p. 367) and is one of the important concerns of online shoppers, so the success of e-commerce depends largely on user interface design. As the web site interface is the foundation of e-commerce success, a prot-focused e-organisation tries to design a web site that attracts web users, responds to their expectations, induces them to become regular visitors of the web site and converts them into customers. Chen and Tan (2004) stress that a poorly designed web site affects the visitors shopping experience negatively; hence, in todays competitive environment, it is vital for survival that a web site becomes preferred and accepted by the majority. According to Lu and Yeung (1998), functionality and usability are the sub-features of usefulness, which is a critical factor of acceptability of a system. Users tend to use functional and usable products more frequently. It is also possible that a functional web site is not usable or vice versa (Yeung and Law, 2004; Lu and Yeung, 1998; Nielsen, 2003). Only functionality-focused designs fail to meet usability needs (Klein and Seffah, 2005), therefore both usability and functionality, which in web design are interrelated, should be taken into account in the design processes (Weir et al., 2007). Which factors of usability and functionality are more important for using online auction and shopping web sites remains ambiguous. Therefore, this study aimed to examine the relative importance of both usability and functionality factors for online auction and shopping web sites from the perspective of young consumers in Turkey. Furthermore, this study aimed to identify the complex interrelationships between various features of usability and functionality and their impact on each other by examining two Turkish online auction, and shopping web sites (gittigidiyor.com and hemalhemsat.com). These two online auction and shopping web sites, Hem Al Hem Sat and Gitti Gidiyor, are the most popular Turkish online auction and shopping web sites. They offer the same or similar product categories to customers. In 2008, the Turkish Board of I Statistics (TUK) reported that in Turkey the internet usage rate by individuals aged between 16 and 74 was 35.8 per cent. The age group with the highest internet usage is 16 to 24 year olds followed by the 24 to 35 age group. According to the rst three months statistics for 2008, the percentage of individuals who purchased goods or services via the internet was 7.2 per cent. In addition, according to the Turkey Information Technology Report Q4, Turkeys e-commerce transactions increased to 54.1 million in 2007 from 21.7 million the previous year (BMI, 2009). These statistics show that Turkey offers a vast market for e-commerce activities aimed at young citizens who are willing to adopt information technologies. The next section discusses the usability and functionality factors examined, generated through literature review and expert opinions. This is followed by the evaluation of the online auction and shopping web sites via the Analytical Network Process (ANP) application. The paper concludes with a discussion of the ndings, resulting in a model and recommendations for future research.

Usability and functionality factors 421

OIR 34,3

422

Factors of usability and functionality Usability refers to the extent to which a web site facilitates users to utilise its functions easily and appropriately. Usability is an important factor in commercial web site success. Palmer (2002) suggests that B2C web site success is strongly related to usability, alongside design and media richness (Thelwall, 2000). Usability also plays an important role in gaining and sustaining the trust of customers (Roy et al., 2001; Casalo et al., 2007). Functionality refers to the extent to which the web site operates in the way it is structured and is expected to perform as users desire (Bertot et al., 2006; Nielsen, 2003). The functionality of a web site becomes obvious to users through its interface and users interact with the system via this interface. Although it would appear that the usability feature is related only to the interface and not to the logic of available functions, Seffah et al. (2008) assert that internal attributes (i.e. functionality) of a system can affect the usability of the whole system. Functionality can have a positive impact on the usability of the system, such as a back function (Bass and John, 2003). After all, evidence shows that the majority of users usually use only a small part of the available functions of a system (Furnell, 2005). At some point, the greater the functionality offered to the users, the more skilled users must become to cope with the complex and time-consuming structure of the system (Kavadias et al., 2007). In their research on design factors affecting the usability of web sites, Tarafdar and Zhang (2005) found that, unlike other factors, security and customisation inuence usability negatively because of complicated functionality. In their research on the functionality of commercial web sites, Benslimane and Yang (2007) determined that adding advanced functionalities to a full-edged web site has a negative effect on web site efciency. Arguing on the usability of integrated software development environments, Klein and Seffah (2005) stated that usability problems faced by programmers were due to functionality-focused programme designs. Therefore, superuous functionality can actually bring about a decrease in the usability of the web site (Saarloos et al., 2008; Furnell, 2005). To ensure that users needs are met, it is critical to balance functionality and usability in the design of the web site. Evaluating web sites through functionality and usability factors can help organisations improve and update their targets, interface structures and services (Bertot et al., 2006). An evaluation that combines both usability and functionality features enables them to provide the right degree of trade off between usability and functionality that generally meets the expectations of users. The relative importance of usability and functionality factors would reect the expectations of the users. The factors of usability and functionality examined in this study were selected and classied on the basis of a literature review and the personal judgement of the expert authors. These factors are summarised in Table I. Usability factors The usability factors examined in this study were navigation, interaction, learnability, ease-of-use, response time, memorability, efciency and satisfaction. Navigation. This refers to nding ones way to the desired information through menus, graphical components, links and page sequence, and layout (Palmer, 2002) while continuing to know where one is in the site (Roy et al., 2001). A web site requires easy navigation (Nel et al., 1999). Users should be able to see from the rst page what is

Factors Usability factors Navigation

Denition This refers to nding ones way to the desired information through menus, graphical components, links and page sequence, and layout (Palmer, 2002) as well as, even while doing this, knowing where one is in the site (Roy et al., 2001) This refers to responses produced by the system to the users actions (Palmer, 2002) and the interaction between the system and user This is associated with the skill levels of a web site user and thereby the level of effort needed to learn how to operate the system (Calero et al., 2005) This refers to being able to operate a web site without experiencing any difculty and trouble This is the time needed by the system to respond to the activity of a user (Palmer, 2002) This is the ease of recall of the main functions and their presentation on the web site when a user revisits the page (Nielsen, 2003) This is the ability of the web site to allow users to work quickly (Reilly et al., 2003; Nielsen, 2003) to attain their desired goal with the minimum number of clicks This is the general pleasure a user feels when using a web site The security features provided by the web site to protect customers privacy Web sites offer both simple and advanced search strategies and enable additional eliminations in retrieved results (Bertot et al., 2006; Kapoor and Goyal, 2007) This supplies customers with adequate information about a company, and its products and services (Waite and Harrison, 2007) These are purposive services or facilities offered to the customer to assist in achieving the related goal of the site The web site offers customers uncustomisable (such as FAQ) and/or customisable help (such as online help) (Waite and Harrison, 2007), and describes the necessary information about these steps which users can follow when they have a request or when they encounter a problem This is the exibility to change web site navigation to a level that meets users needs or preferences

Usability and functionality factors 423

Interaction Learnability Ease of use Response time Memorability Efciency Satisfaction Functionality factors Security Search options Information provision Services/facilities User guidance or support

Customisability

Table I. Factors related to usability and functionality, and their denitions

available on the site and decide what actions they have to take to achieve their desired result (Thelwall, 2000). Navigation has been includeded in several studies as a factor under usability (Chakraborty et al., 2005; Lavie and Tractinsky, 2004; Casalo et al., 2007; Roy et al., 2001; Thelwall, 2000; Pearson et al., 2007; Bertot et al., 2006; Palmer, 2002). In their research on trustworthiness of web retailers, Roy et al. (2001) found that navigation is related to the dependability of the retailer and has an impact on the perceived ability and perceived goodwill of the vendor. According to Turban and Gehrke (2000), customers of commercial web sites prefer to shop from navigable sites. Helpful navigation reduces the cognitive load on the user visiting a web site (Tarafdar

OIR 34,3

424

and Zhang, 2005) and the web sites success is related to the navigability of the site (Palmer, 2002). Interaction. Responses to the users actions are produced by the system (Palmer, 2002). Along with navigability, interaction with the site helps users to easily nd the desired information on a web site (Chakraborty et al., 2005). Ghose and Dou (1998) suggest that interactivity is a component which has a fortifying effect on the attractiveness of an e-commerce web site. Interaction was a usability factor in the research of Chakraborty et al. (2005) and Palmer (2002). In our research, interaction did not include customisability. Learnability. This is associated with the skill levels of web site users and thereby the level of effort needed to learn how to operate the system (Calero et al., 2005). For success of a web site, the time a user needs to learn how to operate the system should be very short (Nielsen, 2000a). Consistent, logical and intuitive interface design, simple and plain language, etc. are the key steps to a learnable web site. Learnability is related to the reliability of a commercial web site (Roy et al., 2001) and has been included in several studies as a usability factor, sometimes called ease of learning (Chakraborty et al., 2005; King and Jannik, 2005; Roy et al., 2001; Reilly et al., 2003; Calero et al., 2005; Ferreira and Pithan, 2005). Learnability is about the cognitive load imposed on a user when he/she accesses a web site for the rst time. Thus, as noted by Tarafdar and Zhang (2005) good navigation is critical to reduce the cognitive load, and it can be said that the navigation and learnability of a web site are related. Ease of use. This refers to being able to operate a web site without experiencing any difculty and trouble. Several studies have treated ease of use as a usability factor for web sites (Chakraborty et al., 2005; Lavie and Tractinsky, 2004; Casalo et al., 2007; Pearson et al., 2007). Ease of use is an important criterion for web users as it affects web sites effectiveness and user sentiment towards the web site (Chakraborty et al., 2005). According to Pearson et al. (2007) ease of use is one of the most important components along with navigation for usability of web sites, and plays a signicant role in the adoption of a web site by users. It is a much more important factor for female web users than for male web users. Tarafdar and Zhang (2005) and Palmer (2002) specify the navigation mechanism to be one of the main design elements for easy operation of the site. Response time. This is the time needed by the system to respond to the activity of a user (Palmer, 2002). Response time is considered a feature of usable web interfaces (Nielsen, 1993). Short download times are essential to provide short response times (Nielsen, 2000b). Minimal download times are a critical consideration in preventing users getting bored and switching to another web site (Thelwall, 2000). The shorter a sites response time the more the customers perceive the site as high quality and the more gratifying they nd the experience (Novak et al., 2000). Although download time is greatly affected by the amount of trafc on the net and the speed of the users modem, Thelwall (2000) suggests that minimising the aggregate size of the les generating the web page would help shorten the download time. Pearson et al. (2007) indicate that the maximum wait time acceptable for a page to download is only a few seconds. Where wait times exceed the acceptable level, users should be informed of the progress of the download process (Nielsen, 1993). Memorability. This is the ease of recall of the main functions and their presentation on the web site when a user revisits the page (Nielsen, 2003). According to Ferreira and Pithan (2005) a web site that is constructed in conformity with the model visualised by

users is expected to have higher memorability because users tend to remember structures that appear logical to them much better. Seffah et al. (2008) point out that an inconsistent interface structure raises the memory load on users. In studies by Ferreira and Pithan (2005) and King and Jannik (2005) memorability was included as a usability factor. Efciency. This is the ability of the web site to allow users to work quickly (Reilly et al., 2003; Nielsen, 2003) to attain their desired goal with the minimum number of clicks. In their research, Ferreira and Pithan (2005) point out that inefciencies caused by technical errors or poor functionalities of the site can make users feel insecure and disappointed, and such feelings affect their satisfaction negatively. According to De Marsico and Levialdi (2004), navigation by offering logical and suitable links and paths through pages of a web site ensures users an experience of efcient usage and thereby strongly inuences the efciency of the web site. King and Jannik (2005), Reilly et al. (2003) and Ferreira and Pithan (2005) consider efciency as one of the usability factors. Satisfaction. This is the general pleasure a user feels in using a web site. Satisfaction is primarily affected by the perceived efciency and effectiveness, and emotions and thoughts arising from the usage of the web site (Ferreira and Pithan, 2005). Satisfaction may be an especially critical factor for non-compulsory usage of web sites, because it is generally expected that users tend to revisit a web site when they nd it satisfactory on their rst visit (Ha and Janda, 2008). In several studies (King and Jannik, 2005; Reilly et al., 2003; Ferreira and Pithan, 2005) satisfaction was included as a usability factor. Functionality factors Functionality factors examined in this study were security, search options, information provision, services/facilities, user guidance or support, and customisability. Security. The security features provided by the web site protect customers privacy. Consumers concerns over the security of a site have an impact on their decision to use a commercial web site. Web sites usually accumulate customers personal information to service them appropriately and to determine the customers site prole (Yang et al., 2005). Accordingly, keeping safe the personal information collected becomes an important consideration in rendering the web site trustworthy. Security is one of the major concerns of commercial web site customers (Bontis and De Castro, 2000). Calero et al. (2005), Stefani et al. (2006) and Seffah et al. (2008) included security as a functionality factor in their research. Search options. Web sites offer both simple and advanced search strategies and enable additional eliminations in retrieved results (Bertot et al., 2006; Kapoor and Goyal, 2007). The search function helps users nd, quickly and precisely, what they are looking for. Nielsen (1999) suggests that web sites with over 200 pages should have a search component. Bertot et al. (2006), Kapoor and Goyal (2007) and Wilson et al. (2002) all named search options a functionality feature in their research. Information provision. This supplies customers with adequate information about a company, and its products and services (Waite and Harrison, 2007). Yang et al. (2005) put forward adequacy of information as a factor of service quality because users need adequate information to identify the products and services offered on a commercial web site. Services/facilities. These are purposive services or facilities offered to the customer to assist in achieving the related goal of the site (Kapoor and Goyal, 2007).

Usability and functionality factors 425

OIR 34,3

426

User guidance or support: Web sites offer customers uncustomisable (such as FAQs) and/or customisable help (such as online help) (Waite and Harrison, 2007), and describe the necessary information about these steps which users can follow when they have a request or when they encounter a problem. According to Roy et al. (2001), an adequate user guidance and support feature would reduce their cognitive load and pave the way for them to learn how to operate the web site. It also has a positive effect on perceived ability and, related to this, on trustworthiness of the organisation responsible for the web site. User guidance or support was used as a functionality factor in Waite and Harrisons (2007) research. Customisability. This is the exibility to change web site navigation to a level that meets users needs or preferences. Palmer (2002) points out that customisation is an extension of the interaction provided by the web site. Liang et al. (2007) suggest that customisation increases user satisfaction by limiting information overload on users with respect to their preferences (Pearson et al., 2007). However, this would only work for users who know what kind of information they want to obtain from the web site, not for web site surfers. According to a study conducted in 2003 (Huang), only 6 per cent of corporate web sites have the customisability function. Huang also declared that by accumulating personal preference information about the users, organisations would be able to present personalised web site options according to their customers choices. Methodology The Analytical Network Process (ANP) was used in this study. ANP is a methodology that allows groups or individuals to deal with the interconnections (of dependence and feedback) between factors of complex structure in the decision-making process (Saaty, 1980, 1996). The ANP model consists of a network of factors of the problem and the logical groupings of these factors into clusters. Relationships between factors within the same cluster are called inner dependencies, whereas links between an element in one cluster and another one in a different cluster are called outer dependencies (Saaty, 2005). If there are outer dependencies between two clusters in both directions, this relation is called feedback. Inner and outer dependencies are the best way decision-makers can capture and represent the concepts of inuence between factors. Pair-wise comparisons are made systematically for all combinations using the fundamental comparison scale (1-9) of the Analytic Hierarchy Process (AHP) that is used to indicate how many times an element dominates another. The decision-maker can express relative dominance between each pair of elements verbally as equally important, moderately more important, strongly more important, very strongly more important and extremely more important. These descriptive judgments are then translated into numerical values 1, 3, 5, 7, 9 respectively, with 2, 4, 6, and 8 as intermediate values for comparisons between two successive points. The ANP, the general form of AHP, does not require a strictly hierarchical structure and therefore can treat problems that have complex interrelationships among factors, enabling it to cope with the complexities of real world problems for making societal, governmental and corporate decisions (Saaty, 2003; Jharkharia and Shankar, 2007; Shyur and Shih, 2006). Unfortunately, ANP applications have been noticeably limited when compared with AHP due to its complexity and time-consuming nature. So far the ANP approach has proven successful when expert knowledge is used within business,

social and manufacturing decision contexts or used to predict economic turns (Burnaz and Topcu, 2006). As mentioned, not only are functionality and usability of a web site interrelated, but some of the sub-factors are also related to each other. These interactions go to create a complex model composed of dependence and feedback among the factors. In evaluating online auction and shopping web sites, such a model can be treated with the Analytic Network Process (ANP) proposed by Saaty (1996) in order to determine the relative importance of both usability and functionality factors. First, at the structuring stage, the usability and functionality factors were determined. Then three experts in usability engineering lled in a pair-wise relationship matrix separately. Then these matrixes were aggregated into a single matrix using majority rule (see Figure 1). The entries in this matrix represented the existence of a direct relationship of factor i to factor j: if factor i affects factor j, the entry aij was marked with an asterisk ( *). Where there was no relationship, the entry was left blank. As the possible relationships between a pair of factors may or may not be symmetrical, the whole matrix should be lled. Then, utilising the design module of the Super Decisions software, n.d., the authors constructed the ANP model representing the relations among factors (see Figure 2). In the assessment stage of the process, young Turkish consumers of the two online auction and shopping web sites, Hem Al Hem Sat (hemalhemsat.com) and Gitti Gidiyor (gittigidiyor.com), completed a questionnaire in paper format. The questionnaire consisted of three main parts. The rst part covered demographic questions designed to solicit information about the respondents. The second part asked the respondents to make pair-wise comparisons of the factors related with functionality and usability according to the relationship matrix structured by the experts. The third part asked the respondents to compare the two web sites based on each factor. An element of the questionnaire relating to the second and third part can be seen in Figure 3. In total, 60 questionnaires were collected from the undergraduate students of Istanbul Technical University. The demographic proles of the respondents can be seen in Table II. The average age of the respondents was 22.9 years and most were male. The majority of the

Usability and functionality factors 427

Figure 1. Aggregated pairwise relationship matrix

OIR 34,3

428

Figure 2. The analytic network model

Figure 3. A part of pairwise comparison questionnaire

students had an income of more than 1,000 TL per month and most lived far away from their families. All of the respondents were internet users, and the average internet use per week was 24.8 hours per week. They were familiar with using e-commerce web sites, with an average frequency of two purchases over the internet and an average online shopping expenditure of 118.6 TL per month. It should also be noted that all of the respondents had prior experience with both the Hem Al Hem Sat and Gitti Gidiyor web sites. The use of student samples in research has been a source of concern for a long time (e.g. Sackett and Larson, 1990). The most cited problem has been that a student sample may not fully represent the entire population. As a result, student sample studies lack external validity. However, there might be situations where a student sample is appropriate. For example, college students have been a protable group for online marketers since they comprise one of the largest internet user segments (Davis, 1999). A recent study by Biswas and Biswas (2004) also supports the use of student samples since they mirror closely the typical internet user. Therefore, the use of a student sample in this study was deemed acceptable given the nature of the study and their literacy and use of the internet.

n Age Max Min Average Gender Male Female Monthly income TL (%) 0 . . . . # 250 250 . . . . # 500 500 . . . . # 750 750 . . . . # 1,000 $1,000 Residence (%) At home with family Dormitory At home without family Internet usage per week (hour) Max Min Average The average of purchasing frequency through internet per month Max Min: Average The average of total expenses through internet per month (TL) Max Max Min Average 25 18 22.9 52 8 0 14.3 8.6 22.9 54.3 34.3 42.9 22.9 45 12 24.8 5 1 2.03 300 183 35 118.6

Usability and functionality factors 429

Table II. Demographic proles of the respondents

The respondents judged the relative importance of the affecting nodes (factors and alternatives) on the affected node for all possible pairs. Then the geometric means of all paired comparison judgements for each question were computed in order to arrive at the aggregated group judgements. Using the assessment/comparison module of the Super Decisions software, these responses were arranged in pair-wise comparison matrixes. In the nal stage of the process, the relative importance of the factors was computed using the computations module of the same ANP software, which performed all the algebraic computations of the matrix. Details of these computations for interested readers are as follows. The computed eigenvector of the affecting nodes with respect to the affected node is placed to the column representing the affected node and the rows representing the affecting nodes. This matrix is called as supermatrix (the supermatrix formulated in this research is given in Table III). If the column sum of any column in the composed supermatrix is greater than one (i.e. there is more than one eigenvector),

OIR 34,3

430

gittigidiyor hemalhems B1 security B2 search B3 informa B4 services B5 user gui B6 customi A1 navigati A2 interact A3 learnab A4 ease of A5 respons A6 memori A7 efcien A8 satisfac

Table III. Supermatrix

B2 search 0.64413 0.35587 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.25891 0.13407 0.07639 0.53063 0.00000 0.00000 0.00000 0.00000 0.69231 0.30769 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.81308 0.18692 0.16667 0.83333 0.00000 0.00000 0.00000 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.71910 0.28090 0.00000 0.00000 0.65398 0.00000 0.00000 0.34602 0.00000 0.74874 0.00000 0.00000 0.00000 0.25126 0.00000 0.00000 0.69605 0.30395 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.21242 0.55863 0.00000 0.22895 0.00000 0.00000 0.75124 0.24876 0.05094 0.32916 0.10293 0.05305 0.34256 0.12136 0.00000 0.57344 0.00000 0.00000 0.11803 0.30853 0.00000 0.00000 0.72603 0.27397 0.04952 0.12108 0.11246 0.47469 0.24226 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.70501 0.29499 0.00000 0.10844 0.14141 0.27664 0.39355 0.07996 0.07051 0.11529 0.00000 0.47082 0.06141 0.28196 0.00000 0.00000 B3 B4 B5 user B6 A1 A2 A3 A4 ease A5 A6 A7 A8 informa services gui customi navigati interact learnab of respons memori efcien satisfac 0.81203 0.18797 0.00000 0.12581 0.47114 0.20164 0.162333 0.03907 0.21527 0.55180 0.13951 0.00000 0.00000 0.09342 0.00000 0.00000 0.74359 0.25641 0.15966 0.84034 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.76526 0.23474 0.00000 0.19130 0.15474 0.40943 0.18230 0.06222 0.18373 0.00000 0.58573 0.13926 0.00000 0.00000 0.09128 0.00000 0.65517 0.34483 0.08063 0.23182 0.06520 0.17562 0.11254 0.33420 0.43911 0.00000 0.08900 0.29935 0.06535 0.10718 0.00000 0.00000 0.81203 0.18797 0.02830 0.05017 0.15013 0.46444 0.21384 0.09312 0.15475 0.26365 0.07523 0.22535 0.04435 0.07926 0.15743 0.00000 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.80354 0.19646 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.16240 0.12592 0.00000 0.17123 0.00000 0.00000 0.54046

B1 gittigidiyor hemalhems security

1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000

that column will be normalised. Such a supermatrix is called a weighted supermatrix. The weighted supermatrix is then raised to a signicantly large power in order to have the converged or stable values. The values of this limit matrix (importance of related criteria and preference for alternatives) are the desired priorities of the elements of the decision network with respect to the goal. The output of the limit matrix can be converted to the descending priority order: the global preferences for the alternatives (see Table IV) and the relative importance of the factors (see Table V). Results As can be seen in Table IV, Gitti Gidiyor was preferred over Hem Al Hem Sat as an online auction and shopping web site. This nding indicates that the global preference values of the web sites are compatible with the average of their page views for the three-month period of 1 October to 31 December 2008 (www.alexa.com). Page views per user for gittigidiyor.com were 12.96 while page views per user for hemalhemsat.com were 4.77. The actual values for these web sites were normalised: (0.7310, 0.2690). The Saaty Compatibility Index (SCI) is computed in order to nd the closeness of the two priority vectors (Creative Decisions Foundation, n.d.). Saaty (2005) illustrates compatibility of a vector w (w1, w2, . . . , wn), by itself. He demonstrates that the Hadamard product or element-wise product of matrix of all possible ratios W (wij) (wi/wj) and its transpose WT is eeT where eT is a (1, 1, . . . , 1) vector of

Usability and functionality factors 431

E-commerce site Gittigidiyor Hemalhemsat

Priorities (%) 72.82 27.18 Table IV. The preferences for the alternatives

Clusters and sub-factors Usability A1 navigation A2 interaction A4 ease of use A6 memorability A3 learnability A5 response time A8 satisfaction A7 efciency Functionality B2 search options B5 user guidance or support B3 information provision B4 services/facilities B6 customisability B1 security

Limiting priorities (%) 14.02 9.06 5.07 3.93 2.50 1.15 0.51 0.23 7.98 5.15 4.94 4.61 2.26 1.19

Normalised priorities (%) 38.44 24.83 13.91 10.78 6.85 3.14 1.41 0.64 30.54 19.71 18.90 17.62 8.67 4.56

Table V. The importance of the factors

OIR 34,3

432

size n. The sum of the elements of the eeT is n2 therefore compatibility index is eT Wo WT e / n2 1. In general, for given two priority vectors v and w if (eT V o WT e / n2) is not more than 1.1 and is less than this value for small size matrixes then v and w are compatible. If the vector of actual data and the vector of predicted outcomes of the ANP model are compatible, then this result suggests that the proposed ANP model is valid and therefore one can rely on the factors of the model and the relationships among factors determined, as well as the importance of the related factors revealed. In this study, as mentioned, the actual data were the normalised page views and the predicted outcomes were the preferences for the alternatives revealed from the ANP. The resulting SCI value was 1.000049, which is very close to 1, and suggests that the proposed ANP model is valid. Necessary calculations can be seen in the Appendix, Figure A1. On the one hand, according to the normalised priorities representing the relative importance of factors, the most important usability factor is navigation (38.44 per cent) followed by interaction (24.83 per cent) (Table V. On the other hand, among functionality factors, search options was found to be the most important one (30.54 per cent). User guidance or support (19.71 per cent) and services/facilities (17.62 per cent) are the next most important functionality factors. Conclusions and further suggestions The development of the internet greatly changes the ways of buying and selling products and services for customers and organisations. Although practitioners and academicians bestow great importance on e-commerce-related issues, the factors that are decisive in customers preferences in web sites remain ambiguous. This paper explored this aspect of the relative importance of usability and functionality factors for online auction and shopping web sites. The Analytic Network Process (ANP) was used to determine the relative importance of the usability and functionality factors. ANP consists of three successive stages: the experts lled in a pair-wise relationship matrix where the entries in the matrix represent the direct relationship between the factors; an assessment/comparison stage where a pair-wise comparison questionnaire of users was conducted; and a synthesis/computation stage where Super Decisions software was used. The ndings of this study are important from the point of view that it is the rst to examine the relative importance of the usability and functionality factors of online auction and shopping web sites. The most signicant aspect of the ndings of this study is that the usability factors of navigation and interaction are regarded as the most important among the factors. This shows that customers wish to nd the relevant information with the least effort through menus, graphical components, hyperlinks and sequences, and layout of pages. Furthermore, interactive features such as feedbacks and system responses support the information retrieval process. These ndings are consistent with those of Roy et al. (2001) and Palmer (2002). Navigation was found to be one of the important factors for concluding online transactions and evaluating trustworthiness of web sites (Roy et al., 2001). Along with interaction, navigation is also critical for success of a web site (Palmer, 2002). In addition, Pearson et al. (2007) point out that the ease of use and navigation factors are

more important than accessibility, download speed and customisation. In our study, ease of use was found to be less important than navigation. This may be because most of the respondents in our study consisted of male undergraduate students, whereas Pearson et al. (2007) found ease of use relatively more important for females than for males. Another signicant nding was that the search option is the most important factor among functionality criteria. The search options factor also contributes to the ease of use of the web sites. Palmer (2002) indicated that searching, alongside navigation, is critical in making a web page easier to use. This result also conrms that ease of use of a web site is an important criterion for customers. Nielsen (1999) points out that the search function is the rst feature many users look for right after they access a web site, and Wilson et al. (2002) assert that users of eBooks also tend to value the search functionality. Another important nding of our study is that security, response time, satisfaction and efciency are the least important factors from the young consumers point of view. On the one hand, in contrast to our ndings, Bontis and De Castro (2000) mention that security is one of the main concerns of commercial web site customers. On the other hand, Tarafdar and Zhang (2005) found that security inuences usability negatively because of complicated functionality. Our ndings on security are conrmed by the results of Tarafdar and Zhang (2005), since respondents bestowed higher importance on the usability factors. According to Palmer (2002), download delay is an important factor in users perceptions of web site usability. The low importance of the response-time factor in our ndings is explained by the high-speed connection infrastructure in the university campus because students generally connect to the internet from campus labs. Therefore, they may not realise the importance of response time since they do not have to wait as long for a response to their actions as they do when accessing the internet through a potentially slower home connection. According to De Marsico and Levialdi (2004), navigation exerts a strong inuence on the efciency of a web site. However, the relatively low importance of efciency in our study shows that young consumers merely want to reach the desired goal without considering how much effort they expend in the process. The low importance of satisfaction implies that students are not concerned about the emotions and feelings they have towards web sites. That satisfaction and efciency were found to be the least important in our study is consistent with the ndings of Ferreira and Pithan (2005). They state that satisfaction is mainly affected by the perceived efciency and effectiveness. Practical implications Our study has a number of signicant suggestions for those that are in the early stages of design of an online auction and shopping web site. First, customers assign higher priority to the usability factors. Thus, early in the design phase, designers should give more importance to navigation- and interaction-related issues. Second, designers of web sites should include basic and advanced search options to increase the number of web site users. Additionally, before the implementation of the web site, it should be tested by potential users to determine their needs, and expectations from it in terms of navigation, interaction and search options.

Usability and functionality factors 433

OIR 34,3

434

Limitations and future research Although the ndings of the present study contribute to a better understanding of the relative importance of the usability and functionality factors for online auction and shopping web sites, there are several limitations that merit future study: (1) Pair-wise comparison questionnaires completed by users and web site designers may reveal their different perspectives. (2) Another group of experts could complete a pair-wise comparison matrix to conrm the consistency of the results of this study. (3) The participant population in our study comprised mostly undergraduate students at Istanbul Technical University. Therefore the results might differ with other participant populations. Future studies should include various populations to enhance external validity. A similar study examining this subject with a broader sample of users located in several different countries could serve to further extend and improve these ndings. (4) A longitudinal research design is essential to conrm the relative importance of the factors that may be critical for optimising online auction and shopping web site usage.
References Alexa.com (2008), The Web Information Company, available at: www.alexa.com (accessed 13 October 2008). Bass, L. and John, B.E. (2003), Linking usability to software architecture patterns through general scenarios, The Journal of Systems and Software, Vol. 66 No. 3, pp. 187-97. Benslimane, Y. and Yang, Z. (2007), Linking commercial web site functions to perceived usefulness: a free disposal hull approach, Mathematical and Computer Modeling, Vol. 46 Nos 9-10, pp. 1191-202. Bertot, J.C., Snead, J.T., Jaeger, P.T. and McClure, C.R. (2006), Functionality, usability and accessibility, Performance Measurement and Metrics, Vol. 7 No. 1, pp. 17-28. Biswas, D. and Biswas, A. (2004), The diagnostic role of signals in the context of perceived risks in online shopping: do signals matter more on the web?, Journal of Interactive Marketing, Vol. 18 No. 3, pp. 30-45. BMI (2009), Turkey Information Technology Report Q4 2009, Business Monitor International, London. Bontis, N. and De Castro, A. (2000), The 1st World Congress on the Management of Electronic Commerce: review and commentary, Internet Research: Electronic Networking Applications and Policy, Vol. 10 No. 5, pp. 365-73. Burnaz, S. and Topcu, Y.I. (2006), A multiple criteria decision-making approach for the evaluation of retail location, Journal of Multi-Criteria Decision Analysis, Vol. 14 Nos 1-3, pp. 67-76. Calero, C., Ruiz, J. and Piattini, M. (2005), Classifying web metrics using the web quality model, Online Information Review, Vol. 29 No. 3, pp. 227-48. Casalo, L.V., Flavian, C. and Guinaliu, M. (2007), The role of security, privacy, usability and reputation in the development of online banking, Online Information Review, Vol. 31 No. 5, pp. 583-603.

Chakraborty, G., Srivastava, P. and Warren, D.L. (2005), Understanding B2B corporate web sites effectiveness from North American and European perspective, Industrial Marketing Management, Vol. 34, pp. 420-9. Chen, L. and Tan, J. (2004), Technology adaptation in e-commerce: key determinants of virtual stores acceptance, European Management Journal, Vol. 22 No. 1, pp. 74-86. Creative Decisions Foundation (n.d.), Creative Decisions Foundation, available at: www.creative decisions.net Davis, J.F. (1999), Effectiveness of internet advertising by leading national advertisers, in Schumann, D.W. and Thorson, E. (Eds), Advertising and Worldwide Web, Lawrence Erlbaum, Mahwah, NJ, pp. 81-97. De Marsico, M. and Levialdi, S. (2004), Evaluating web sites: exploiting users expectations, International Journal of Human-Computer Studies, Vol. 60 No. 3, pp. 381-416. Ferreira, S.M. and Pithan, D.N. (2005), Usability of digital libraries, OCLC Systems and Services, Vol. 21 No. 4, pp. 311-23. Furnell, S. (2005), Why users cannot use security, Computers and Security, Vol. 24, pp. 274-9. Ghose, S. and Dou, W. (1998), Interactive functions and their impacts on the appeal of internet presence sites, Journal of Advertising Research, Vol. 38 No. 2, pp. 29-43. Ha, H. and Janda, S. (2008), An empirical test of a proposed customer satisfaction model in e-services, Journal of Services Marketing, Vol. 22 No. 5, pp. 399-408. Huang, A.H. (2003), An empirical study of corporate web site usability, Human Systems Management, Vol. 22, pp. 23-36. Jharkharia, S. and Shankar, R. (2007), Selection of logistics service provider: an analytic network process (ANP) approach, Omega, Vol. 35 No. 3, pp. 274-89. Kapoor, K. and Goyal, O.P. (2007), Web-based OPACs in Indian academic libraries: a functional comparison, Program: Electronic Library and Information Systems, Vol. 41 No. 3, pp. 291-309. Kavadias, C.D., Rupp, S., Tombros, S.L. and Vergados, D.D. (2007), A P2P technology middleware architecture enabling user-centric services deployment on low-cost embedded networked devices, Computer Communications, Vol. 30 No. 3, pp. 527-37. King, H.J. and Jannik, C.M. (2005), Redesigning for usability, OCLC Systems and Services, Vol. 21 No. 3, pp. 235-43. Klein, L.R. and Quelch, J.A. (1997), Business-to-business market making on the internet, International Marketing Review, Vol. 14 No. 5, pp. 345-61. Klein, R.B. and Seffah, A. (2005), Evaluation of integrated software development environments: challenges and results from three empirical studies, International Journal of Human-Computer Studies, Vol. 63 No. 6, pp. 607-27. Lavie, T. and Tractinsky, N. (2004), Assessing dimensions of perceived visual aesthetics of web sites, International Journal of Human-Computer Studies, Vol. 60 No. 3, pp. 269-98. Liang, T., Lai, H. and Ku, Y. (2007), Personalised content recommendation and user satisfaction: theoretical synthesis and empirical ndings, Journal of Management Information Systems, Vol. 23 No. 3, pp. 45-70. Lu, M. and Yeung, W. (1998), A framework for effective commercial web application development, Internet Research: Electronic Networking Applications and Policy, Vol. 8 No. 2, pp. 166-73.

Usability and functionality factors 435

OIR 34,3

McAfee, R.P. and McMillan, J. (1987), Auctions and bidding, Journal of Economic Literature, Vol. 25 No. 2, pp. 699-738. McCart, J.A., Kayhan, V.O. and Bhattacherjee, A. (2009), Cross-bidding in simultaneous online auctions, Communications of the ACM, Vol. 52 No. 5, pp. 131-4. Nel, D., Van Niekerk, R., Berthon, J.P. and Davies, T. (1999), Going with the ow: web sites and customer involvement, Internet Research: Electronic Networking Applications and Policy, Vol. 9 No. 2, pp. 109-16. Nielsen, J. (1993), Usability Engineering, Morgan Kaufmann, New York, NY. Nielsen, J. (1999), User interface directions for the web, Communications of the ACM, Vol. 42 No. 1, pp. 65-72. Nielsen, J. (2000a), End of web design, Current Issues in Web Usability, 23 July, available at: www.useit.com/alertbox/20000723.html (accessed 7 September 2008). Nielsen, J. (2000b), Designing Web Usability, New Riders Publishing, Indianapolis, IN. Nielsen, J. (2003), Usability 101: introduction to usability, Current Issues in Web Usability, 23 August, available at: www.useit.com/alertbox/20030825.html (accessed 7 September 2008). Novak, T.P., Hoffman, D.L. and Yung, Y. (2000), Measuring the customer experience in online environments: a structural modeling approach, Marketing Science, Vol. 19 No. 1, pp. 22-42. Palmer, J.W. (2002), Web site usability, design, and performance metrics, Information Systems Research, Vol. 13 No. 2, pp. 151-67. Park, C. and Kim, Y. (2003), Identifying key factors affecting consumer purchase behavior in an online shopping context, International Journal of Retail & Distribution Management, Vol. 31 No. 1, pp. 16-29. Pearson, J.M., Pearson, A. and Green, D. (2007), Determining the importance of key criteria in web usability, Management Research News, Vol. 30 No. 11, pp. 816-28. Reilly, E.F., Leibrandt, T.J., Zonno, A.J., Simpson, M.J. and Morris, J.B. (2003), General surgery residency program web sites: usefulness and usability for resident applicants, Current Surgery, Vol. 61 No. 2, pp. 236-40. Roy, M.C., Dewit, O. and Aubert, B.A. (2001), The impact of interface usability on trust in web retailers, Internet Research: Electronic Networking Applications and Policy, Vol. 11 No. 5, pp. 388-98. Saarloos, D.J.M., Arentze, T.A., Borgers, A.W.J. and Timmermans, H.J.P. (2008), A multi-agent paradigm as structuring principle for planning support systems, Computers, Environment and Urban Systems, Vol. 32 No. 1, pp. 29-40. Saaty, R.W. (2003), Decision Making in Complex Environments, Creative Decisions Foundation, Pittsburgh, PA. Saaty, T.L. (1980), Multicriteria Decision Making: The Analytic Hierarchy Process, RWS Publishing, Pittsburgh, PA. Saaty, T.L. (1996), Decision Making with Dependence and Feedback: The Analytic Network Process, RWS Publishing, Pittsburgh, PA. Saaty, T.L. (2005), Theory and Applications of the Analytic Network Process, RWS Publishing, Pittsburgh, PA. Sackett, P.R. and Larson, J.R. Jr (1990), Research strategies and tactics in industrial and organizational psychology, in Dunnette, M.D. and Hough, L.M. (Eds), Handbook of

436

Industrial and Organizational Psychology, Consulting Psychologists Press, Palo Alto, CA, pp. 419-89. Seffah, A., Mohamed, T., Habieb-Mammar, H. and Abran, A. (2008), Reconciling usability and interactive system architecture using patterns, The Journal of Systems and Software, Vol. 81 No. 11, pp. 1845-52. Shyur, H. and Shih, H. (2006), A hybrid MCDM model for strategic vendor selection, Mathematical and Computer Modelling, Vol. 44 Nos 7-8, pp. 749-61. Stefani, A., Vassiliadis, B. and Xenos, M. (2006), On the quality assessment of advanced e-learning services, Interactive Technology and Smart Education, Vol. 3 No. 3, pp. 237-50. Super Decisions Software for Decision-making (n.d.), available at: www.superdecisions.com Tarafdar, M. and Zhang, J. (2005), Analyzing the inuence of web site design parameters on web site usability, Information Resources Management Journal, Vol. 18 No. 4, pp. 62-80. Thelwall, M. (2000), Effective web sites for small and medium-sized enterprises, Journal of Small Business and Enterprise Development, Vol. 7 No. 2, pp. 149-59. I TUK (2008), 2008 yl hanehalk bilisim teknolojileri kullanm arastrmas sonuclar, T.C. Basbakanlk Turkiye statistik Kurumu Haber Bulteni 138, available at: www.bilgi I toplumu.gov.tr (accessed 8 September 2008). Turban, E. and Gehrke, D. (2000), Determinants of e-commerce web site, Human Systems Management, Vol. 19 No. 2, pp. 111-20. Waite, K. and Harrison, T. (2007), Internet archaeology: uncovering pension sector web site evolution, Internet Research, Vol. 17 No. 2, pp. 180-95. Weir, C., McKay, I. and Jack, M. (2007), Functionality and usability in design for eStatements in eBanking services, Interacting with Computers, Vol. 19 No. 2, pp. 241-56. Wen, H.J., Chen, H. and Hwang, H. (2001), E-commerce web site design: strategies and models, Information Management & Computer Security, Vol. 9 No. 1, pp. 5-12. Wilson, R., Landoni, M. and Gibb, F. (2002), A user-centered approach to e-book design, The Electronic Library, Vol. 20 No. 4, pp. 322-30. Yang, Z., Cai, S., Zhou, Z. and Zhou, N. (2005), Development and validation of an instrument to measure user-perceived service quality of information presenting web portals, Information and Management, Vol. 42 No. 4, pp. 575-89. Yeung, T.A. and Law, R. (2004), Extending the modied heuristic usability evaluation technique to chain and independent hotel web sites, International Journal of Hospitality Management, Vol. 23, pp. 307-13.

Usability and functionality factors 437

(See over for Appendix)

OIR 34,3

Appendix

438

Figure A1. SCI for ANP

About the authors Fethi Calisir is Professor of Industrial Engineering at Istanbul Technical University (ITU). He graduated with a BSc from ITU in 1989, an MSc from the University of Miami in 1993 and a PhD in Industrial Engineering from Purdue University in 1996. His current research interests include IT project management, software usability and human-computer interaction. His research papers have appeared in: Computers in Human Behavior, Computers & Industrial Engineering, Total Quality Management, Accident Analysis & Prevention, Technovation, Managing Service Quality, International Journal of Information Management, Management Research News, and Internet Research. Fethi Calisir is the corresponding author and can be contacted at: calisirfet@itu.edu.tr

 A. Elvan Bayraktaroglu is a PhD Student and Research Assistant in the Department of Industrial Engineering at Istanbul Technical University (ITU). She graduated with a BSc in Electrical Engineering and has an MSc in Industrial Engineering from ITU. Her interests are human-computer interaction and usability. Cigdem Altin Gumussoy is a Research Assistant in the Department of Industrial Engineering at Istanbul Technical University (ITU). She graduated with a BSc from Marmara University in 2001, an MSc from ITU in 2003 and a PhD from ITU in 2009. Her current research interests are IT project management and technology acceptance. Her research papers have appeared in Technovation, International Journal of Information Management, Management Research News, and Computers in Human Behavior. Y. lker Topcu is an Associate Professor of Decision Sciences in the Department of Industrial I Engineering, Istanbul Technical University (ITU). He holds a BSc in Industrial Engineering and an MSc in Engineering Management, both from ITU. He completed his PhD at Leeds Universitys School of Business and ITUs Faculty of Management. His research interests include group decision making, DSS and expert systems, logistics, transportation planning and business ethics. He has published in Journal of the Operational Research Society, European Journal of Operational Research, Lecture Notes in Economics and Mathematical Systems, Energy, Building and Environment, and Transportation Research Part A: Policy and Practice. Tezcan Mutlu graduated with a BSc from Istanbul Technical University in 2008. His current research interests include the usability and functionality of web sites.

Usability and functionality factors 439

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints