The International Journal of Human Resource Management, Vol. 19, No.

10, October 2008, 19761988

The gap between research and practice: a replication study on the HR professionals beliefs about effective human resource practices
Karin Sandersa*, Maarten van Riemsdijkb and Bianca Groena
a

University Twente, The Netherlands; bUniversity of Twente and at Saxion University, the Netherlands In 2002 Rynes, Colbert and Brown asked human resource (HR) professionals to what extent they agreed with various HR research ndings. Responses from 959 American participants showed that there are large discrepancies between research ndings and practitioners beliefs about effective human resource practices. The current research is a replication of the Rynes et al. study among 626 Dutch HR professionals. The results show remarkable similarities with the American study: there are large discrepancies between research ndings and practitioners beliefs in some content areas, especially recruitment and selection. Dutch practitioners are somewhat more likely to agree with research ndings when there education level is higher, when they read HRM professional journals more frequently and when they have a positive attitude towards the applicability and usefulness of academic research. Keywords: Dutch HR professionals; HR practices; practice-research gap; replication research

Introduction The question about the gap between research ndings and the knowledge practitioners use is raised frequently. For the domain of HRM this it is a relevant question, as it can be assumed that knowing the facts will lead to more effective HR practices. More effective HR practices, in their turn, will lead to better organizational performance, e.g. nancially (Huselid 1995), with regard to customer relations (Schneider and Bowen 1995) and even related to the internal organizational safety climate (Zacharatos and Barling 2004). For instance Huselid 1995; see Rynes, Colbert and Brown 2002, p. 149, found that a one-standard-deviation increase in scores on a high-performance HR practices scale, including such practices as regular attitude surveying, paying for performance, formal communication programs, and use of employment tests, is associated with a 23% increase in accounting prots and an 8% increase in economic value. Research also shows that companies whose HR professionals read academic research literature have higher nancial performance than companies whose HR staff does not read such literature (Terpstra and Rozell 1997). There are several possible explanations why organizations often fail to adopt practices that research has shown to be effective (e.g. Johns 1993; Rogers 1995). One explanation is that HR professionals lack awareness of research ndings (Gannon 1983). This could be because HR professionals have little time to read. Another possible explanation is that professionals are actually aware of research ndings, but fail to implement the ndings (Pfeffer and Sutton 2000). In this sense one could wonder if the research-practice gap may be primarily a knowing-doing gap rather than a knowing gap. As a rst step Rynes et al. explicitly investigate what practitioners actually know. In 2002 they reported results of their study on the congruence between the beliefs of human resource (HR) practitioners and research ndings about effective human resource practices. In the

*Corresponding author. Email: k.sanders@utwente.nl

ISSN 0958-5192 print/ISSN 1466-4399 online q 2008 Taylor & Francis DOI: 10.1080/09585190802324304 http://www.informaworld.com

The International Journal of Human Resource Management

1977

research, a group of 959 HR professionals (with a response rate of at least 19.2%) was asked to score 35 items true, false, or uncertain. The questions addressed a number of topics, such as recruitment and selection, general employment practices, and development of staff. The scores showed a fairly large discrepancy between what we know from research and what practitioners in the eld think is true. On average, respondents scored 57% of the 35 items correct, but HR professionals clearly know some topics better than others. Recruitment and selection seemed to be problematic, as the average score in this area was only 39%. Much better results were reported for compensation and benets, with an average of 50% correct answers. Both general employment practices and development items were scored best with 68 and 71% correct answers respectively. In addition, Rynes et al. (2002) examined the various ways in which HR professionals obtain information about HR practices. They found HR professionals get their information mainly from other HR professionals in their organization and from the Internet. Research articles came in third as a source of information. Moreover, they reported clear positive relationships between the knowledge level of respondents and their job level, having SPHR certications and reading the relevant academic literature. In this article we present the results of a replication study in the Netherlands. We ask: Can we nd a similar gap between the beliefs of HR practitioners and research ndings about effective human resource practices for the Dutch situation as Rynes et al. (2002) found for the US situation? As in the American study, we will take several characteristics of the respondents, such as job and educational level, age, the kind of information sources they use, and their attitude towards academia into account. Method Respondents for this research were found in two ways. First of all we placed a call to join the research in cooperation with IntermediairPW, a prominent Dutch professional journal for HRM (http://www.intermediairpw.nl). A letter asking respondents to complete a web-based questionnaire was sent together with the journal. We chose to work with this particular journal because of its prominence and penetration rate in the professional HR community. The journal has the largest number of subscribers in the HR eld in the Netherlands. Readers of the journal typically work in medium-sized to large organizations, but also in recruitment and selection companies, outplacement bureaus, consultancy rms and training and development companies. On average, the readers of this journal are higher educated and usually hold a (senior) management position. The call resulted in a somewhat disappointing 197 respondents over a two week period. We therefore issued a second call using a commercial organization that specializes in placing interim HR staff, Van den Boogaart Personeelszaken (Van den Boogaart Personnel; http://www. vandenboogaart.nl). This bureau sent our call to its entire afliated staff, resulting in another 429 completed web-based questionnaires within a week. We checked the characteristics of the two groups of respondents and they are not signicantly different from each other. This cooperation with IntermediairPW and Van den Boogaart Personeelszaken unfortunately had a price as well. We had to accept some modications to the questionnaire to make it more user-friendly and therefore shorter. We did so by reducing the number of questions, eliminating those questions that seemed less relevant for the Dutch situation, or that were unclear according to the board of IntermediairPW. We also had to reduce the answer categories to two (true/false; Rynes et al. (2002) used three: true/false/uncertain), as will be indicated in the results when applicable. In spite of these alterations, we aimed to retain the core of the questionnaire.

1978

K. Sanders et al.

To examine the extent to which the beliefs of HR professionals are consistent with established research ndings we used 25 items of the 35 item questionnaire that Rynes et al. (2002, p. 150) constructed. The original survey content (all 35 items) was based on ve of the seven dimensions covered by the Human Resource Certication Institutes Professional in Human Resources (PHR) exam. The included dimensions were: Management Practices (motivation, leadership, performance management, employee involvement, and HR roles), General Employment Practices (legal issues, performance appraisal, and employee attitudes); HR Development (training and development, and evaluation of training effectiveness), Stafng (recruitment, selection, and career planning), and Compensation and Benets (job pricing, pay structures, compensation strategies, and effectiveness). The dimensions of Safety and Labor Relations were not included because many HR departments do not have responsibility for these particular functions (Rynes et al. 2002, p. 150). The original questionnaire was pre-tested to a sample of 59 highly prolic researchers in HR and industrial and organisational psychology. On the basis of these researchers responses and feedback, problematic items were either reworded or replaced (p. 151). Our selection of the 25 items out of the original 35 was based on their expected relevance for a study in the Netherlands. We deleted for instance four out of the eight items on compensation and benets. Research shows that there is a difference in the acceptance of merit pay systems between Dutch and American employees (Van Silfhout 2000; Atwater, Waldman, Ostroff, Robie and Johnson 2005). The item on drug testing (item 12; general employment practices) was deleted because drug testing is not common at all in the Netherlands (Cowan 1987; Verstraete and Pierce 2001). Furthermore we examined the list of limitations of the current research (see Rynes et al., Table 1, fourth column pp. 152 158), and checked these limitations for the Dutch situation. When limitations were also found for the Dutch situation this item was deleted. The items were translated into Dutch, and respondents were asked to indicate whether they agreed or disagreed with the item. The 25 Dutch items were pre-tested on a group of 16 HR professionals, all members of the Dutch association of HR professionals (NVP), as a start to a discussion on the matter with one of the authors. On the basis of the responses and feedback, problematic items were reworded. The 25 items of the Dutch questionnaire, whether the item is true or false, and the remaining items of the Rynes et al. (2002) questionnaire are given in Table 1. All items, responses and supporting evidence are reported in Rynes et al. (2002, Table 1, pp. 152 158). In addition, in line with the data collection of Rynes et al. (2002, p. 151) information was collected about what types of reading HR professionals do, where they get help with HR problems or issues, on their attitudes towards various sources of HR information and on their attitude toward academic knowledge and HR research. Both archival (e.g. web sites, Dutch professional and scientic journals) and social (e.g. consultants, academics, other HR practitioners) information sources were assessed as well. Furthermore, a few general questions about the personal characteristics of the respondents, like gender, age, job, and educational level were asked. Our efforts produced 626 respondents (Rynes et al. had 959 participants) 241 male (38%), and 385 female, ranging from 20 to 62 years of age (mean 38.9; SD 8.9). Of the respondents 58% graduated from a higher education program (BA level), 41% have an MA or MSc degree, and 18% of the respondents has a degree in psychology. Of the respondents 75% think their training is relevant for their current job. On average, the respondents had worked 5.2 years in their present job (SD 4.8). In Rynes et al.s research respondents had 13.8 years of experience in the eld of HRM (SD 7.9). In Table 2 an overview is given of the job level of the respondents.

The International Journal of Human Resource Management

Table 1. Overview of the correctly answered question for the different samples1. True/ false Management Practices 1. Leadership training is ineffective because good leaders are born, not made. 2. The most important requirement for an effective leader is to have an outgoing, enthusiastic personality. 3. Once employees have mastered a task, they perform better when they are told to do their best than when they are given specic, difcult performance goals. 4. Companies with vision statements perform better than those without them. 5. In order to be evaluated favourably by line managers, the most important competency for HR managers is the ability to manage change. 6. On average, encouraging employees to participate in decision-making is more effective for improving organizational performance than setting performance goals. 7. Companies with very low rates of professionals turnover are less protable than those with moderate turnover rates. 8. If a company feels it must downsize employees, the most protable way to do it is through targeted cuts rather than attrition. General Employment Practices 9. Most managers give employees lower performance appraisals than they objectively deserve. 10. Poor performers are generally more realistic about their performance than good performers are. 11. Teams with members from different functional areas are likely to reach better solutions to complex problems than teams from single areas. 12. Despite the popularity of drug testing, there is no clear evidence that applicants who score positive on drugs tests are any less reliable or productive employees. 13. Most people over evaluate how well they perform on the job. 14. Most errors in performance appraisals can be eliminated by providing training that describes the kinds of errors managers tend to make and suggesting ways to avoid them. HR Development 15. Lecture-based training is generally superior to other forms of training delivery. 16. Older adults learn more from training than younger adults. 17. The most important determinant of how much training employees actually se on their jobs is how much they learned during training. 18. Training for simple skills will be more effective if it presented in one concentrated session than if it is presented in several sessions over time. False False False True True False Corrected Dutch sample % 87 43 86 90 40 32

1979

Items Rynes et al. (2002) % 96 (2) 82 (4,5) 82 (6) 62 (15) 50 (12) 18 (9)

False True

62 (23) 54 (17)

False False True False True False

72 85 69 29

94 (3) 88 (3) 88 (5) 57 (21) 54 (4) 25 (5)

False False False False

85 82 86 66

96 (2) 68 (17) 60 (11) 59 (11)

1980
Table 1 continued

K. Sanders et al.

True/ false Stafng 19. The most valid employment interviews are designed around each candidates unique background. 20. Although people se many different terms to describe personalities, there are really only four basic dimensions of personality, as captured by the Myers-Briggs Type Indicator (MBTI) 21. On average, applicants who answer job advertisements are likely to have higher turnover than those referred by other employees. 22. Being very intelligent is actually a disadvantage for performing well on a low-skilled job. 23. There is very little difference among personality inventories in terms of how well they predict an applicants likely job performance. 24. Although there are integrity tests that try to predict whether someone will steal, be absent, or otherwise take advantage of an employer, they dont work well in practice because so many people lie on them. 25. One problem with using integrity tests is that they have high degrees of adverse impact on racial minorities. 26. On average, conscientiousness is a better performance of job performance than is intelligence. 27. Companies that screen job applicants for values have higher performance than those that screen for intelligence. Compensation and Benets 28. When pay must be reduced or frozen, there is little a company can do or say to reduce employee dissatisfaction and dysfunctional behaviours. 29. Most employees prefer to pay on the basis of individual performance rather than on team or organizational performance. 30. Merit pay systems cause so many problems that companies without them tend to have higher performance than companies with them. 31. There is a positive relationship between the proportion of managers receiving organizationally based pay incentives and company protability. 32. New companies have a better chance of surviving if all employees receive incentives based on organization-wide performance. 33. Talking about salary issues during performance appraisal tends to hurt morale and future performance. 34. Most employees prefer variable pay systems (e.g. incentive schemes, gain sharing, stock options) to xed pay systems. 35. Surveys that directly ask employees how important pay is to them are likely to overestimate pays true importance in actual decisions. False False

Corrected Dutch sample % 38

Items Rynes et al. (2002) % 70 (6) 49 (23)

True False False False

45 54 57 31

49 (13) 42 (12) 42 (30) 32 (34)

False False False

47 32

31 (50) 18 (10) 16 (27)

False True False True True False False False

86 60 88 31

72 (13) 81 (8) 66 (7) 62 (23) 59 (17) 51 (10) 40 (12) 35 (10)

Note: The nature of the research ndings that support each item, along with sample research citations and limitations of the current research in some areas can be found in Rynes et al. (2002, pp. 152158).

The International Journal of Human Resource Management

Table 2. Job level of the two samples. Rynes et al. (2002) % 3 28 26 14 9 6 14 Professionals Director Vice-president HR managers

1981

Dutch sample professionals Director Head of HRM HRM advisor HRM senior advisor HRM consultant Personnel advisor Other

% 26 18 49

Other

It shows that most of our respondents are either head of a HRM department or (senior) HR advisor. The others are Directors, HRM advisers, HR employees or assistants; some are still named personnel ofcers. In general the job level corresponds to the education level of the respondents. Table 2 also shows the job level of the respondents in Rynes et al.s research. Nearly half of the Rynes et al. (2002) sample is director or vice president; the other half is HR manager. Rynes et al. (2002) do not distinguish between HRM (senior) advisor, HRM co-worker or HRM assistant. This means that the Dutch sample is different from the Rynes et al. (2002) sample in that it is more focused on HR professionals rather than general and senior management positions. Results What HR professionals believe: US and the Netherlands In Table 1 the percentages of corrected items for the Rynes et al. (2002) and for the Dutch sample are presented. The percentage of respondents who answered uncertain about this item is shown in brackets. In the Rynes et al. (2002) study, the percentage of this answer category varied between 2 (item 1) and 34 (item 24) for items also used in the Dutch questionnaire. The 626 Dutch respondents had a mean score of 15.6 correct answers out of the 25 items (62%; SD 2.61). In the Rynes et al. (2002) sample, respondents answered 57% of the items correctly on average. However, and in line with the Rynes et al. (2002) sample, a large variation in the extent of agreement showed up (see Figure 1); 103 respondents (16%) answered 16 items correctly; three respondents reached the highest score with 22 correctly answered items, and only two respondents answered just ve items correctly. The item Most errors in performance appraisals can be eliminated by providing training that describes the kinds of errors managers tend to make and suggesting ways to avoid them (answer: false) was answered correctly by 29% respondents (Rynes et al.s sample, 25%). The item Companies with vision statements perform better than those without them (answer: true) was, with 90% of the respondents answering correctly, the highest correctly answered question (Rynes et al., 62%). In Table 3 the average percentage of correct answers clustered in the different domains are related to the Rynes et al. (2002) sample (both the 25 items we used and the 35 items are presented). With respect to (items in) the Management Practices domain the Rynes et al. (2002) and our sample show a high resemblance: a clear majority of practitioners answered the different items correctly. There were, however, differences between the two samples at item level. This was especially the case for the item The most important requirement for an effective leader is to have an outgoing, enthusiastic personality (answer: false). While 82% of the US sample answered this item correctly, this was true for only 43% of the Dutch sample. On the other hand

1982

K. Sanders et al.

Figure 1.

Histogram of knowledge scores.

while one-third (32%) of the Dutch sample correctly answered the item On average, encouraging employees to participate in decision-making is more effective for improving organizational performance than setting performance goals (answer: false), less than one-fth (18%) of the US sample answered this item correctly. In line with the Rynes et al. (2002) sample in our sample only 40% of the respondents answered the item In order to be evaluated favourably by line managers, the most important competency of HR managers is the ability to manage change (answer: true; 50% of the US sample). Although, as for the Management Practices domain, at the domain level there was no difference for the General Employment Practices, differences were found at item level. The Dutch sample answered the item Most people over evaluate how well they perform on the job (answer: true) more correctly (69% versus 54%), while the US sample answered the item Most managers give employees lower performance appraisals than they objectively deserve (answer: false) more correctly (94% versus 72%). The item Poor performers are generally more realistic about their performance than good performers are (answer: false) was predominantly answered correctly in both samples, while the item Most errors in performance appraisals can be eliminated by providing training that describes the kinds of errors managers tend to make and suggesting ways to avoid them (answer: false) was predominantly answered wrong in both samples. A clear majority of the respondents in both samples answered the four items in the HR Development domain correctly. However, at the domain level the Dutch sample exceeds the US sample: 80% versus 71% correctly answered questions. Only in the item Lecture based training
Table 3. Mean percentages correctly answered items for the two samples (both 25 and 35 items of the Rynes et al.s sample). Domains Management practices General employment practices HR development Stafng Compensation and benets Total Dutch sample (25 items) % 63 64 80 43 66 61 Rynes et al. sample (25 items) % 65 65 71 38 49 56 Rynes et al. sample (35 items) % 63 68 71 39 50 57

The International Journal of Human Resource Management

1983

is generally superior to other forms of training delivery (answer: false) does the Rynes et al.s sample exceed the Dutch in this domain (85% of the Dutch and 96% of the US sample answered this item correctly). In line with the Rynes et al.s sample the items in the Stafng domain were answered less well: 38% answered correctly in the US sample, and 43% in the Dutch sample. In both samples less than one-third answered the items Although there are integrity tests that try to predict whether someone will steal, be absent, or otherwise take advantage of an employer, they dont work well in practice because so many people lie on them (answer: false) and Companies that screen job applicants for values have higher performance than those that screen for intelligence (answer: false) correctly. While more respondent in the US sample answered the item The most valid employment interviews are designed around each candidates unique background (answer: false) correctly (70%), this was the case for only 38% of the Dutch sample. With respect to the Compensation and Benets domain the Dutch sample exceeds the US sample again (66% versus 49% correctly answered). The item Most employees prefer variable pay systems (e.g. incentive schemes, gain sharing, stock options) to xed pay systems (answer: false) explains most of the difference on domain level: 80% of the Dutch sample correctly answered this item, versus 40% of the US sample. The item Surveys that directly ask employees how important pay is to them are likely to overestimate pays true importance in actual decisions (answer: false) was answered correctly by almost one third in both samples. Where do HR professionals get their information? Related to the Rynes et al. (2002) study we asked where respondents get their information, and about their attitudes toward different informational sources. First, we asked the respondents to rate three Dutch academic journals (Tijdschrift voor HRM, Tijdschrift voor Arbeidsvraagstukken and Gedrag and Organisatie; all peer-reviewed) and ve professional journals (Intermediair PW, Gids voor Personeelsmanagement, HR Rendement, PandO Actueel and Personeelsbeleid). We used four answer categories: 1 never, 2 sometimes, 3 frequently, and 4 always. Because we asked to rate only Dutch journals it seems not very relevant for this Journals audience to present the frequency and standard deviations on how often these Dutch journals are read by our respondents. However, the overall results were clear: the professional journals were read more frequently than the academic journals. The only journals that are read more often than sometimes are three professional journals (Intermediair PW, Gids voor Personeelsmanagement and HR Rendement). Not surprisingly the journal with the highest ranking was the journal we used to attract part of the respondents (2.83 on a range from 1 to 4). The three academic journals showed the lowest rankings for the frequency with which they are read (Tijdschrift voor HRM: 1.40; Tijdschrift voor Arbeidsvraagstukken 1.31; and Gedrag and Organisatie: 1.24). Furthermore their ranking matches their scientic impact perfectly: the journal with the highest scientic impact factor is read least. Table 4 provides additional information on how practitioners seek or get information about HR issues. In this table the ratings of both samples are given but a comparison is not valid and highly tricky. Instead of the Rynes et al. question (How often do you use the different sources of help for solving HR problems?) and their answer categories (1 rarely or never, 2 a few times per year, 3 about once a month, 4 several times per month, and 5 almost daily), we asked the question How important are the different sources of help for solving HR problems for you? with answer categories ranging from 1 completely unimportant to 5 completely important. The results presented in Table 4 should therefore be interpreted with caution; Dutch respondents for instance nd web sites to be very relevant as a source for information (4.21 on a range from 1 to 5). Interpreting this result with the Rynes et al. (2002)

1984
Table 4.

K. Sanders et al.
Sources of help for solving HR problems (ranking between brackets). Dutch sample Rynes et al. sample SD 0.73 0.71 0.73 0.92 0.87 Mean 2.74 (3) 3.15 (1) 2.62 (4) 1.70 (5) 2.98 (2) SD 1.10 1.38 1.10 .811 1.04

Resource Web sites Other HR professionals HR research literature Consultants and academics Web site NVP (SHRM)

Mean 4.21 (1) 4.15 (2) 3.90 (3) 3.37 (4) 2.75 (5)

Notes: 15 scale, where 1 rarely or never, 2 a few times per year, 3 about once a month, 4 several times per month, 5 almost daily for the Rynes et al. sample, and 15 scale ranging from 1 completely unimportant to 5 completely important for the Dutch sample; This mean and stand deviation is the average of two items in the Rynes et al. sample, namely consultants, and academics; in the Dutch sample this was one item.

response categories would lead to the erroneous idea that Dutch respondents search the net for information daily, and much more frequently than their US counterparts (2.74). However, we do think that the table contains relevant information. It shows that for the Dutch sample the most important sources to which practitioners turn when faced with HR problems are web sites and other HR professionals, while for the US sample the most frequently used sources are other HR professionals and the SHRM web site. For the Dutch in contrast, the web site of the professional association (NPV) is relatively unimportant. In line with Rynes et al.s study it is interesting to note that in spite of the very low rates of academic journal reading reported in the Dutch sample, practitioners still report that the research literature is more important to them to get information than consultants and academics. Finally, Table 5 shows respondents attitudes toward academics and academic research. For this purpose four items of Rynes et al. (2002) were used. Respondents had to indicate on a scale from 1 (I do not agree at all) to 5 (I totally agree) to what extent an item applied to them. Two items were about the amount of time respondents (want to) spend on reading academic literature in the eld of industrial and organizational psychology or on talking to academics about HR problems. The other two items asked whether respondents think results of academic literature make sense and whether they are useful in practice. This table shows again a remarkable similarity between the Dutch and the US sample. The practitioners are close to neutral in questions about the applicability of research ndings (items 3 and 4 Dutch sample mean 2.56; US sample 2.91; on a ve-point scale) but that
Table 5. Practitioner attitudes towards academics and HR research. Dutch sample Attitude 1. I wish I had more time to read about academic HR research ndings 2. I would like to spend more time talking to academics about HR problems 3. Most research ndings make sense in theory, but dont work well in practice 4. I generally dont nd academic HR research to be very useful Mean 3.65 3.38 2.71 2.42 SD 0.94 0.92 0.76 0.82 Rynes et al. sample Mean 3.91 2.86 3.04 2.78 SD 1.03 1.09 .84 .91

Note: 15 scale ranging from 1 completely unimportant to 5 completely important.

The International Journal of Human Resource Management

1985

they nevertheless wish they had more time to read about them and to talk to academics about their HR problems (items 1 and 2 Dutch sample mean 3.51; US sample 3.38). When combining the two items concerning time (item 1 and 2) and the two items about the applicability and the usefulness of results of academic research (item 3 and 4) we found a positive relationship between both two-item-scales (r .16, p , .01): the more HR professionals would like to spend more time to read and talk about HR problems, the more they nd research ndings applicable and useful. In addition, Rynes et al. (2002) report a negative relationship between the experience-level (tenure) of the practitioners and their desire to learn about academic research (items 1 and 2). In our sample no relationships were found between the desire to learn more (the time items) and the applicability and usefulness of results from academic research on one hand and tenure on the other hand (resp. r 2 .03, n.s.; r .03; n.s.). What characteristics are associated with research knowledge? We nally try to answer the question whether the differences in research knowledge are reliably associated with differences in individual characteristics (e.g. education) or information-seeking strategies (e.g. academic reading). To assess this question, we constructed a correlation matrix of the major demographic characteristics, the two most common information-seeking strategies (HR professionals and Internet), HR reading, and the number of correct responses (knowledge). Because we were curious if the attitude concerning time and results of academic research were related to the knowledge of practitioners we include these two subscales into the correlation matrix as well. Although Rynes et al. (2002) used job level of the respondents; we include education level of the respondents in the correlations matrix. Job levels in the Dutch situation are rather vague and non-descript (see also Table 2). There is no clear certication structure, like in the US situation and a distinction between an HR advisor, a HRM consultant and a personnel advisor is difcult to make. Furthermore, job levels certainly give little clue about the knowledge level of the respondent. The education level on the other hand, is crystal clear. Finally, due to the small numbers of practitioners reading academic journals, we only include HR reading; meaning the mean frequency practitioners read the HR journals. The results of the correlation analyses are presented in Table 6. Rynes et al. (2002) found that the number of correct responses (knowledge) was positively related with job level, SPHR certication, academic reading and the use of consultants to seek information. As Table 6 indicates, in our sample we found a positive relationship between knowledge on the one hand and education level (r .14, p , .01), the attitude about results of academic research on the other (r .15, p , .01), and HR readings (r .17, p , .01). Interestingly, we found a positive relationship between HR knowledge and having a degree
Table 6. Correlations among major study variables. Mean SD 1. 2. 3. 4. 5. 6. 7. 8.

1. Tenure 5.20 4.80 2. Education 4.39 .52 2.09* 3. Major psychology 1.82 .39 .06 2 .38** 4. HR reading 1.86 .46 .18** 2 .07 5. Web site 4.21 .73 .02 .01 6. HR professionals 4.15 .71 2.04 .04 7. Attitude (time) 3.51 .78 2.03 .04 8. Attitude (results) 3.44 .66 .03 .19** 9. Knowledge 15.60 2.61 2.02 .14**

.10* 2 .04 .01 2 .06 2.08 .06 2 .01 .07 2 .01 2 .06 2 .13** 2.03 .04 .01 .16** .09* .17** .02 2 .02 .01 .15**

1986

K. Sanders et al.

in psychology (r . 09, p , .05): practitioners who studied psychology were associated with lower levels of agreements with research ndings. Finally, like Rynes et al. (2002) in their research, we conducted a regression analysis with all the independent variables. This analysis showed that three of the variables remained signicant in the multivariate analysis: education level (b .11, p , .05), HR reading (b .12, p , .01) and a positive attitude about results of academic research (b .12, p , .01). The overall predictability was quite modest, but a little bit higher than in the Rynes et al. (2002) sample (R2 .24; vs R2 .20). The variables that remained signicant in the multivariate analyses of Rynes et al. (2002) were: job level, having SPHR certication and reading academic literature. Discussion In this article we focus on the gap between HR professionals beliefs about effective human resource practices and results from I/O psychology research, and replicate the study of Rynes et al. (2002) for the Dutch situation. Pfeffer and Sutton (2000) proclaimed that information dissemination has become so efcient that everyone is likely to know about best practices. Therefore, they argue, best practices are no longer likely to be sources of competitive advantage. In clear contrast to this assertion the results of Rynes et al. (2002) were published in 2002. These suggest that there are in fact very large differences across companies in what their HR leaders know about best practices in HR, and furthermore that the average level of knowledge does not appear to be very impressive. In line with the Rynes et al. (2002) study, and contrary to the assertion of information dissemination, in a Dutch sample we found a large gap between practice (HR professionals beliefs about effective human resource practices) and the results of academic research as well. This holds both for the general conclusion and for the differences between the distinguished content areas. For both the American and Dutch sample there was enormous variation in the extent to which respondents (as a group) agreed with particular items. Related to this, for both samples we found that the items within the recruitment and selection (stafng) area show the largest gap between HR practitioners beliefs about best practices and research ndings. Rynes et al. (2002, p. 164) already give some possible explanations for this nding. First, selection research is more technical than research in many other areas of HR, frequently involving such complex procedures as meta-analysis, corrections for measurement errors and utility analysis. Second, the frame of reference with respect to selection appears to be different for academics and practitioners. While selection researchers focus on gathering knowledge about rather abstract characteristics of people, recruitment and selection activities are mostly designed around the job. A third explanation might reside in the continued negative coverage that intelligence and ability testing receive in the popular press (e.g. Goleman 1998). Independent of the reasons for the discrepancies between beliefs and research ndings, lack of awareness, especially in the selection area, can be very costly to an organization. But the variation in knowledge within the Dutch and American sample is not the only similarity. Other important issues show remarkable similarities as well. For the sources of information, the preference of practitioners to read HR professional journals rather than academic journals stands out. Furthermore, their attitude toward academic research and their desire to learn more from academic research is about the same. In both countries practitioners are a little bit ambiguous about the usefulness and the applicability of academic research. However, both groups would like to have more time to read results from academic research and to talk to academics about their HR problems. Although in the end the regression comparison includes different signicant factors there is some similarity between the two samples. The American sample showed that job level, having

The International Journal of Human Resource Management

1987

a SPHR certication and reading academic literature are the most important factors in explaining the knowledge of practitioners. The Dutch sample showed that educational level, the attitude about results of academic research and HR reading are important. Similar are the importance of reading (either academic or HR reading), and the educational versus job level. Most important difference between the two countries seems to be the importance of certication; while in the Netherlands there is no strong professional body and hence no clear certication structure, in the US situation this does seem to help to enhance knowledge. But we also found out once again that doing a replication study in itself can be a challenge. As a result, there are some limitations in our study. First of all, to gain access to our research population, we opted for cooperation with a professional journal. This proved helpful, but at a price. We had to reduce the number of items and some of the answer categories. The reduction in the number of items we think preserved the core of the research, as we were able to delete items that were not applicable in the Dutch situation, or that were tricky in the original research. The difference in answer categories presents a somewhat bigger problem. Rynes et al. (2002) used true, false or uncertain and we used only true or false. However, if we look at the results in Rynes et al.s study, we see that most of the items score low percentages in the uncertain category. Furthermore, a few items that scored highly on that category fortunately have been removed from the Dutch questionnaire because of lack of relevance in the Dutch context. If we look at the two populations, there are some differences. It seems that the respondents in the Rynes et al. (2002) sample were of a more general management level, whereas the Dutch respondents were all HR professionals. Does that mean we should expect somewhat better results in our sample? A difcult point, as we would expect senior management to understand HR issues relatively well too. Looking to the future Rynes et al. (2002) suggested that it would be useful to put more research content into practitioners journals and other formats commonly used by HR practitioners. In the Netherlands this is explicitly stated and aimed for in the mission of one of the professional journals, the Tijdschrift voor HRM. Unfortunately, its subscription level is rather low. Another way to stimulate and promote a broader perspective concerning the relevance and value of psychological research for improving HR practices and organizational functioning are the nine articles of the special issue of Human Resource Management in 2004 (Burke, Drasgow and Edwards 2004). In these articles information is given for understanding the contributions of psychological theories and research ndings to HR management and, consequently, for closing the science-practice knowledge gaps. For instance the article in this special issue concerning Attracting and selecting (Ryan and Tippins 2004) focuses on the misperceptions regarding research ndings in the area of employee selection, and gives an overview on which selection tools work, which recruitment strategies work, how selection-tool use relates to workplace diversity and which stafng and recruitment processes lead to positive applicant perceptions. Besides the request for more replication studies of the Rynes et al. (2002) study, the articles in the special issue show a promising way of handling the science-practice knowledge gap. References
Atwater, L., Waldman, D., Ostroff, C., Robie, C., and Johnson, K.M. (2005), Self-other Agreement: Comparing its Relationship with Performance in the US and Europe, International Journal of Selection and Assessment, 13, 25 40. Burke, M.J., Drasgow, F., and Edwards, J.E. (2004), Closing Science-practice Knowledge Gaps: Contributions of Psychological Research to Human Research Management, Human Resource Management, 43, 299 304. Cowan, T.R. (1987), Drugs and Workplace. To Drug Test or Not to Drug Test, Public Personnel Management, 16, 313 322. Gannon, M.J. (1983), Managerial Ignorance, Business Horizons, May June, 26 32.

1988

K. Sanders et al.

Goleman, D. (1998), Working with Emotional Intelligence: Why It Can Matter More Than IQ for Character, Health, and Lifelong Achievement, New York: Bantam Books. Huselid, M.A. (1995), The Impact of Human Resource Management Practices on Turnover, Productivity, and Corporate Financial Performance, Academy of Management Journal, 38, 635 672. Johns, G. (1993), Constraints on the Adoption of Psychology-based Personnel Practices. Lessons from Organizational Innovation, Personnel Psychology, 46, 569 592. Pfeffer, J., and Sutton, R. (2000), The Knowing-doing Gap, Boston, MA: Harvard Business School Press. Rogers, E. (1995), The Diffusion of Innovations (4th ed.), New York: The Free Press. Ryan, A.M., and Tippins, N.T. (2004), Attracting and Selecting: What Psychological Research Tell Us, Human Resource Management, 43, 305 318. Rynes, S.L., Colbert, A.E., and Brown, K.G. (2002), HR Professionals Beliefs about Effective Human Resource Practices: Correspondence Between Research and Practice, Human Resource Management, 41, 149 174. Schneider, B., and Bowen, D.E. (1995), Winning the Service Game, Boston, MA: Harvard Business School Press. Terpstra, D.E., and Rozell, E.J. (1993), The Relationship of Stafng to Organizational Level Measures of Performance, Personnel Psychology, 46, 27 48. Van Silfhout, R. (2000), Inequality in Pay within Organizations: Normative and Instrumental Perspectives, Tilburg, NL: Tilburg University Press. Verstraete, A.G., and Pierce, A. (2001), Workplace Drug Testing in Europe, Forensic Science International, 121, 2 6. Zacharatos, A., and Barlings, J. (2004), High Performance Work Systems and Occupational Safety, in The Psychology of Workpace Safety, eds. J. Barlings and M.R. Frone, Washington, DC: American Psychological Association, pp. 203 222.