whitepaper

Selection: The Validity of Behaviorally Based Interviews

The following bibliography briefly outlines key technical articles in the literature surrounding the validity of behaviorally based interviews. The articles contained in this bibliography describe interviewing paradigms that are similar in design to the six components of Targeted Selection , DDIs behaviorally based selection system, in many or, in some cases, all facets. Evidence supporting the superior benefits of behavior-based interviewing is presented in terms of validity coefficients. Validity coefficients can range from 0.0 to 1.0 (although values near the high end of the scale are extremely uncommon). Ratings of job performance are the most commonly used criteria for evaluating the validity of different interviewing methods. This summary clearly demonstrates the superiority of behavior-based, structured interviews compared to alternative methods of interviewing. Each article is briefly summarized; the reader should access the original sources for a more complete understanding.

Article summaries
Campion, M.A., Campion, J.E. (1994). Structured interviewing: A note on incremental validity and alternative question types. Journal of Applied Psychology, 79 (6), 998-1002. A sample of 70 paper mill employees underwent a structured interview containing both situational and experience-based questions. The interview content was based on a formal job analysis. Job performance data served as the criterion.

Validity of experience-based questions = .51, p < .05. Validity of situational questions = .39, p < .05.

Janz, T. (1982). Initial comparisons of patterned behavior-based interviews versus unstructured interviews. Journal of Applied Psychology, 67 (5), 577-580. Fifteen teaching assistants (TAs) underwent both patterned, behavior-based interviews and standard interviews; interviewers then predicted the student ratings that TAs would receive. Actual student ratings of the TAs served as criterion data.

Validity of behavior-based interviews = .54, p < .001. Validity of standard interviews = .07, ns.

To learn more, call your local DDI office or contact: The Americas ....... 412.257.0600 Toll-free Canada... 800.668.7971 Toll-free U.S. ........ 800.933.4463 Southeast Asia.... 65.339.5255 Australia.............. 61.2.9466.0300 Greater China ..... 852.2526.1188 France ............. 33.1.41.96.86.86 Germany ......... 49.2159.91680 New Zealand ... 64.9.377.6742 United Kingdom 44.1628.810800 E-mail ...............info@ddiworld.com Web ..................www.ddiworld.com

DEVELOPMENT DIMENSIONS INTERNATIONAL

McDaniel, M.A., Whetzel, D.L., Schmidt, F.L., & Maurer, S.D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79 (4), 599-616. Details a meta-analysis of 245 validity coefficients from more than 86,000 subjects. Results:

Orpen, C. (1985). Patterned behavior description interviews versus unstructured interviews: A comparative validity study. Journal of Applied Psychology, 70, 774-776. Sixteen interviewers were randomly selected from the employee population of a large life insurance company; each was then randomly assigned to behavior-based interviewer training or unstructured interviewer training. Nineteen sales position applicants were then interviewed by each type of interviewer; interviewers then rated the potential success of each applicant. Criterion data (supervisor ratings & sales dollars) were collected after applicants had been in the position one year (all applicants were hired, regardless of interviewer ratings). Results:

Mean corrected validity of structured, jobrelated interviews = .44 for studies using job performance as the criterion. Mean corrected validity of structured, jobrelated interviews = .34 for studies using training performance as the criterion. Mean corrected validity of structured, multirater interviews = .38 for studies using job performance as the criterion.

Motowidlo, S.J., Carter, G.W., Dunnette, M.D., Tippins, N., Werner, S., Burnett, J.R., & Vaughan, M.J. (1992). Studies of the structured behavioral interview. Journal of Applied Psychology, 77 (5), 571 - 587. A series of five psychometric studies designed to assess the value of structured behavioral interviews was conducted. Criterion data consist of supervisor ratings of job performance. Results:

Validity of behavior-based interview = .48 using supervisor ratings as the criterion; validity = .61 using sales dollars as the criterion. Validity of the standard interview = .08 using supervisor ratings as the criterion; validity = .05 using sales dollars as the criterion.

Mean criterion-related validity estimate (uncorrected) = .22 across 500 subjects.

Pulakos, E.D. & Schmitt, N. (1995). Experiencebased and situational interview questions: Studies of validity. Personnel Psychology, 48, 289 - 308. A sample of 108 subjects underwent a structured, situational interview while another 108 subjects underwent a structured, experience-based interview. Supervisory ratings were the criterion. Results:

Validity of experienced-based interview = .32, p < .05. Validity of situational interview = - .02, ns.

2 Selection

DEVELOPMENT DIMENSIONS INTERNATIONAL

Wiesner, W. H. & Cronshaw, S. F. (1988). A meta-analytic investigation of the impact of the interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275-290. A meta-analysis of employment interviews was conducted to assess the impact of interview format (individual vs. multiple interviewers) and interview structure (unstructured vs. structured) on interview validity. Relevant findings include the following:

Summary
This bibliography briefly outlines the obvious benefits to be derived from utilizing behaviorally based interviewing techniques that stand upon the results of a formal job analysis in a selection system. Targeted Selection, DDIs behaviorally based selection system, offers all of the benefits described in the included articles and its success has been well documented in numerous types of organizations over the years.

Structured interviews with multiple interviewers have a mean validity coefficient of .60, across 7,873 subjects. Also, a consensus approach to combining ratings in the multiple interviewer studies enhanced structured interview validity. Structured interviews based upon a formal job analysis yielded a higher mean validity coefficient (.87) than those structured interviews that are based upon informal job analyses (.59) or an undefined job analytic method (.56).

Additional studies containing useful information:

Ghiselli, E.E. (1966). The validity of a personnel interview. Personnel Psychology, 19, 389 - 394. Harris, M.M. (1989). Reconsidering the employment interview: A review of recent literature and suggestions for future research. Personnel Psychology, 42, 691-726. Janz, T. (1989). The patterned behavior description interview: The best prophet of the future is the past. In Eder, R.W. & Ferris, G.R. (Eds.). The employment interview: Theory, research, and practice. Newbury Park, CA: Sage. 158-167.

Wright, P.M., Lichtenfels, P.A., & Pursell, E.D. (1989). The structured interview: Additional studies and a meta-analysis. Journal of Occupational Psychology, 62, 191-199. This study involved a meta-analysis of six previously published and seven previously unpublished structured interview validity studies. Results:

A mean corrected validity coefficient of .35 was obtained for structured interview predictions with supervisor ratings as criterion measures.

Bibliography prepared by: DDI Center for Applied Behavioral Research (DDI-CABER)

MKTSEWP01-898.0MA

3 Selection

Development Dimensions International, Inc. MCMXCVIII. All rights reserved.