MOS384 Sel 07 test

Slide1: 

Management andamp; Organizational Studies MOS 384a Section 001 Fall 2006 Personnel Recruitment and Selection Selection II: Testing

MOS 384a - Testing: 

MOS 384a - Testing Readings: Textbook (CWHM): Chapter 8 Your Lecture Notes

MOS 384a - Testing: 

MOS 384a - Testing Overview: Using Psychological Tests Tests of Dispositional Constructs (AOs): Cognitive abilities Psychomotor, sensory, physical abilities General personality traits Integrity tests and other COPS Behavioural measures (KSs) Work sample tests, simulations Situational judgment tests Assessment centres

MOS 384a - Testing: 

MOS 384a - Testing A little quiz from the textbook

MOS 384a - Testing: 

MOS 384a - Testing Using Psychological Tests: Sources to consider Professional Guidelines: CPA Guidelines, APA Standards, SIOP Principles Technical Manuals: reliability, validity, norms, fairness/bias, user reactions, (utility) Code of Ethics (CHRP): Respect for human dignity and rights Foster accepted codes of moral behaviour Balance employer and employee interests

MOS 384a - Testing: 

MOS 384a - Testing Using Psychological Tests: Applicants’ legal rights Informed consent Access to test results Privacy and confidentiality Accommodation of disadvantages due to language and disability

MOS 384a - Testing: 

MOS 384a - Testing Tests of cognitive abilities General mental ability ('g': Spearman, 1904) Specific abilities: Thurstone: 7 primary factors Cattell: fluid vs. crystallized intelligence Guilford: 120 specific abilities Little evidence of additional (incremental) validity of specific abilities beyond 'g'

MOS 384a - Testing: 

MOS 384a - Testing Tests of cognitive abilities Examples of tests: Wonderlic Personnel Test (WPT), WAIS, Raven Progressive Matrices Sample items

MOS 384a - Testing: 

MOS 384a - Testing Tests of cognitive abilities Hunter andamp; Hunter metaanalysis (1984): 'g' predicts performance with an average validity of .51  best single predictor we have ! More valid in complex than simple jobs (.23 to .58) Large mean differences between job groups Major problem: Racial differences (adverse impact)

MOS 384a - Testing: 

MOS 384a - Testing Tests of other abilities Psychomotor abilities (e.g., finger dexterity, multi-limb coordination, reaction time) Sensory/perceptual abilities (e.g., far vision, auditory attention, visual color discrimination) Physical abilities (e.g., static and dynamic strength, stamina, balance) ρ = .40 to .50 in specific job groups Back to the job analysis and identifying the appropriate As

MOS 384a - Testing: 

MOS 384a - Testing Ability Testing: Fuzzy Constructs I Practical intelligence / tacit knowledge: The implicit 'know-how' to solve practical problems Difficult to distinguish from procedural knowledge Early studies showed incremental validity beyond GMA, but independent replications failed Job-specific knowledge tests more valid (.45)

MOS 384a - Testing: 

MOS 384a - Testing Ability Testing: Fuzzy Constructs II Emotional intelligence Definition I: EI as a set of personality traits Definition II: EI as s set of abilities to understand and regulate emotions Def. I adds little beyond personality theory; Def. II shows some potential Def. I associated with misleading management folklore

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: Some History Personality as unconscious motives Projective techniques (inkblot, TAT, etc.) Trait approach to personality Personality questionnaires (16PF, CPI, HPI, NEO, etc.) Crisis in the 1960/70ies: Guion andamp; Gottier (1965), and Mischel (1968) doubt that there is any notable relation between traits and behaviour / performance 'Comeback' of personality in the 1980/90ies: Five-Factor Model/Big Five; COPS

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: The 'Big Five' / FFM Neuroticism (vs. Emotional Stability): tense, nervous, depressive, anxious, (impulsive, easily upset) Extraversion (vs. Introversion): outgoing, sociable, talkative, (assertive, dominant) Openness to experience: (creative), open to new ideas, actions, feelings, non-conforming values Agreeableness: altruistic, modest, trusting, compliant, trouble-avoidant, (honest) Conscientiousness: reliable, self-disciplined, orderly, (achievement-striving)

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing Metaanalytic evidence on FFM validities (Barrick et al., 2001; Judge et al., 2002):

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: COPS Validity coefficients of FFM dimensions modest at best (but independent of abilities) Possible causes of modest validity: Dimensions too broadly defined Inventories not specifically job-related Possible alternative: Criterion-Focused Occupational Personality Scales (COPS)

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: Integrity Tests Attempt to predict whether an employee will engage in counterproductive behaviour Two types: Overt – measure attitudes toward counterproductivity (often also admissions of actual behaviours) Personality-based – Personality characteristics like risk taking, trouble avoidance, and reliability are measured Substantial validity for predicting both performance (ρ =.41) and counterproductivity (ρ =.30 to .40)

MOS 384a - Testing: 

MOS 384a - Testing IBES Integrity Test Sample Subscales and Items

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: Major current issues What is beyond the Big Five? Bandwidth-fidelity: Are the Big Five too broad? Faking, social desirability, or impression management Discuss implications of faking for selection

MOS 384a - Testing: 

MOS 384a - Testing Personality Testing: Findings on Faking Group means: Instructions to fake and applicant settings lead to inflated scores Construct validity: Desirable traits tend to be more highly correlated in applicants Criterion-related validity: No clear-cut effect !

MOS 384a - Testing: 

MOS 384a - Testing Some not so useful alternatives to psychological testing HIV and genetic testing: With few exceptions, illegal as mandatory screening out procedure Drug testing: very limited validity; very restrictive legal limitations Polygraph: questionable validity and legal concerns; non-fakability is a myth ! Graphology: zero validity, indefensible

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Work Sample Tests Historically oldest type of psychological selection procedure (Münsterberg, 1910) Develop smaller, (standardized) tests of actual task behaviour – replicas of the job e.g., Driving test, typing test, lecture Earlier metaanalyses report validities in the .50’s; recently corrected downwards to .33 (Roth et al., 2005)

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Simulations Simulate work environment for jobs where poor performance in real settings would be disastrous e.g., business games, flight simulator Validities typically around .30, but can be higher in high-fidelity simulations

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Situational Judgment Tests Paper-and pencil measures tapping applicant judgment in work settings 'How would you handle a situation in which…' 'What should an employee do in a situation…' Incremental validity over personality tests, job experience, and cognitive ability tests ρ = .34 for job performance More valid if based on job analysis (CIT)

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Assessment Centres (ACs) Assessors observe performance on a standardized set of predictors Multiple assessors, assessees, exercises, and dimensions Typically 1-3 days; 2 candidates per assessor; assessor not direct supervisor Extensive feedback given to candidates Mostly used for managerial positions

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Typical AC Exercises In-basket: Assessee required to respond to a series of job-related scenarios (Leaderless) Group Discussion: Designed to tap managerial attributes from interactions in a small group Presentations Role plays Business simulations and case studies

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures AC Exercise by dimension matrix

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Evaluation of ACs Criterion-related validity (ρ = .31) somewhat disappointing given the costs Can be improved… Employ psychologists as assessors, moderators Use valid exercises only, add interviews / tests Avoid rating too many dimensions

MOS 384a - Testing: 

MOS 384a - Testing Work-sample-like procedures Evaluation of ACs Construct validity is a major problem Sackett andamp; Dreher (1982): Different dimensions measured by same exercise more highly correlated than same dimension measured by different exercises (low convergent and discriminant validity) Very positive user reactions