Unproctored Internet Testing UIT

1
Unproctored Internet Testing (UIT) The New
Normal?
2
www.chinaculture.org
3
Unproctored Internet Testing (UIT) The New
Normal?

• What does UIT look like in businesses today?
• Does UIT work?
• Is UIT ethical?

4
Virtual Recruitment

• Fewer traditional sources
• Ad in the classifieds
• Employment agencies
• Job fairs
• More Internet-based sources
• Job Boards
• Company websites

5

• Modes of testing - ITC Computer and Internet
  testing Guidelines (2005)
• Open No human supervision and no ID verification
• Controlled No human supervision but test takers
  are known and invited to take test
• Supervised/Proctored Human supervision of
  testing and ID verification
• Managed High level of human supervision (ID
  verification and monitoring of testing process)
  as well as training of administrators (e.g.,
  testing center)

6

• Revising the ITC Definitions
• Open (no change) Not the recommended UIT
  approach
• Controlled We never know 100 that ID has been
  verified (continuum of knowing)
• Supervised/Proctored Often still unproctored
• Managed (proctored) external or internal testing
  center

7
PreVisors Top 50 clients
8
Business Scenario

• A business unit has requested unproctored
  testing for a high volume entry level position
  where 40 of applicants will be screened out by a
  validated, unproctored, Internet-based test. The
  other 60, the passers, will be invited in for
  two interviews, one by HR and the other by a
  hiring manager. Hiring decisions will be made
  after the interview.

9
Business Scenario

• Group 1 I think cheating would invalidate the
  process because cheaters would get through the
  system and therefore I WOULD NOT DO unproctored
  testing.
• Group 2 The potential impact of cheating
  concerns me I MIGHT ASSUME THE RISKS if research
  suggests unproctored testing works.
• Group 3 Im NOT CONCERNED about cheating
  (cheating is not that prevalent that it matters
  or wed detect the cheaters through the
  interview)

10
The New Normal Unproctored Internet Testing
(UIT)

• What does UIT look like in businesses today?
• Does UIT work?
• Is UIT ethical?

11
Norm analysis Entry Level

• Industries
• Retail Wholesale Personal services
• Telecom Computers Technology
• Health and Human Services
• Finance Insurance
• Solution Content
• Empirically scored multiple choice
• Biodata
• Personality
• Situational judgment

12
UIT norm trends Entry
Jan-Feb 2006
13
UIT norm trends Professional
Jan-Feb 2006
14
Cognitive UIT norms Client 1
15
Cognitive UIT norms Client 2
Entry level customer service job
16
UIT Validity studies (slide from SIOP 2005
Spring 2006 PPsyc)
17
Recent UIT Validity studies
(Competency/Solution Scores)
18
UIT and Diversity Client 1

• Fortune 100 Client
• UIT (Supervised) in place for nearly 3 years
• Analyzed applicant data for pre/post UIT
• Professional level job
• PRE/POST UIT Comparison
• 10 increase in female applicants
• 35 increase in minority applicants

19
UIT and Diversity Client 2

• Fortune 500 Client
• UIT (Controlled) in place for 2.5 years
• Analyzed hire data for pre/post UIT
• Entry level jobs
• RESULTS
• Same of race minorities hired in first year of
  UIT implementation
• of race minority hires increased approximately
  5 each year
• of female hires has doubled since UIT
  implementation

20
UIT and Diversity Client 3

• Fortune 100 Client
• UIT (Open) in place for 3 years
• Analyzed applicant data for pre/post UIT
• Entry level job

21
UIT and Diversity Client 3 change in EEO
applicant flow
22
Cheating Risk
Cheating risk High
Easily identifiable testing process
Little or no penalty for cheating
Testing process is not easily identifiable
Severe penalty for cheating

• High Stakes
• exec level
• test is obvious hurdle

Cheating risk Low

• Low Stakes
• entry to mid level
• test not obvious hurdle

23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Cheating on the Internet

• Braindumps
• Collusion
• Proxy examinee
• Mickey Mouse effect (usually 1 or less)

28
Is UIT Professionally Ethical?

• Is testing worth the risk?
• Is it realistic to tell business they just need
  to stop?
• Will the legal response be no way or within
  limits, UIT can be made fair enough given its
  business necessity

29
Defending against cheating

• Document the business case for implementing
  unproctored testing - documentation is critically
  important to developing a world-class and
  defensible system.
• Design your system with a single point of entry
  into the online hiring process - utilize an
  applicant tracking system which collects
  significant identification information and
  employs a one-use url (applicants can only
  complete the test one time).
• Maintain current recruiting efforts. Our research
  suggests that diversity goals can be maintained
  with UIT but also that a focus on encouraging
  older applicants to apply is warranted.

30
Defending against cheating

• Unproctored tests should be followed by a
  proctored assessment where the identity of the
  applicant can be confirmed - the cheating or
  honest failers dont make it to the second,
  proctored stage in the proctored stage the
  cheating passers from the first stage are likely
  to score low and cause the honest passers to rise
  to the top.
• Warn applicants not to cheat - begin with strong
  warnings to applicants not to cheat and explain
  the consequences if they are caught cheating.
• Monitor and validate test scores over time.
  Despite data that show UIT scores in
  pre-employment testing are typically stable, UIT
  scores should be monitored and validated over
  time for any changes.