23rd Annual Conference

1
23rd Annual Conference September 11-13, 2003 Wes
tin Harbour Castle
Toronto, Canada
2
Technology, Service and Delivery
September 12th, 2003 130 300 p.m. Dan Luk
CTO, Capstar

3
First TopicPoint and Click Using the
Internet to Enhance your Testing Program
4

• Why the Internet?
• Convenient
• Fast
• 24 X 7
• Efficient
• Cost-Effective

5

• Why Not the Internet?
• Security
• Control
• Fairness

6
Web Usage and GrowthSource http//www.nielsen-net
ratings.com/new.jsp on 1/8/2003
7
When to use the Internet?
Pre-Test
Post-Test
8
Test Development

• Web-based Test Development Tools
• Collaborative Computing
• Item Development and Review
• Resolve Geographical Differences
• Security
• Authentication
• Access Restrictions
• Encryption

9
Communications

• Types of Technology
• Push Technology
• e.g. e-mail
• Pull Technology
• e.g. Web Site

10
Push Technology

• Main Advantages
• Selective or Targeted
• Customized Contents
• Concise
• Main Disadvantages
• Spam
• Frequent Address Changes

11
Pull Technology

• Main Advantages
• General Availability
• Broad-based Information
• Information Available On-Demand
• Consistent Messaging
• Interactive Contents
• Main Disadvantages
• Must be Easy to Navigate
• Must Visit Frequently to View New Contents
• Random Arrival

12
Types of Information

• Bulletins
• Application Procedures
• Application Forms
• Study Materials
• Practice Exams
• Etc.

13
Registration/Scheduling

• Self-Service Vs. Call Center
• Accessibility
• Skills
• Online Help Vs. Help Desk
• High Availability
• Easy to Review Information or to Make Changes
• Implementation Methods
• Real-time
• Back-end Processed

14
Registration/Scheduling (Continued)

• Business Rule Enforcement
• Upper Vs. Lower Case
• Data Consistency Vs. Data Cleansing
• Browser Compatibility
• ADA Accommodation such as Section 508
  Compliancy
  
• E-commerce (Electronic Payment Required)
• Identity Theft

15
Test Delivery

• High Stake Vs. Low Stake
• Anyone Can Test Vs. Positive ID
• Proctored Vs. Self-proctored
• Connection Performance
• Timed Exams
• Test Driver Features
• Test Security
• Browser Compatibility

16
Data Management

• On-Demand Vs. FTP
• Dynamic Vs. PDF
• Registration/Scheduling Records
• Eligibility Processing
• Test Reporting
• Psychometric Analysis

17
Capstar Internet Usage

• Exceeded Expectations
• Anticipate Continued Growth
• Continued Area of Focus

18
Second TopicInnovative Uses of Technology In
Testing
19
When to use Technology?
Pre-Test
Post-Test
20
When to use Technology?
Lie!
Cheat!
Steal!
21
Lie

• Whom Are You Really Testing?

22
Lie

• Typical Preventive Measures
• Policies and Procedures as Deterrent
• Positive Identification Via Proctor

23
Traditional Identification Techniques

• Government Issued Pictured Ids
• ATT
• Tamper Resistance Paper and Printing
• Smart Cards
• Holographic Images
• Microprints
• Bar Code
• Data Strip

24
Biometrics
Recently, the term "Biometrics" has also been
used to refer to the emerging field of technology
devoted to identification of individuals using
biological traits, such as those based on retinal
or iris scanning, fingerprints, or face
recognition. http//stat.tamu.edu/Biometrics/

25
Biometrics (Continued)

• Access to Testing Area
• Authentication Process
• Re-entry to Testing Area (Same Day Vs. Future)
• Re-authentication Process (Same Day Vs. Future)
• Initial Testing Vs. Repeat Testing
• Same Person Vs. Different Person

26
Cheat

• Typical Preventive Measures
• Policies and Procedures as Deterrent
• Rules for Taking Breaks
• Business, Eligibility or Re-take Rules
• Proctored Exams
• Clean Test Rooms

27
Traditional Monitoring Techniques

• Active Proctoring
• Proctor Window
• Video and Audio Surveillance

28
Other Monitoring Techniques

• Remote Proctoring
• Remote Video and Audio Surveillance
• Back-end Monitoring
• Extreme Low Score
• Extreme High Score
• Sudden Score Fluctuation
• Time-to-respond
• Item Performance
• Form/Test Performance
• DB Cross-checking

29
Other Monitoring Techniques

• Biometrics
• Candidates Photo
• Access to Testing Area
• Authentication Process
• Re-entry to testing area
• Re-Authentication Process

30
Steal

• Reasons for Prevention to Include
• Intellectual Property Protection
• Exam Integrity
• Public Safety

31
Steal

• Typical Preventive Measures
• Policies and Procedures
• Business, Eligibility or Re-take Rules
• Rules for Taking Breaks
• Active Proctoring
• Proctor Window
• Video and Audio Surveillance

32
Other Techniques

• Remote Proctoring
• Remote Video and Audio Surveillance
• Back-end Monitoring
• Extreme Low Score
• Extreme High Score
• Sudden Score Fluctuation
• Time-to-respond
• Item Performance
• Form/Test Performance
• DB Cross-checking

33
Other Techniques (Continued)

• Initial Testing Vs. Repeat Testing
• Same Person Vs. Different Person
• New Items
• New Forms
• Continuous Testing Vs. Windowed Testing
• Software Algorithms
• Randomization
• Computer Adaptive
• Item Algorithms

34
Trade-off
Security
Cost
35
Other Technologies Used in Testing

• Improved Paper-and-pencil Process
• Dynamic Scanning and Scoring
• Fax-based Scoring
• Simulation-based Testing
• Innovative Item Types
• Computer-based Practical Testing
• 3-D Display

36
Other Technologies Used in Testing (Continued)

• Standards
• SCORM
• IMS QTI
• Other Testing Devices
• PDAs
• Tablet PCs
• IVR/Cell Phones
• Alternate Devices
• Wireless Technologies

37
Q A
38
Presentation Follow-up

• Please pick up a handout from this presentation
  AND/OR-
  
• Please give me your business card to receive an
  e-mail of the presentation materials OR-
  
• Presentation materials will be posted on CLEARs
  website

39
Technology, Service and Delivery
September 12th, 2003 130 300 p.m. Dan Luk
CTO, Capstar