BroadBased Office Skills Testing

1
Broad-Based Office Skills Testing
Los Angeles County Office of Education PERSONNEL
COMMISSION
2
The Problem

• Many clerical/secretarial classifications
• Differ on specific job content and levels of
  responsibility
• Many obstacles to class consolidation -
  logistical, political, fiscal
• Each class has a unique examination
• Variably content and quality
• Not consistently updated
• 2-3 month recruitment examination cycle time
• No overall competency architecture across the
  classes
• Core competencies and competency progression
• Pass point paradoxes (lower pass points for
  higher level jobs based on selection ratios)

3
Purpose and Objectives

• Improve testing quality, reliability, and
  efficiency
• Improve coherence of clerical examination plans
• Reduce redundant testing
• Reduce time to place candidates on eligibility
  lists
• Ensure eligibility lists are always available

4
Strategy

• Core test battery for as many classifications as
  possible
• Annual calendar of assessment opportunities
  (e.g., quarterly)
• Computer-based testing
• Score banking and certification to different exam
  plans
• Different weights and cut-off scores for
  different classifications based upon relevancy
  and level needed.

5
Strategy Similar to SAT ACT for applying to
college

• Sign up and take the test
• Apply to the colleges of choice
• Send scores
• Colleges set their own pass-points and weights
  for final selection decisions

• Sign up and take the test
• Apply to the jobs of choice
• Scores populated
• Each job has its own pass-points and weights for
  final selection decisions

6
Score Data Flow
7
Development of the Test Battery

• Reviewed job analyses from Agency-Wide
  Classification Study - 2004-2007
• Re-stated duties into generalized work activity
  (GWA) statements with no reference to specific
  job content.
• Map GWAs to each classification
• Identify relevant assessment tools for GWAs
• Via GWAs, map assessment tools to Classifications
• Derive relevancy and weight of assessment tools
  to classifications.

8
Example GWAs

• Greet, interact, and provide information to
  others in person and on the telephone
• Verify, input, and correct data and information
• Receive, sort, amd categorize information and
  materials
• Understand, interpret, explain, and follow
  directions, rules, and regulations
• Operate a personal computer and other common
  electronic office equipment
• Format, edit, keyboard and type documents.
• Gather compile, collate, and assemble information
  and materials
• Maintain and coordinate calendars and schedule
  meetings and events
• Receive, track, and record monetary/numerical
  information and data

9
Matrix of GWA with Classifications
10
Competencies Assessed - Tests

• Following Instructions (MC)
• English Usage Grammar (MC)
• Microsoft Outlook (MC)
• Microsoft Excel (Performance, OPAC)
• Microsoft Word Skills (Performance, OPAC)
• Telephone - Customer Service (SJ, OPAC)
• Sequencing and ordering speed and accuracy
  (timed)
• Computational speed and accuracy (timed)
• Checking and comparing speed and accuracy (timed)
• Data Entry speed and accuracy (OPAC)

11
Matrix of Tests with GWAs
12
Matrix of Tests with Classifications
13
Result

• Linkage of tests-to-tasks-to-classifications to
  establish job relatedness
• Derived weighting scheme for each classification
• Relevant tests
• Relevant weights
• Job specific competencies identified in the job
  analyses are assessed in specialized examination
  parts
• Min Quals
• Interview
• Performance

14
Applicable Classifications

• Clerk
• Intermediate Clerk
• Senior Clerk
• School Clerk
• Senior School Clerk
• Data Control Clerk
• Senior Data Control Clerk
• Typist Clerk
• Intermediate Typist Clerk
• Temporary Office Worker
• Senior Typist Clerk
• Department Assistant, Dance
• Department Assistant, Music

• Department Assistant, Theater
• Department Assistant, Visual Arts
• Receptionist
• Reader
• Information Resources Specialist
• Media Dispatching Clerk
• Secretary
• Division Secretary
• Legal Secretary
• School Administrative Secretary
• Senior Division Secretary
• Executive Legal Secretary
• Executive Assistant

15
Affected Positions
16
Administrative Practices

• Score Life
• 2 years
• Test Administration
• Every 3 months
• Test Retake
• Once every 6 months
• Retake Scope
• Whole Battery not individual tests for
  logistical reasons
• Weighted or Qualifying-Only
• Qualifying Pass/Fail-only for Promotional Exams
• Weighted for Open recruitment

17
Administrative Practices

• Effective Scores
• Only the candidates most recent score, not the
  highest score, can be certified to a list.
• For candidates whose scores enable them to be
  placed on an eligibility list (pass the exam),
  the certified scores stands for the duration of
  eligibility on the respective list.
• Scores already certified are not updated by
  retakes
• Ones score (and relative ranking) on a list can
  not be improved until eligibility expires (one
  year).
• Multiple (10) Pass Points
• Low individual test pass points
• Reduced Adverse Impact
• 3 apparent factors
• Computer skills, speed and accuracy, verbal

18
Recruitment

• Continuous offering of the BBC
• Continuous recruitment for all affected
  classifications
• Establishes a pre-tested pool of potential
  candidates for the affected classes
• For most classifications, limited or no public
  posting for job specific recruitments
• Notifications sent to pre-tested pool, based on
  test cut-off score
• Invitations to apply to take the specialized exam
  parts for the classification

19
No need to go to the general population for each
recruitment
20
Status

• Officially launched on February 9th, 2009
• Advertised on our PC website Weekly Job
  Opportunities emails.
• Two exam administrations in March April phased
  in the BBC in the context of specific
  classification recruitments.
• June and October 2009 BBC administrations as part
  of annual calendar

21
Process Improvements Since Start

• Candidates are automatically notified via email
  when particular job classifications are open for
  recruitment.
• Candidates receive their score report along with
  an Examination Information Sheet to address
  frequently asked questions.
• Scores can (soon) be viewed on-line with ID and
  Password
• Candidates who miss the Scantron exam portion can
  re-register to take the next exam administration.
  
• Candidates who are unable to attend the first
  Computer Exam can attend the Computer Make-up
  exam typically scheduled the following week.

22
Benefits Received and Expected

• Test quality improvement through
• Data monitoring and psychometric analysis
• Empirical validation research
• Development of alternate forms
• Administration efficiencies through
• Managed efficiencies in test administration and
  score managements
• Minimizing retesting of the same candidates for
  different classifications
• Improved communications with candidates
• Recruitment enhancement through
• Maintaining a pre-tested pool of applicants
• Having something to offer at job fares
• Reduced eligibility list production cycle time

23
Next Project BBA

• Broad-Based Administrative Classifications
• Administrative, Analyst, and Professional Series
• Focus on college grads
• Same general model
• Launch in Q1, 2010

24
Thank you!

• Questions?
• Comments?