A Validation Model for Functional Job Analysis

1
A Validation Model for Functional Job Analysis

• Richard G. Best1
• Steven F. Cronshaw2,3
• Sylvia J. Hysong1
• Frank I. Moore1,4
• Melissa Warner2,3
• Lynda Zugec2,3

• Veterans Evidence-Based Research Dissemination
  and Implementation Center, South Texas Veterans
  Health Care System
• Department of Psychology, University of Guelph
• Organization Management Solutions, Inc.
• School of Public Health, University of Texas
  Health Science Center at Houston

2
Questions Addressed by Five Validation Strategies
3
Five FJA Validation Strategies Cross-Referenced
To Inferences Tested In This Study
4
Linguistic Validation

• Is the job language in the task bank adequately
  controlled as required by FJA theory and
  methodology?
• Control is a fundamental principle(Fine
  Cronshaw, 1999)

5
Aspects of Job Language

• Semantics - Concerned with the meaning and
  meaningfulness of terms and sentences in the FJA
  task bank
• Syntax - Concerned with the rules for linking
  terms (words) into correctly constructed task
  statements
• Pragmatics - Involves the use of language by
  flesh and blood speakers to achieve their goals
  when developing and using the FJA task bank
  (Baghramian, 1998, 2002)

6
Achieving Linguistic Validation Across All 3
Aspects

• Systematic, theory driven approach (FJA)
• Correspondence between FJA principles
  statements (FJA Certified, Edited, Consensus)
• Quality Control (Rating, Interrater Agreement,
  Focus Group Feedback/Clarity)

7
Who has an Influence on Linguistic Validity???

• SMEs Facilitators Editors Raters

8
Experiential Validation

• Answers the question Does the task bank
  describe the workers experience on the job?

9
Validating Task Data Experientially

• Edit task statements so they are linguistically
  valid
• Ask of your SMEs
• How clearly and accurately does each task
  statement describe the actual work performed?
• What percent of the work you do is captured in
  this list of task statements?
• Refine task statements as necessary

10
Experiential Validation from Primary Care Job
Titles
11
The Challenge

• How do you validate a task bank experientially
  when the task bank includes multiple job titles
  with overlapping responsibilities?

12
Experiential Validation of a Domain of Work

• Validate as usual for each job title
• Refine and merge overlapping and/or redundant
  tasks (with SME help)
• Compile tasks from all job titles into master
  task bank
• Survey SMEs across all job titles
• Task endorsement
• Frequency / Duration
• Criticality
• of work captured by tasks endorsed

13
Why the extra work?

• Eliminate redundant tasks across job titles
• Capture variability in experiential validity
  across worker types (i.e., job titles)
• Identify and control for biases

14
Domain of Work ??(Job Titles)
15
Ecological Validation

• Requires complete coverage of the work context
  being analyzed
• Individual position or
• Job/work domain
• Situations are the unit of analysis
• Are adequate situations included in the sample?
• Differs from traditional focus on people

16
Ecological Validation

• VHA primary care work the relevant context
• Substantial variability in facilities!!!
• Size, academic affiliation, original focus,
  geographic/regional differences, etc.
• Transportability system-wide is a huge issue!
• Uptake and utilization requires ecological
  validity

17
Ecological Validation

• Inherent Assumption
• Primary care work is consistent across settings
• Who performs the work is NOT consistent across
  settings

18
Ecological Validation

• Ecological sampling frame
• Inclusion criteria recommended by EAP
• Designed to maximize sampling adequacy
• 3 primary criteria
• Size
• Academic affiliation
• Degree of service line implementation

19
Ecological Validation

• Roll-out process across settings
• Redundant tasks removed from final task bank
• Tasks from core titles were merged into the final
  task bank
• External validity effort
• Survey to all PC personnel in all participating
  sites

20
Ecological Validation

• PC Physician
• Identified 158 tasks endorsed by physicians
• 70.3 of these performed by 2 or more physicians
  in 4 of the 7 sites

21
Hypothetico-Criterial Validation

• Extent to which the task bank can be used to test
  theoretically and empirically grounded hypotheses
• Measured at task level
• Parallels construct and criterion-related
  validation

Conceptual FJA Theory, including the
Functional Hierarchy
Measurement Functional Job Analysis Ratings of
Tasks Statements
22
Hypothetico-Criterial Validation

• Integration of two Theories
• FJA Theory
• Sociotechnical Systems Theory
• Empirical evidence for integration of theories to
  produce scientifically-testable hypotheses
• Cronshaw and Alfieri (2003 2005)
• Current research

23
Hypothetico-Criterial Validation Next Steps

• Test more propositions drawn from FJA Theory
  through operational definition via the FJA scales
  (TDP functional skills)
• Correlate FJA ratings (TDP) to relevant
  performance outcomes

24
Social-organizational Validation

• Does the task bank provide users and other
  organizational participants with relevant
  information that they will be motivated to
  consult and use?
• Extrinsic motivation
• 60-70 percent of the non capital costs of health
  care are for personnel compensation, training and
  turnover.

25
Social-organizational Validation

• Does the task bank provide users and other
  organizational participants with relevant
  information that they will be motivated to
  consult and use?
• Extrinsic motivation
• Information to improve HR decisions is limited to
  worker traits and experience and to a lesser
  extent the context of the job. Information about
  the job content is lacking.

26
Social-organizational Validation

• Does the task bank provide users and other
  organizational participants with relevant
  information that they will be motivated to
  consult and use?
• Extrinsic motivation
• There is a legal requirement for the use of
  validated job content as the basis for hiring,
  promotion and performance evaluation.

27
Social-organizational Validation

• Does the task bank provide users and other
  organizational participants with relevant
  information that they will be motivated to
  consult and use?
• Intrinsic motivation
• A language to describe ones work and its
  contribution to the organizations goals and
  objectives in intrinsically satisfying.

28
Social-organizational Validation

• Does the task bank provide users and other
  organizational participants with relevant
  information that they will be motivated to
  consult and use?
• Intrinsic motivation
• Fairness and justice in performance assessment
  depend upon a common understanding of the work to
  be performed and the standards used to assess it.

29
Social-organizational Validation

• What are the social-organizational barriers to
  use of task banks?
• Information sharing Power sharing
• Both Professional organizations and Employers
  claim the right to define the job role.
• Educators also claim a Divine right to
  determine competencies for worker qualifications

30
Social-organizational Validation

• What are the social-organizational barriers to
  use of task banks?
• A common language for discussing work content and
  the associated worker traits that can be linked
  to the organizations purpose, goals and
  objectives is not readily understood by all the
  types of users.

31
Social-organizational Validation

• So, what can we do to overcome the barriers to
  use?
• Involve the leaders of employee unions,
  professional associations, employer alliances and
  educators in the design of the task data
  development.
• Assure all users have a stake in the outcome
• Use Subject Matter Experts to generate the task
  statements from the perspective of the worker.

32
Social-organizational Validation

• So, what can we do to overcome the barriers to
  the use of the task bank?
• Verify, Verify, Verify
• Keep local partners in the loop and involved in
  the refinements
• Co-create applications of the TDB to local
  problems and questions.
• Facilitate the use by demonstrations and
  coaching.