Human Performance Assessment and Modeling Laboratory

1
The Human Performance Assessment and Modeling
(HPAM) Laboratory

• Director Ling Rothrock, Ph.D
• Associate Professor,
• The Harold and Inge Marcus Department of
  Industrial and Manufacturing Engineering
• The Pennsylvania State University
• 210 Leonhard Building
• University Park, PA 16802
• Phone [phone]
• Email lxr28_at_engr.psu.edu

2
Laboratory Mission
To form a mutually-supporting triad with
human-in-the-loop simulations informing the
development of inductive learning models and
testing hypotheses grounded in cognitive
science
3
Investigating Team Performance Using a
Human-in-the-loop Discrete-Event Simulation System
Construction and utilization of the Team Aegis
Simulation Platform (TASP) to investigate
Modeling for Effective Team Training (METT)
Sponsor US Navy
4
Roles
Performance
HITL Test Bed TASP
Interaction
Scenarios
User Model
Scenario Pool
5
Investigating Compensatory and Noncompensatory
Decision Making Strategies in Dynamic Task
Environments

• Research Team
• Ling Rothrock, Yin Jing (Penn State)
• Alex Kirlik (UIUC)
• Sponsor NSF DRMS

6
Compensatory Lens Model (CLM)
Rule-based Lens Model (RLM)
7
Systematic Analysis of Operator Strategies in
Missile Defense Implications toward
Visualization Design

• Sponsor Alion Science and Technology
• (MicroAnalysis and Design)

8
Experiment 1 Format
Experiment 2 Format
Question 1. The country is currently under attack
and the following two defense plans have been
proposed If Plan A is adopted, there is a
0.33 probability that 4,600,000 people will be
saved, there is a 0.66 probability that
4,400,000 people will be saved, and there is a
0.01 probability that no one will be saved If
Plan B is adopted, 4,400,000 people will be
saved
Experiment 3 Format
9
Development of VHMS Prognostic Capability for the
Heavy Brigade Combat Team (ongoing)

• Sponsor US Army

10
(No Transcript)
11
Investigating Refinery Control Room Display
Visualizations (ongoing)

• Sponsor Abnormal Situation Management Consortium
  (Honeywell)

12
Heat Map
Visual Thesaurus
Parallel Coordinates
13
Publications

• Altuntas, B., Wysk, R. A., Rothrock, L. (2004).
  A Formal Approach to Include A Human Material
  Handler in A CIM System. Paper presented at the
  Institute of Industrial Engineers Annual
  Conference, Houston, TX.
• Altuntas, B., Wysk, R. A., Rothrock, L. (2007).
  A Formal Approach to Include a Human Material
  Handler in a CIM System. International Journal of
  Production Research, 45(9), 1953-1971.
• Barron, K., Simpson, T. W., Rothrock, L.,
  Frecker, M., Barton, R. R., Ligetti, C. (2004,
  Sep 28-Oct 2 2004). Graphical user interfaces for
  engineering design Impact of response delay and
  training on user performance. Paper presented at
  the 2004 ASME Design Engineering Technical
  Conferences and Computers and Information in
  Engineering Conference, Salt Lake City, UT,
  United States.
• Bejarano, D., Rothrock, L., Warner, J. (2001,
  August 10, 2001). Using Time Windows to Aid
  Situation Comprehension in a Highly Automated
  Control Environment. Paper presented at the 9th
  International Conference on Human-Computer
  Interaction, New Orleans, LA.
• Bhandarkar, D., Thiruvengada, H., Rothrock, L.,
  Campbell, G. E., Bolton, A. E. (2004, May
  15-18, 2004). TASP A Toolkit to Analyze Team
  Performance in a Complex Task Environment. Paper
  presented at the Annual Conference of the
  Institute of Industrial Engineers, Houston, TX.
• Bolton, A. E., Campbell, G. E., Buff, W. L.,
  Rothrock, L., Bhandarkar, D., Kukreja, U.
  (2004). TASP A Flexible Alternative for Team
  Training and Performance Research. Paper
  presented at the Interservice/Industry Training,
  Simulation Education Conference (I/ITSEC),
  Orlando, FL.

14
Publications (cont.)

• Harvey, C. M., Daigle, T., Darisipudi, A.,
  Rothrock, L., Nabatilan, L. (2006).
  Multidisciplinary Systems Teams. In Handbook of
  Industrial and Systems Engineering (pp. 11-11 to
  11-19). Boca Raton, FL CRC Press.
• Harvey, C. M., Koubek, R. J., Rothrock, L.,
  Darisipudi, A., Kim, J. W., Munch, J. (2005).
  Cognitive Ergonomics. In R. Proctor K.-P. L. Vu
  (Eds.), Handbook of Human Factors in Web Design
  (pp. 70-89). Mahwah, NJ Lawrence Erlbaum
  Associates.
• Hodge, K. A., Rothrock, L., Kirlik, A. C.,
  Walker, N., Fisk, A. D., Phipps, D. A., et al.
  (1995). Training for Tactical Decision Making
  Under Stress Towards Automatization of Component
  Skills (Human Attention Performance Laboratory
  Technical Report-9501 Georgia Institute of
  Technology, School of Psychology.
• Hodge, K. A., Rothrock, L., Krosnick, W., Nagel,
  K. L., Kirlik, A. C., Walker, N., et al. (1993).
  Training for Tactical Decision Making Under
  Stress An Empirical Investigation of Part-Task
  Training for Perceptual and Rule-based Decision
  Making (Human Attention Performance Laboratory
  Technical Report-9301 Georgia Institute of
  Technology, School of Psychology.
• Jin, J., Rothrock, L. (2005). A Visualization
  Framework for Bounding Physical Activities --
  Towards a Quantification of Gibsonian-based
  Fields. Paper presented at the 49th Annual
  Meeting of the Human Factors and Ergonomics
  Society, Orlando, Florida.

15
Publications (cont.)

• Jin, J., Rothrock, L., McDermott, P., Barnes,
  M. (2010). Using Analytic Hierarchy Process to
  Examine Judgment Consistency in a Complex
  Multi-Attribute Task. IEEE Transactions on
  Systems, Man and Cybernetics, Part A, 40(5),
  1105-1115.
• Kim, N., Rothrock, L., Wysk, R. A., Joo, J.
  (2010, December 5-8). An Affordance-based
  Formalism for Modeling Human-involvement in
  Complex Systems for Prospective Control. Paper
  presented at the Winter Simulation Conference,
  Baltimore, Maryland.
• Kim, N., Shin, D., Wysk, R. A., Rothrock, L.
  (2008, May 18-20, 2008). The Hierarchical
  Modeling of Human Planning Activities with
  Affordances Using Finite State Automata (FSA).
  Paper presented at the Annual Conference of the
  Institute of Industrial Engineers, Vancouver, BC.
• Kim, N., Shin, D., Wysk, R. A., Rothrock, L.
  (2009). Using finite state automata (FSA) for
  formal modelling of affordances in human-machine
  cooperative manufacturing systems. International
  Journal of Production Research, 48(5), 1303-1320.
• Kirlik, A., Fisk, A. D., Walker, N., Rothrock,
  L. (1998). Feedback Augmentation and Part-Task
  Practice in Training Dynamic Decision-Making
  Skills. In J. A. Cannon-Bowers E. Salas (Eds.),
  Making Decisions Under Stress Implications for
  Individual and Team Training (pp. 91-113).
  Washington, D.C. American Psychological
  Association.

16
Publications (cont.)

• Ma, J., Kim, N., Rothrock, L. (2011).
  Performance Assessment in an Interactive Call
  Center Workforce Simulation. Simulation Modelling
  Practice and Theory, 19, 227-238.
• McDermott, P., Hutchins, S., Barnes, M.,
  Koenecke, C., Gillan, D., Rothrock, L. (2003).
  The Presentation of Risk and Uncertainty in the
  Context of National Missile Defense Simulations.
  Paper presented at the Human Factors and
  Ergonomics Society 47th Annual Conference,
  Denver, Colorado.
• Munch, J., Rothrock, L. (2003). Modeling Human
  Performance in Supervisory Control Informing
  Adaptive Aid Design. Paper presented at the
  Annual Conference of the Institute of Industrial
  Engineers, Portland, OR.
• Oyewole, S. A., Haight, J. M., Freivalds, A.,
  Cannon, D. J., Rothrock, L. (2010). Statistical
  evaluation and analysis of safety intervention in
  the determination of an effective resource
  allocation strategy. Journal of Loss Prevention
  in the Process Industries, 23(5), 585-593.
• Park, S., Bolton, G. E., Rothrock, L., Brosig,
  J. (2010). Toward an Interdisciplinary
  Perspective of Training Intervention for
  Negotiations Develop Strategic Negotiation
  Support Contents. Decision Support Systems, 49,
  213-221.
• Park, S., Rothrock, L. (2007). Systematic
  Analysis of Framing Bias in Missile Defense
  Implications toward Visualization Design.
  European Journal of Operational Research, 182,
  1383-1398.

17
Publications (cont.)

• Repperger, D. W., Rothrock, L. (2002). A Dual
  Haptic Interface Investigation for Improved
  Human-Computer Interaction (Technical Report
  AFRL-HE-WP-TR-2002-0189). Wright-Patterson AFB,
  OH United States Air Force Research Laboratory.
• Repperger, D. W., Rothrock, L. (2002, March 5,
  2002). Using Genetic Algorithms to Help Design
  Human Machine Interface Systems. Paper presented
  at the Dayton-Cincinnati Aerospace Science
  Symposium, Dayton, OH.
• Repperger, D. W., Rothrock, L. (2005). Using
  GA-based Intelligent Control Means to Enhance
  Human-Machine Interfaces. The International
  Journal of Intelligent Automation and Soft
  Computing (AutoSoft), 11(2), 123-140.
• Rothrock, L. (1999). Genetics-Based Inductive
  Inference Model to Represent Human Decision
  Strategies in a Supervisory Control Task. Paper
  presented at the Artificial Neural Networks in
  Engineering Conference, St. Louis, MO.
• Rothrock, L. (2000). Developing Performance
  Metrics in Pre-Deployment Systems Using the
  Abstraction Hierarchy. Paper presented at the
  Military, Government and Aerospace Simulation
  Symposium of the 2000 Advanced Simulation
  Technologies Conference, Washington, D.C.

18
Publications (cont.)

• Rothrock, L. (2000). Domain-relevant Performance
  Analysis in Human-in-the-loop Discrete-event
  Simulations. Paper presented at the 2000 Summer
  Computer Simulation Conference, Vancouver, B.C.
• Rothrock, L. (2001). Using Time Windows to
  Evaluate Operator Performance. International
  Journal of Cognitive Ergonomics, 5(1), 1-21.
• Rothrock, L. (2002, 10 April 2002). Adaptive
  Aiding Using Physiological Operator Functional
  State Assessment. Paper presented at the Third
  Annual Joint AFRL/DAGSI Research Symposium,
  Dayton, OH.
• Rothrock, L., Barron, K., Simpson, T. W.,
  Frecker, M., Ligetti, C., Barton, R. R. (2004,
  Sept. 20-24). Applying the Proximity
  Compatibility and the Control-Display
  Compatibility Principles toward Engineering
  Design Using Configural Interfaces. Paper
  presented at the 48th Annual Meeting of the Human
  Factors and Ergonomics Society, New Orleans, LA.
• Rothrock, L., Barron, K., Simpson, T. W.,
  Frecker, M., Ligetti, C., Barton, R. R. (2005).
  Applying the proximity compatibility and the
  control-display compatibility principles to
  engineering design interfaces. Journal of Human
  Factors and Ergonomics in Manufacturing, 16(1),
  61-81.
• Rothrock, L., Cohen, A., Yin, J., Thiruvengada,
  H., Nahum-Shani, I. (2009). Analyses of Team
  Performance in a Dynamic Task Environment.
  Applied Ergonomics, 40(4), 699-706.

19
Publications (cont.)

• Rothrock, L., Harvey, C. (2002, 27-31 January,
  2002). Synthetic Task Environments for the study
  of collaboration in supervisory control teams.
  Paper presented at the 2002 Western
  Multiconference, Collaborative Technologies
  Symposium, San Antonio, TX.
• Rothrock, L., Harvey, C. M., Burns, J. (2005).
  A Theoretical Framework and Quantitative
  Architecture to Assess Team Task Complexity in
  Dynamic Environments. Theoretical Issues in
  Ergonomics Science, 6(2), 157-172.
• Rothrock, L., Kirlik, A. (2000). The Use of
  Inductive Inference Models to Understand Human
  Performance in Supervisory Control Domains. Paper
  presented at the 2000 IEEE International
  Conference on Systems, Man, and Cybernetics,
  Nashville, TN.
• Rothrock, L., Kirlik, A. (2003, June 22-27).
  Explorations in Modeling Human Decision Making in
  Dynamic Contexts. Paper presented at the 5th
  International Conference on Engineering
  Psychology and Cognitive Ergonomics, Crete,
  Greece.
• Rothrock, L., Kirlik, A. (2003). Inferring
  rule-based strategies in dynamic judgment tasks
  toward a noncompensatory formulation of the lens
  model. IEEE Transactions on Systems, Man, and
  Cybernetics, Part A, 33(1), 58-72.

20
Publications (cont.)

• Rothrock, L., Kirlik, A. (2006). Inferring Fast
  and Frugal Heuristics from Human Judgment Data.
  In A. Kirlik (Ed.), Adaptive Perspectives on
  Human-Technology Interaction Methods and Models
  for Cognitive Engineering and Human-Computer
  Interaction (pp. 131-148). Oxford Oxford
  University Press.
• Rothrock, L., Koubek, R., Fuchs, F., Haas, M.,
  Salvendy, G. (2002). Review and Reappraisal of
  Adaptive Interfaces Toward Biologically-Inspired
  Paradigms. Theoretical Issues in Ergonomics
  Science, 3(1), 47-84.
• Rothrock, L., Nembhard, D. A. (2006).
  Team-in-the-loop Simulations Advances in the
  Study of Collaboration and Conflict. In W.
  Karwowski (Ed.), International Encyclopedia of
  Ergonomics and Human Factors (2nd ed., pp.
  2407-2414). Boca Raton CRC/Taylor Francis.
• Rothrock, L., Park, S., Barnes, M., McDermott,
  P., Hutchins, S., Gillan, D. (2003). Systematic
  Analysis of Risk Visualization Strategies for
  Homeland Defense. Paper presented at the IEEE
  International Conference on Systems, Man
  Cybernetics, Washington, D.C.
• Rothrock, L., Prasad, K. S., Harvey, C.,
  Narayanan, S. (2004). A configurable telerobotics
  system for human factors education and research.
  International Journal of Modelling and
  Simulation, 24(4), 239-249.
• Rothrock, L., Ramachandran, K. (2002, July
  15-19, 2001). Use of a Configurable Telerobotics
  System to Measure and Model Human Supervisory
  Control Performance. Paper presented at the
  Summer Computer Simulation Conference 2002, San
  Diego, CA.

21
Publications (cont.)

• Rothrock, L., Ventura, J., Park, S. (2003). An
  Optimization Methodology to Investigate Operator
  Impact on Quality of Service. Paper presented at
  the Interservice/Industry Training, Simulation,
  and Education Conference (I/ITSEC), Orlando, FL.
• Rothrock, L., Wysk, R. A., Kim, N., Shin, D.,
  Son, Y.-J. (2011). 20th ICPR A Formalism for
  Human-Machine Cooperative Systems. International
  Journal of Production Research, 49(14),
  4263-4273.
• Rothrock, L., Yin, J. (2008). Integrating
  Compensatory and Noncompensatory Decision Making
  Strategies in Dynamic Task Environments. In T.
  Kugler, J. C. Smith, T. Connolly Y. J. Son
  (Eds.), Decision Modeling and Behavior in
  Uncertain and Complex Environments (pp. 125-141).
  New York Springer.
• Shin, D., Rothrock, L. (2005, August 13-15). A
  Formal Model of Human-Automation Interaction.
  Paper presented at the International Workshop on
  Robot and Human Interactive Communication,
  Nashville.
• Shin, D., Wysk, R. A., Rothrock, L. (2004,
  IMECE 2004, Nov. 13-19 2004). Modeling and
  analysis of human task-performing process in
  manufacturing systems with human task and error
  classifications. Paper presented at the 2004 ASME
  International Mechanical Engineering Congress and
  Exposition, Anaheim, CA.
• Shin, D., Wysk, R. A., Rothrock, L. (2006). A
  Formal Control-Theoretic Model of
  Human-Automation Interactive Manufacturing System
  Control. International Journal of Production
  Research, 44(20), 4273-4295.

22
Publications (cont.)

• Shin, D., Wysk, R. A., Rothrock, L. (2006).
  Formal Model of Human Material-Handling Tasks for
  Control of Manufacturing Systems. IEEE
  Transactions on Systems, Man and Cybernetics,
  Part A, 36(4), 685-696.
• Shin, D., Wysk, R. A., Rothrock, L. (2006). An
  Investigation of a Human Material Handler on Part
  Flow in Automated Manufacturing. IEEE
  Transactions on Systems, Man, and Cybernetics,
  Part A, 36(1, Special Issue on Advances in
  Heterogeneous and Complex System Integration),
  123-135.
• Simpson, T. W., Barron, K., Rothrock, L.,
  Frecker, M., Barton, R. R., Ligetti, C. (2007).
  Impact of Response Delay and Training on User
  Performance with Text-Based and Graphical User
  Interfaces for Engineering Design. Research in
  Engineering Design, 18, 49-65.
• Simpson, T. W., Frecker, M., Barton, R. R.,
  Rothrock, L. (2007). Graphical and Text-based
  Design Interfaces for Parameter Design of an
  I-Beam, Desk Lamp, Aircraft Wing, and Job Shop
  Manufacturing System. Engineering with Computers,
  23, 93-107.
• Simpson, T. W., Iyer, P., Rothrock, L., Frecker,
  M., Barton, R. R., Barron, K., et al. (2005,
  April 18-21). Metamodel-Driven Interfaces for
  Engineering Design Impact of Delay and Problem
  Size on User Performance. Paper presented at the
  46th AIAA/ASME/ASCE/AHS/ASC Structures,
  Structural Dynamics Materials Conference,
  Austin, TX.

23
Publications (cont.)

• Thiruvengada, H., Bhandarkar, D., Kukreja, U.,
  Rothrock, L. (2004, May 17-20, 2004). TASP A
  toolkit for human experimentation in a synthetic
  environment with a hybrid human-agent team. Paper
  presented at the Behavior Representation in
  Modeling and Simulation, Crystal City, VA.
• Thiruvengada, H., Bhandarkar, D., Rothrock, L.
  (2006). Time Windows-based Team Performance
  Measures Design and Implementation. Paper
  presented at the 2006 IEEE International
  Conference on Systems, Man, and Cybernetics,
  Taipei, Taiwan.
• Thiruvengada, H., Kim, N., Rothrock, L., Wysk,
  R. A. (2008, May 18-20, 2008). A Systematic
  Approach for Developing a Computable Model for
  Gibson's Affordance Using Finite State Automata.
  Paper presented at the Annual Conference of the
  Institute of Industrial Engineers, Vancouver, BC.
• Thiruvengada, H., Rothrock, L. (2006, May
  21-24, 2006). HCI Implications of a Colored Petri
  Net Model for Gibson's Affordance. Paper
  presented at the Annual Conference of the
  Institute of Industrial Engineers, Orlando, FL.
• Thiruvengada, H., Rothrock, L. (2007).
  Affordance-based Computational Model of Driver
  Behavior on Highways A Colored Petri Net
  Approach. Paper presented at the 2007 IEEE
  International Conference on Systems, Man and
  Cybernetics, Montreal.

24
Publications (cont.)

• Thiruvengada, H., Rothrock, L. (2007). Team
  Training in Complex Environments. In D. A.
  Nembhard (Ed.), Workforce Cross Training (pp.
  181-207). New York CRC Press.
• Thiruvengada, H., Rothrock, L. (2007). Time
  Window-based Performance Measures A Framework to
  Measure Team Performance in Dynamic Environments.
  Cognition, Technology Work, 9(2), 99-108.
• Wheatly, M., Narayanan, S., Koubek, R. J.,
  Harvey, C., Rothrock, L., Smith, P., et al.
  (2003). Biologically Inspired Analysis of Complex
  Systems Back to Nature. Paper presented at the
  2003 Human-Computer Interaction Conference,
  Crete, Greece.
• Yin, J., Rothrock, L. (2006). A Rule-based Lens
  Model. International Journal of Industrial
  Ergonomics, 36(5), 499-509.
• Yin, J., Rothrock, L., Kirlik, A. (2005). Using
  the Lens Model Framework to Characterize
  Compensatory and Noncompensatory Decision Making
  Strategies. Paper presented at the 11th
  International Conference on Human-Computer
  Interaction, Las Vegas, NV.