Shake, Rattle and Roles: Earthquake Engineering as HRO

1
Shake, Rattle and Roles Earthquake Engineering
as HRO

• Dan Horn
• Jeremy Birnholtz
• November 5, 2003

2
Presentation Outline

• Introduction
• High Reliability Organizations
• Earthquake Engineering
• Methods
• Findings
• Implications
• Next steps

3
High Reliability Organizations

• Weick (1999) lists 3 key characteristics
• Environment rife with errors to be detected
• Constant monitoring and reporting
• No anomaly too small
• Reluctant to simplify interpretations
• Integrate multiple, redundant sources
• Ongoing sensitivity to operations
• Collective mindfulness, heedful interrelating,
  the bubble etc.

4
Why study HROs here?

• Error detection is a key user goal, and therefore
  a key aspect of design
• Lessons are broadly applicable
• Weick suggests that ordinary orgs become HRO-like
  in crises
• Newell (1997) suggests that under extraordinary
  circumstances, the ordinary user becomes
  extraordinary

5
Experimental Structural Earthquake Engineering
(EE)

• Large scale physical test equipment
• Many forces that are complex and interacting
• Potential danger!

6
Structural Labs as HROs

• We argue that structural EE labs are a form of
  HRO
• 3 types of risk faced in these labs
• Catastrophic specimen failure
• Losing laboratory and field autonomy as
  (Galison, 1997) discusses in physics
• Risk of significant social cost concentrated
  social risk (Sims, 1999)

7
NEES Telepresence

• Teleobservation
• Watching tests from a distance
• Potentially many observers
• Passive vs. active
• Teleoperation
• Controlling apparatus during test
• Remote operations

8
Methods

• Interviews with 75 earthquake engineers
• Faculty, students and technicians
• Questions about
• Sequence of a typical test
• What they are looking at during a test
• Ongoing observation of EE tests
• Coding for themes (Huberman and Miles, 1994)

9
Findings

• Local Failure detection
• Variable Likelihood of failure
• Integrating sensory cues
• Multiple collocated persons
• Beliefs about telepresence
• Remote Failure Detection One story

10
Variable likelihood of failure

• More likely early and late
• Im always there for the first test on a
  particular specimen, because I need to train the
  students on the things they need to do like
  making sure the test frame is not creating a
  physical anomaly. Students have a tendency to
  just roll forward without checking these things.
• Early failure
• Dangerous and Costly
• Late failure
• Predictable and can be prepared for this

11
Integration of sensory cues

• Students rely on visual cues
• Visual displays of data (e.g. graphs)
• Walking around and looking at the specimen
• if we cant explain the graphs, we stop
  immediately. If we get data that are surprising,
  but not crazy well keep going
• More experienced integrate more cues
• Sound Even after we had fixed a problem with
  the test setup, there was still a lot of noise.
  I might have pushed the emergency stop button,
  as it was very noisy.
• Feel

12
Multiple Collocated Persons

• Reliance on multiple viewpoints
• different accounts of what happened, like
  peoples reports at the scene of a car accident
• Technicians say they will send somebody out to
  stand in a particular place and keep an eye on
  things.
• Frequent interaction
• When things go awry, we tend to powwow in the
  labto sort out whats going on.
• You have to argue

13
Beliefs About Telepresence

• Value in remote observation
• Dog-and-pony show value
• Real researchers plan to be at their tests
• Fears of remote operation
• Dont mess with my actuators
• Low-fidelity means low value

14
Remote Participation One Story

• One faculty member had remote participants in his
  shake table test when he was a graduate student.
  He had primitive video via html and people
  watching could email him.
• Valuable in that
• youre concentrating on one thing like maybe
  running the test, and someone emails you and
  says, hey whats that thats going on? and you
  look right there and you get a whole other
  opinion about whats going on
• Email is low-cost, persistent and not real-time
  Dont have 20 people yelling at you at once.

15
Implications

• Additional local observers are valuable
• But tend to have more correlated views
• Remote observers are constrained
• Cues are less rich
• Cues are mediated
• Can we design representations that exploit these
  constraints?
• Increase statistical probability of error
  detection

16
Formal Model

• P(DF)1-(1-P(d1F))(1-P(dnF))
• Person 1 .5
• Person 2 .4
• P(DF)1-(.5)(.6).7 70
• ASSUMES STATISTICAL INDEPENDENCE

17
Next step Lab Studies

• Background
• Task
• Expected results

18
Experiment Background

• Tower of Hanoi (TOH) Problem
• Rule 1 Only move 1 piece at a time
• Rule 2 Only move smallest FROM a peg
• Rule 3 Only place smallest ONTO a peg

19
Experiment Background

• Waitress and Orange TOH Isomorph (Zhang Norman,
  1994)
• Converts part of external representation into
  internal representation
• Leads to less efficient performance

20
(No Transcript)
21
Experiment Background

• Distributed Representations in TOH (Zhang, 1998)
• Waitress and Orange Problem
• Different Levels of Knowledge
• Expert R123
• Mid-level R12 or R13
• Novice R1
• Pairs Taking Turns
• R123-R1 vs R12-R13 vs R123-R123 vs R123

22
Zhangs Results Solution Time
23
Zhangs Results Steps
24
Zhangs Results Errors
25
Zhangs Results Summary

• Two experts are always better than one
• Two mid-levels are never better than one expert
• A novice is as helpful as a second expert in
  reducing errors

26
Beyond Zhang Proposed Study

• Payoff (to be piloted)
• 1.00 per solved problem
• -0.10 per move
• -0.40 per error
• Local person
• Knows all rules
• Complex display, full information
• Makes all moves

27
Beyond Zhang Proposed Study

• Remote person(s)
• Knows all rules
• Complex vs. Simple display (full vs. partial
  info)
• Can only reject moves
• Illegal
• Inefficient
• Either illegal or inefficient
• Rejections cost 0.20 each

28
Proposed Study

• Expected Outcomes (Remote Interface)
• Simple gt Complex in Error Detection
• Simple lt Complex in Inefficiency Detection

29
Implications

• How these lessons will inform design of these
  systems
• Could show value for remote folks
• Allow us to take this to EE context
• New expert-remote participants
• Must overcome speedbump of inefficiency

30
Questions?
31
(No Transcript)