Lecture 2: Development of Expertise

1
Lecture 2 Development of Expertise

• IE 8541 Intelligent Decision Support Systems

2
Lecture Outline

• My contract info and background
• Development process for a DSS. Course structure
  mirrors this process.
• Course topics
• Schedule for Jan/Feb, deadlines
• Institutional Review Board
• Intro to Human Information Processing
• Discussion of Article Development of Expertise

3
My Contact Information

• Instructor Prof. Hayes
• Email hayes_at_me.umn.edu
• Phone [phone]
• Office ME 2110
• Office Hours By appointment
• Course Website
• www.me.umn.edu/education/courses/ie8541/index.html

4
My Background

• Interdisciplinary mix of
• Engineering,
• Computer Science and
• Psychology.
• PhD, Carnegie Mellon University, Robotics, 1990
• Faculty at U. Illinois, Computer Science for 7
  years
• Faculty at U. Minnesota, Mechanical Engineering
  for 9 years.

5
Past and Present Work

• Decision Support for complex problem solving
• Manufacturing
• Engineering and Architectural Design
• NASA space operations
• Military planning
• Information analysis and visualization

• Basis Research on
• Differences in how experts and non-experts make
  decisions
• How to assess performance in complex problem
  solving
• Making globally distributed teams of product
  designers more effective
• Other
• Robotic assistance for the handicapped.

6
The development process of a DSS

• Identifying appropriate task and role for DSS
• Requirements gathering
• Design specification
• Early prototype construction mock up or paper
  prototype
• Early prototype assessment and user feedback
• Redesign
• Prototype system construction
• Usability and performance testing

7
The Design Process for a DSS Human Centered
Design
A typical spiral design process
Prototype Testing
Prototype Construction
Requirements Gathering
Final Performance Evaluation or Comparison
Design Specification
Design Review
8
DSS Designers must consider all of
Tools (DSSs, software, etc.)
9
Course structure

• Topics and articles will reflect the development
  stages of DSSs to assist you in developing a
  successful project
• How do humans solve problems?
• Common methods for requirements gathering,
• Examples of decision support systems, and
  intelligent technologies employed,
• Early prototyping approaches paper prototypes,
  mock-ups.
• More on how people think at what tasks are the
  good or bad?
• DSS evaluation measuring software usability and
  problem solving performance, cost effectiveness,
  ROI.
• Issues impacting acceptance and adoption of DSSs
  work practices, who pays the cost and who reaps
  the benefits?

10
January/Early Feb. Schedule

• Tues 1/16 Class Introduction
• Thurs 1/18 Development of Expertise
• Tues 1/23 Verbal Reports as Data
• In class exercise demonstrating protocol
  analysis
• Thurs 1/25 Janus,
• Tues 1/30 Making Pedagogical Agents More
  Socially Intelligent,
• Thurs 2/1 Wickens Chapter 2, Research Methods
  Select article for presentation, IRB discussion
• Tues 2/6 Wickens Chapter 3, Design and Evaluation
  Methods, Project proposal due
• Tues 2/13 Ethnographic Approach to Design, IRB
  application due

11
Institutional Review Board

• An important part of any evaluation where human
  subjects interact with a program is IRB approval.
• IRB is aimed at insuring that the rights of human
  subjects are protected,
• IRB is also a coach in insuring success of
  experiments
• Next submission deadline Mon. 2/19
• http//www.research.umn.edu/irb/

12
An Introduction to Human Information Processing

• Perception
• Memory
• Cognition/decision making
• Response

13
Cognitive Engineering

• Improving the effectiveness, efficiency and
  safety of tasks performed primarily in the mind
  (e.g. decision making, planning, design, etc.)
• If we are to do so, we must first understand
• Information
• Human information processing (capabilities and
  limitations)

14
Cognitive Engineering

• Design or redesign the tools, work process, or
  work environment to improve cognitive task
  effectiveness

Motivation Incentives/rewards
Organizational structure
Work Process
Work environment
Tools (DSSs, software, etc.)
15
Information

• Cybernetics the study of information. Started
  in 1940s and 1950s.
• Bits a measure of Information one bit can
  represent two alternatives 1 or 0.
• H bits can represent n 2H alternatives
• H log2 n (n number of alternatives).

16
Example

• The 26 letters of the alphabet can be represented
  with 5 bits
• H log2 26 4.7 (round up to next integer)
• The word panda can be represented with 5
  letters x 5 bits/letter 25 bits.

17
Information Properties

• Bandwidth bits/second that can be transferred
  through a communication channel.
• There is much redundancy in written letters
  (66), spoken language, and communication systems
  in general.
• Redundancy can be important for error correction.

18
Human Information Processing Model (Sanders and
MCormick, 1993)
19
Sensory Stores Very short term memory for
sensory information

• One sensory store associated with each sensory
  input channel (vision, hearing, )
• Decay of information is very rapid,
• Information gone in 1 2 seconds.

20
Perception

• Perception categorization of incoming stimuli
  (Neibel and Freivalds).
• Detection (simplest form)
• Decide Is signal present or not present?
• Example peripheral vision test raise hand when
  you see a white dot appear.
• Classification determine the category in which a
  stimulus belongs. E.g. Red dot or white dot?
• Scene analysis understand a complex image.

21
Human Information Processing Model (Sanders and
MCormick, 1993)
22
Memory 3 types

• Sensory Store
• Short Term Memory (STM) or working memory
• Long Term Memory (LTM)
• Good review of memory in The Complete Problem
  Solver, by J. R. Hayes, 1981, Chapter Four The
  Structure of Human Memory.

23
Short Term Memory (STM)

• Items in current focus of attention,
• Holds 7 /- 2 chunks,
• Half life of 3 items is about 7 seconds,
• All items gone in 18 seconds.

24
Information Theory and the Brain

• During the 1950s attention to cybernetics raised
  the question in the minds of psychologists
• How much information can human memory hold?
• George Miller (1956) attempted to answer this
  question for short term memory (STM).

25
Information Capacity of STM

• Problem the amount of information that can be
  stored in short term memory depends on what a
  person knows about those items.
• People can remember in STM
• Many familiar concepts,
• Few unfamiliar concepts.
• Example can you remember this sentence?
• Wir kann mit Zug zu Euch kommen. (155 bits)

26
Information Capacity of STM

• Problem the number of items that can be stored
  in short term memory depends on what a person
  knows about those items.
• People can remember in STM
• Many familiar concepts,
• Few unfamiliar concepts.
• Example can you remember this sentence?
• Wir kann mit Zug zu Euch kommen. (125 bits)
• We can take the train to your place.

27
Chunks

• Chunk a package of information that is treated
  as a unit (Hayes, J. R., 1981).
• Miller (1956) proposed chunks as the units
  stored in memory.
• Chunks do not measure information, per se. They
  are collections of knowledge.

28
Chunks

• A chunk can reference (e.g. point to) a large
  body of knowledge in long-term memory.
• Example
• Last year Alito may have meant little to you.
• This year Alito may be linked to complex set of
  concepts supreme court justice,
  conservative-liberal power struggle, senate
  confirmation process, moderate swing votes

29
Long Term Memory (LTM)

• Information can potentially stay in LTM a
  lifetime.
• With out rehearsal, information in LTM tends to
  decay exponentially, biggest loss in first few
  days.
• One can have information in LTM, but be unable to
  get at it (tip-of-tongue phenomena) (Brown and
  McNeil, 1996)
• Information can be transferred from STM to LTM
  through a) rehearsal, b) association with
  concepts already in STM.

30
Examples of STM gt LTM

• Rehearsal
• Repeat telephone number or social security number
  until you remember it,
• Practice piano piece till you memorize it.
• Association with existing knowledge in LTM
• Lady introducing herself at a party Hi, Im
  Marsha Hoover like the vacuum cleaner.
• Method of Loci Associate items to be remembered
  with items on a path through your house
  described in Hayes, J. R., 1981 pp. 98 104).

31
Method of Loci

• Associate a list of items with a list of stops
  on a route through your house.
• Associate first item with first stop, second
  item with second stop, etc.
• Picture item interacting with something at its
  stop.
• To retrieve the list, walk the route through
  your house in your mind.

32
Attention
33
Attention

• Attention cognitive capacity devoted to a task
  (Neibel and Frievalds).
• Attention is largely a serial process
• If we try to attend to multiple tasks performance
  goes down. Wickens (1984) multiple resource
  theory predicts/explains performance decrement.

34
Attention

• Operator Vigilance sustained attention, ability
  to remain alert over a long period of time.
• Vigilance tends to go down after 30 min or so.
• Important in monitoring and inspection
• Truck driving, flying,
• Nuclear power plant monitoring,
• Visual inspection ..

35
Human Information Processing Model
36
Decision Making and Response Selection

• Decision making the process of considering
  alternatives, and choosing and appropriate
  response.
• Decision making may take
• Seconds what to do when a deer jumps in front of
  your car).
• Minutes what new route should an airplane take
  around a thunderstorm? Emergency room triage.
• Days (or longer) what launch plan should we
  follow for the next space mission?
• Perception, decision making and response
  selection are not always easily separable.
• Decision making may be undertaken by individuals
  or groups.

37
Some observed challenges in human decision making

• People tend to focus on only one alternative or
  hypothesis at a time once selected others are
  often ignored.
• Undue weight given to early clues, later ones
  often ignored.
• More information (even if relevant) can reduce
  decision making effectiveness (information
  overload).

38
More decision making challenges

• Not good at exhaustive exploration of a problem
  space, or remembering what has been explored and
  what has not.
• Not great at working with uncertainty.
• Variables with unknown values often ignored.

39
Class Discussion

• Development of Expertise

40
For Next Tuesday 1/23

• Protocols as Data
• Topic Protocol analysis
• A method for identifying where people can use
  decision support in complex problem solving
  tasks, requirements gathering, evaluation of
  joint performance of human and system