Innovation, Environment and Expertise

1
Innovation, Environment and Expertise

• Kenneth Kotovsky
• Department of Psychology
• Carnegie Mellon University

2
(No Transcript)
3
Introduction

• Goal to link three somewhat disparate ideas to
  make an argument about creative invention or
  innovation, its contexts and development
• Some research done in collaboration with Jonathan
  Cagan on the engineering design process and its
  environmental context, on the assumption that a
  deeper process understanding enhances our chances
  of creating both process and educational
  interventions to stimulate creative innovation.
• Some classic findings with regard to the
  acquisition of expertise.
• A movement in higher education that can be put in
  service of the above.

4
1. Understanding the process problem solving
search within multiple representations(with
Jonathan Cagan)
5
Basic model applied to individual, group and
computational design agent process
6
Stimulating the Design Loop(with Jarrod Moss
Jonathan Cagan )

• What allows environmental input to be
  assimilated?
• Typical Experiment
• Remote Associate Test (RAT) problems (fox peep
  man) ? hole
• Answers for both new and previously unsolved
  problems presented in intervening task

Moss, J., K. Kotovsky, and J. Cagan, The
Influence of Open Goals in the Acquisition of
Problem Relevant Information, Journal of
Experimental Psychology Learning, Memory, and
Cognition, Vol. 33, No. 5, pp. 876-891, 2007.
7
Effect of hint on solution time (solved problems)
8
Experiment 2 Hint Timing
fixation
No fixation
9
Conclusions

• Hints more effective in presence of open goals.
• People carry unsolved problems with them and are
  implicitly sensitized to relevant environmental
  cues.
• Timing matters-hint effective after exploration
  dev. of open goal but before fixation (protocol
  studies).
• Method for impacting the design process by
  delivering useful hints, perhaps even
  automatically.

10
Transfer to Engineering Design(with Ian Tseng
Jarrod Moss Jonathan Cagan)

• How do open goals and hint assimilation affect
  engineering design?
• Generate as many time-keeping devices as possible
  using a list of 14 household objects.
• Explored early/late and close/distant hints

Tseng, I., Moss, R. J., Cagan, J. and Kotovsky,
K. Overcoming Blocks in Conceptual Design the
Effects of Open Goals and Analogical Similarity
on Idea Generation. 2008 ASME International
Design Engineering Technical Conferences and
Computers and Information in Engineering
Conference, IDETC/CIE August 3-6 2008, New York.
11
Presented Hints
Similar Information - Three Clocks

• Distantly Related Information - Three Devices

12
Effects of Open Goals

• Examples are influential in extending both the
  range and novelty of solutions, with devices
  (distant/varied examples) most effective
• Open goals facilitated recognizing analogies in
  distantly related information

13
Transfer to Group Design Tracking
Representations (with Katherine Fu Jonathan
Cagan)

• This study examined
• How engineering design teams converge to a common
  understanding of a design problem and its
  solution using LSA to track representation
  development.
• Effect of good and bad examples on convergence
  and quality
• of produced solutions.
• The problem was to design an automatic peanut
  shelling device for a poor isolated village.

Fu, K., Cagan, J. Kotovsky, K. Design Team
Convergence The Influence of Example Solution
Quality. Proceedings of the ASME 2009
International Engineering Design Technical
Conferences Computers and Information in
Engineering Conference IDETC/CIE 2009 August 30
- September 2, 2009, San Diego, California,
USA  
14
Results Convergence within Groups and
Convergence to Final Design Solution

• Poor example produced decreased convergence
• Good example condition and control not
  significantly different
• All increase over sessions example effect is
  immediate and lasting

15
Results Quality of Final Designs

• Good example produced best quality solutions.
• Poor example produced lower quality solutions.
• Convergence and quality not totally equivalent.

16
Emerging Conclusions From Work on Design Process

• The design environment or context matters
• Hints, both cs and ucs and other people have an
  impact
• Representations can be tracked and do converge
• Timing matters
• Representation tracking potentially allows for a
  fine-grained analysis of multiple issues in
  creative invention
• -- Brainstorming,
• -- Team composition effects,
• -- Expertise effects,
• -- Monitoring impact of environmental input.
• Other work
• Automate the generation of fruitful examples
• Develop a model of the control systems by which
  designers can optimally control their activity
  and resist fixation
• Turn this developing process knowledge into
  teachable-learnable skills.

17
2. Expertise A long time coming

• Hayes others ten year rule for world class
  expertise in many domains (music composition,
  music performance, chess, etc).
• Simon Gilmartin On the order of
  50,000-100,000 memory chunks (for chess
  expertise).
• Ericsson targeted deliberate practice (not just
  amount) leads to expertise.
• Expertise development involves many cognitive
  skills and functions memory strategy (ex.
  chess, problem-solving), perception (chess) and
  representation and knowledge (physics,
  engineering).

Simon H. A. Gilmartin, K. (1973). A simulation
of memory for chess positions. Cognitive
Psychology, 5, 29 -46. Chase, W. G., Ericsson,
K. A. (1982) Sill and working memory. In G. H.
Bower (Ed.), The psychology of learning and
motivation, (Vol. 16). New York Academic Press.
Kotovsky, K., Hayes, J.R. Simon, H. A.
(1985). Why are some problems hard? Evidence from
Tower of Hanoi, Cognitive Psychology, 9,
52-76. Chi, M. T. H., Feltovich, P. J., Glaser,
R. (1981) Categorization and representation of
physics problems by experts and novices,
Cognitive Science, 5, 121-152. Chase, W. G.,
Simon, H. A. (1973). The minds eye in chess.
In W. G. Chase (Ed.), Visual information
processing. New York Academic Press. Moss, J.,
K. Kotovsky, and J. Cagan, Expertise Differences
in the Mental Representation of Mechanical
Devices in Engineering Design, Cognitive
Science, Vol. 30, No. 1, pp. 65-93, 2006.
18
Power Law of Practice

• Skill learning often seems to asymptote
  relatively quickly over training trials.
• In actuality, keeps increasing, with additional
  increments in performance requiring exponentially
  more practice.
• Time Practice b
• log (Time) -b log (Practice)

19
Generating Geometry Proofs
Neves, D. M., Anderson, J. R. (1981) Knowledge
Compilation Mechanisms for the automatication
of cognitive skills. In J. R. Anderson (Ed.)
Cognitive Skills and Their Acquisition.
Hillsdale, NJ Erlbaum.
20
Time to Perform Mental Addition
Crossman, E. R. F. W. (1959). A theory of the
acquisition of speed-skill. Ergonomics, 2,
153-166.
21
3. The Broader Context Educational Implications

• If expertise requires vast time and results in
  continuous cognitive improvement in many diverse
  areas, why not innovative engineering design as
  wellparticularly as we come to better understand
  the specific processes and skills via empirical
  study?
• This is somewhat in contrast to short-term
  focused creativity training courses.
• Focus on in-situ, learning by doing
  concomitant training of skill and knowledge often
  within the milieu of higher education.

22
The Carnegie Commission Boyer Report-Reinventing
Undergraduate Education a Blueprint for
Americas Research Universities

• Educate through research, making research-based
  education the standard.
• Beginning in the freshman year.
• Inquiry-based courses should allow for joint
  projects collaborative efforts.
• Internships can turn inquiry-based learning into
  practical experience.
• Combining a group of students with a combination
  of faculty and graduate students for a semester
  or a year of study of a single complicated
  subject or problem.
• Doesnt focus on the issue of creativity.
• Does have a tilt toward universality (whole
  school approach).

23
The Carnegie Plan

• An older version of some of the main ideas of the
  Boyer Report
• Generated by a forward thinking former president
  of my own university, Robert Doherty, who was
  president from 1936 to 1950.
• The Carnegie Plan provided a well-rounded
  "liberal/professional" education in the context
  of an engineering school.
• Emphasis on being able to do things in addition
  to knowing things
• Students taught to apply fundamental knowledge to
  solve practical problems
• Forerunner to today's focus on an
  interdisciplinary, problem-solving oriented
  university curriculum.
• Enhanced by Herbert Simon with focus on science
  of design and incorporation of meta-cognitive
  skills.

24
Example Engineering Education

• Some Carnegie Plan-like engineering curricular
  examples that in combination can lead toward
  expertise.
• First year engineering courses design, not
  analysis.
• Research internships over multiple years.
• Senior design and Integrated product design
  courses
• Focus on multiple interdisciplinary perspectives
  with faculty from three colleges and diverse
  students.
• Company sponsorship and involvement directed
  toward real-world challenges and resultant
  patents which are often obtained.
• Necessary focus on innovation and creative
  solutions.
• (Similar approach in psychology curriculum but
  issue of creativity)

25
Conclusions

• The basic argument is that creative invention and
  innovation is at least in part a skill set that
  students can acquire expertise in if-
• we understand it at a process level,
• we are clear about the desired behavior and our
  goals of teaching it,
• Students are trained and motivated to do it via
  deliberate training, coaching and sustained
  (motivated) effort over long periods of time
• we all maintain a continued focus on creative
  solutions rather than simply skilled performance.
  
• The policy recommendation is that part of what do
  in this area is to not only foster research on
  creative processes but also the translation of
  the results of that research to methods for
  teaching invention and innovation.

26
The Larger Context
Mediocrity, It takes a lot less time, and most
people won't notice the difference until it's too
late."
Underachievement, The tallest blade of grass is
the first to be cut by the lawnmower.
27
Acknowledgements

• Funding

People

• Lauren Burakowski
• Steven Benders
• Dhruv Datta
• Pasha Gill

• NSF under grants DMI-0627894 and BCS0717957