Convergent Analysis: Extracting the Value from the Research Literature

1
Convergent Analysis Extracting the Value from
the Research Literature

• Cathie Norris, Univ of North Texas
• Jennifer Smolka, Univ of North Texas
• Elliot Soloway, Univ of Michigan

2
Todays Story

• Content Process for extracting value from
  research literature for pratictioners
• Quiet reason Try out a new idea on a friendly
  audience
• Plan
• The Need
• The Typical Response
• Our Proposal

3
Need

• Present
• Address attackers with evidence
• Guidance for practitioners
• Future
• 1.5 Billion for MORE research

4
Attackers

• There is no good evidence that most uses of
  computers significantly improve teaching and
  learning.
• Atlantic Monthly p 45, 7/97

5
WRONG!

• Lots of good data!!
• Lots of positive results!!
• Zillions of literature reviews

6
Need

• Present
• Address attackers with evidence
• Guidance for practitioners
• Future
• 1.5 Billion for MORE research

7
What Works from Science Education

• Learning Cycle Approach
• Cooperative Learning (see example)
• Analogies
• Wait Time
• Concept Mapping
• Computer Simulations
• Microcomputer-based Labs
• Systematic Approaches to Problem Solving
• Conceptual Understanding in Problem Solving
• Science-Technology-Society
• Real-Life Situations
• Discrepant Events

8
Literature Review

• Fairly consistently, results favored older, more
  able students, especially those exposed to
  relatively lengthy treatments that were
  well-grounded in appropriate theoretical
  frameworks.
• Reed, 1996
• NOT helpful to practitioner!

9
Future PCAST Report

• Presidents Council.
• Impeccable credentials
• Heart in the right place
• Sharper than all get out
• Implementing NOW
• Raise 1.5 Billion for Research!

10
How Learn From Research?

• If we dont learn from the research, then why do
  it?
• So,whats wrong with this picture?
• We have good data
• We have lists of what works
• Why havent we learned?

11
Havent Addressed Needs of Teachers

• How would I replicate that study?
• How would I adapt that set of studies?
• Reporting of research doesnt address those
  questions

12
A Telling Analogy
13
A Telling Analogy
14
Stage One Towards Why?

• Old, bad question
• When does technology work?
• Better question
• Under what conditions does technology work?
• (But, as we shall see, not done yet)

15
Profile Rationale

• To answer why. Must attend to the details!
• What ARE the conditions?
• We developed the PROFILE
• 4 iterations, at least
• 75 factors!!

16
Stage Two Beyond Conditions

• Most information not provided by researcher
• Still takes hours to do a Profile
• Profile more than a lit review, less than paper
  itself
• But still not answering practitioners questions

17
Practitioners Questions

• Teacher
• Should I teach keyboarding first in my writing
  class?
• What role should I expect to play when I
  introduce word processors in my writing class?
• Curriculum Coordinator
• Should we hire lab monitors for our computer labs?

18
The Convergent Analysis Method

• Style of argumentation method for extracting
  value from research articles
• Compare, relate, juxtapose research studies
• Focus on common themes

19
Area to Apply

• Writing
• Why? Because, just because...

20
Convergent Analysis Understanding Findings

• Writing, using WP
• Kurth found no improvement
• Owston, Dauite found improvement
• WHY?

21
CA Answering Practitioner Question

• Writing, using WP
• Should my kids first do keyboarding?
• Compensate for no keyboarding by waiting to
  assess impact

Short assignment Extensive previous
Snyder, 94
, Owston, 92
Student Experience TIME
Jones, 88
22
CA Answering Practitioner Question
Shift in Teacher Activities
Snyder, 94
, Riddle, 95
Teacher Experience ROLE
Finkelman, 94

• Writing, using WP
• What role should I expect to play when I
  introduce word processors in my writing class?

23
CA Answering Practitioner Question

• Online Inquiry Curriculum
• Should we hire lab monitors for the scale up in
  Detroit?

24
CA Observations

• CA goes beyond horse race results
• CA takes richness and depth of literature
  seriously
• CA takes hours to do
• CA does answer practitioner questions

25
Challenges Next Steps

• Need
• Visualization strategies How can practitioners
  do CA themselves?
• Reporting strategies Require researchers to
  report the details

26
Website

• http//home.wtd.net/smolka/ca.htm

27
Concluding Remarks

• The only thing worse than learning from
  experience is not learning from experience.
• If we arent going to make use of the research,
  then why do it?
• Use the past as guide to the future.
• Its REALLY hard!
• The Todd Oppenheimers just read the abstracts,
  at best.
• Ultimately researchers practitioners must
  learn to play nice together