Identifying the Impacts of research

1

• Identifying the Impacts of research related
  activities
• A Case Study

FPTT Annual meeting June 2008
2
Review - 2007

• Bibliography Identifying Benefits of Technology
  Transfer Beyond Commercialization (FPTT June
  2007)
• Literature Review/Research Design Developing a
  College-Based, Evidence Informed RD Impact
  Assessment Framework (Niagara College/Centennial
  College December 2007)

3
RD Impact Assessment Study March 2008

• Proof of Concept to test conclusions of
  literature review
• Case Study Research Design
• Logic Model Evaluation Framework
• 11-dimension impact typology (Godin Dore 2004)
• Qualitative, simple approach minimum of
  assumptions

4
Methodology Case Study

• Case study research method (Yin 1994/2003)
• Recognized social science research method
• Design allows for valid generalizations based on
  a hypothesis/propositions
• Not business school or clinical case study
• Salter Martin 2001

5
Methodology Logic Model

• Organizing Framework
• Inputs
• Activities
• Outputs
• Impacts

6
Methodology Impact Typology
Godin Doré (2004) 11-dimension typology of
impacts of research

• Science
• Technology
• Economy
• Culture
• Society
• Policy

• Organization
• Health
• Environment
• Symbolic
• Training

7
Data Collection

• Telephone interviews all stakeholders
• Review of internal/external documents
• Bibliometric analysis
• Unit of Analysis
• A discrete research project over a 10-year period
• Including research, development,
  knowledge/technology transfer activities
  Research Related Activities

8
Results - Case study method

• Effective approach
• Identified intended/unintended results
• Labour intensive 40-50k/200-250 hours
• Apply satisficing
• Limit interviews/use surveys
• Identify all stakeholders and activities to build
  case history

PhillipsKPA 2006
9
Results Logic Model

• Useful for separating inputs, activities, outputs
  and impacts
• Nature of RRA (causality, attribution,
  interactions)
• All stakeholders influence impacts
• Strict logic model probably not suitable too
  linear - serves as model for an organizing
  framework that better reflects complexity of
  interaction
• Separate performer from users (outcomes/impacts)

10
Results Impact Typology

• Compelling matrix for standardizing impacts
  across all types of RRA
• Requires adaptation/refinement
• Confirmed presence/absence of impact
  dimension/sub dimensions not significance,
  frequency, magnitude
• Impacts based on perceptions of stakeholders

11
Conclusions

• Simple method provides qualitative evaluation of
  impacts of RRA that reflect broader expectations
• Further refine approach with additional case
  studies
• Impacts not attributable to single stakeholder
  (e.g. performers) but to entire network Build a
  portfolio of case studies for conclusions about
  impacts
• Need to consider complete RRA (network) all
  stakeholders