PARTICIPANT EVALUATION OF PRESUD

1

• PARTICIPANT EVALUATION OF PRESUD
• TREVOR WREN
• UNIVERSITY OF NEWCASTLE
• UK

2
Aims, Sources Methods

• Participant Evaluation SWOT, Change,
  Improvements
• Across All Participants, Cities, Process
  Components
• Surveys, Interviews, Participant Observations
• 40 (x2) Team members surveys 8 team manager
  interviews
• 17 coordinator full interviews informal
  contacts
• 140 Stakeholders (30 x full, 40 x short, 70
  e-mail/web surveys)
• Months of Participant Observation (In 5 Team
  Reviews, In 2 Reviews Coordination, in Training,
  Management Meetings, Project within Lead City)
• Triangulation and Interpretation of Multiple
  Perspectives

3
Findings Strengths

• Positive Overall (but qualified) Evaluation
• Majority Support for a Core Peer Review Process
• Significant Qualitative Change Impact
• on Administration People Directly Involved
• PRESUD embodies Good Evaluation Practice
• Practitioner Action Research Social Learning
  Organisational Learning Potential Maximises
  Dissemination Utilisation
• See Conference Presentations for others
• Reports of Successes and Direct Participants
  Views

4
Weaknesses Issues

• UK Methodology/audit themes
• Report Delays, Utility, Validation Knock-on
  effects
• Preparation, Language, Context/Culture Limited
  Engagement
• Disputed Review Evidence, Data Gathering and
  Analysis
• Competencies, data gathering recording,
  analysis, theory
• socio-eco analysis
• Stakeholder Engagement can/should be improved
• Preparation, types, numbers, depth, validation,
  who controls?
• Credibility, Exclusion, Data, Wider Ownership
• Little Measurable External Change Attributable or
  Guaranteed All impact is context dependent.

5
Opportunities

• Support for Continuation Extension (if Revised)
• Across stakeholders in current cities and to
  others
• Interest of Stakeholders exceeds their
  Involvement
• People wanted to be more involved more people
  wanted involvement
• Good Practice in Data Gathering, Analysis,
  Validation
• Parallel Web Review Possible and Tested
• Additional Engagement Data, More Robust
• Stakeholder Credibility and Inclusion
• Participants desire more Learning
• Potential not Actual huge additional engagement
  potential

6
Conclusion

• PRESUD has demonstrated what is practically
  possible and what is not
• Participants want it developed - but only if
  revised, more realistic, better inclusion
  engagement, more Participative-Independent
  Robust, and with Systematic Learning as a Major
  Aim

7
Some Recommendations

• Second Stage of Extension, Development and
  Revision
• Reduce Revise Methodology
• Less UK Flexible Participative Broaden Peer
  Review, fix gaps
• Adopt Embed Learning as Goal
• Develop/Evaluate Mechanisms to Satisfy
  Stakeholders
• Incorporate an Independent Parallel Web-Based
  Review
• Improve preparation independence, engagement
  data validation robustness, dissemination
• Adopt Additional Good Practice
• Evaluation Research qualitative analysis
  theory-based
• Include Independent Participant Evaluation in 2nd
  Stage
• See Others/Details in The Evaluation Report

8
The Future?

• Revised Approach
• Solid Foundation and Strengths to Develop
• Some Current/Intrinsic Weaknesses - but Some
  Solutions
• Participative approach will both engage and
  validate
• Internet Review engage and data
• strengthen qualitative internal - successes
• Build in explicit learning aims and mechanisms to
  ensure this
• Project Development and Impact
• European Sustainability Impacts?
• Needs Growth, Roll Out, Continuous Critical
  Improvement
• Comments Contact Trevor.wren_at_ncl.ac.uk
• Acknowledgements