Peer observation of teaching in the online environment

1
Peer observation of teaching in the online
environment
An Action Research Approach Dr Deborah
Swinglehurst Ms Jill Russell Prof Trisha
Greenhalgh
Wednesday 21st June 2006 CDE Fellows Conference
2
Introduction

• Background to the project
• Aims and objectives of the project
• Progress of the action research so far
• Highlights of the project
• Explanation of workshop activities

3
Background

• What is peer observation of teaching (POT)?
• Peer observation is well established in Higher
  Education, but under- explored in online teaching
  settings
• No published literature identified
• Discussion with online teachers from the UoL
  External Programme identified enthusiasm for
  project

4
3 models of peer observation

• Evaluation model
• Developmental model
• Collaborative model
• Gosling 2005

5
Evaluation model

• Judgements made on quality of teaching
• Management led related to quality assurance
• Preparation for external audit
• Linked to probation, promotion, investigating
  under-performance, assessment
• Observation carried out by senior staff, selected
  evaluators
• Observation characterised by a power hierarchy

6
Development model

• Judgements made about areas for improvement
• Aim is to develop and improve teaching competence
  
• Observation carried out by respected expert
  teachers or educational developers
• Observation characterised by an expert / learner
  relationship
• Expert often facilitator of reflection on
  practice

7
Collaborative model

• Non judgemental
• Aim is to improve teaching through dialogue and
  reflection, sharing issues for investigation
• Teachers observe each other
• Observation characterised by equality / mutuality
• Success depends on a culture in which teaching is
  valued and scholarly discourse encouraged

8
Aims of the peer observation project

• To facilitate and systematically study the
  implementation of peer observation of online
  teaching within a collaborative AR framework
• To implement real practical change as well as
  adopting a research focus
• To collaborate with online teachers beyond our
  own course team

9
Objectives

• Establish an online Special Interest Group of
  teachers to facilitate dialogue and information
  exchange
• Map the current role of peer observation within
  online courses represented by these teachers
• Identify the potential and challenges of the
  online environment with respect to peer
  observation
• Implement a programme of peer observation of
  teaching within our MSc course
• Disseminate lessons learned throughout the
  University of London and beyond

10
Potential benefits of the project

• Staff development
• Develop teaching skills / promote reflection
• Course development
• Link peer observation to ongoing course
  development
• Students
• Focus on pedagogical quality and course
  improvement
• Wider educational community
• Collaborative links within External Programme and
  beyond
• Dissemination of findings, presentation and
  publication

11
Questions to explore

• What are the critical success factors for peer
  observation in online teaching?
• What benefits can individual teachers and online
  courses expect to gain from a rigorous programme
  of peer observation?
• What are the challenges associated with
  implementing peer observation in online courses
  and how might these be overcome?

12
The action research approach

• An approach to improving education by changing
  practice and learning from this
• Collaborative and participatory people working
  to improve their own practice, through
  self-critical communities
• Research is with and for participants, not on
  participants
• Emergent approach, developed through a self
  reflective spiral

13
The AR spiral
Planning
Acting
Reflecting
Observing
14
The CRASP model

• Critical collaborative enquiry by
• Reflective practitioners being
• Accountable and making the results of their
  enquiry public
• Self-evaluating their practice and engaged in
• Participative problem-solving and continuing
  professional development
• Zuber-Skerritt 1992

15
The story so far

• Networking to identify potential participants for
  online focus group, via CDE, IoE
• DS attended 2 day multidisciplinary workshop at
  UEA on AR
• E-mail invitation to join online focus group sent
  to all teachers of postgraduate courses within
  UoL External Programme and to other potentially
  interested teachers
• 29 teachers agreed to participate in online focus
  groups
• 2 Yahoo! groups set up

16
Focus groups (1)

• 2 parallel groups (14 participants 17
  participants)
• Participants of many disciplines, 12 departments
• 3 weeks asynchronous online discussion
• Sharing of previous experience of peer
  observation
• Identified and explored many tensions within the
  concept of peer observation
• Discussed perceptions of good online teaching
• Identified challenges to implementing POT online
• Sharing of resources

17
Focus groups (2)

• 130 messages posted by 26 participants
• Very lively scholarly discussion
• Transcripts of discussions prepared and
  circulated to all participants
• Preliminary thematic analysis using NVIVO
  software

18
Tensions identified in POT

• accountability development
• sincere insincere (jumping hoops)
• isolated collaborative experience
• voluntary involuntary / coercive
• external internal
• formative summative
• focus on teaching focus on learning
• formal arrangements informal arrangements

19
Making sense together

• A strong theme emerging from the groups was the
  potential for POT to help teachers reflect
  collaboratively and make sense together
• We are all novices - there is not an
  established right way
• A shared understanding of context seems important
  
• One of the unique features of online teaching is
  the permanency of the record, which can be
  exploited
• Observation does not necessarily need to be real
  time

20
And then

• DS and JR attended a SIG of IoE staff interested
  in teaching with technologies
• Concept of PROP meetings developed and
  introduced to staff on MSc course
• Established an ongoing SIG
• Continued engagement with the UEA AR group
• DS signed up to Hull university course Sharing
  European Perspectives on e-learning

21
The Special Interest Group

• Yahoo! discussion group online
• 15 members, 10 from original focus groups
• Ongoing discussion about progress with
  implementation of peer observation (49 messages)
• Sharing of resources
• Continuing to evolve and new members coming on
  board

22
What is PROP?

• Peer-to-peer Reflection On (pedagogical) Practice
• Monthly meetings of MSc course tutors
• One tutor takes responsibility for a meeting
• Teaching episode used as a focus for reflection
• Tutor presents area for discussion/feedback

23
The PROP process

• Based on an action learning set (McGill, Beaty
  1995)
• Each group member has time for exclusive
  consideration of their issue by the whole group
• Focus is on supporting reflection, challenging
  assumptions
• Learning viewed as a social process
• Deliberate attention to relationship between
  reflection and action

24
Topics explored in PROP so far

• 3 meetings so far
• How do we help our students to write critically?
• How do we assess our students?
• Exploring principles of good practice in giving
  feedback

25
Further project plans

• 3 further PROP meetings planned
• Continued discussions within the online SIG
• Workshop at SAPC conference July 2006
• Plans to publish the cross-disciplinary
  collaborative work of the UEA Action Research
  group
• Preparation of project report
• Publication of this project for wider academic
  community

26
Highlights of theproject

• The success of the collaborative effort
• The UEA AR group 8 professionals of different
  disciplines
• Engaging 34 teachers in online scholarly
  discussion either within the focus groups or the
  special interest group
• Involving tutors from the External Programme and
  beyond
• Enthusiastic involvement of 6 tutors in PROP
• Linking PROP to real changes in practice to
  enhance quality of student learning experience
• Linking PROP to peer review meetings so we can
  close the loop through reflection on pedagogy
• Learning opportunities for DS and tutor team

27
The workshop activity

• 3 groups, each with flipchart / pens
• Each group working on a slightly different
  activity related to the implementation of peer
  observation
• Spend 5 minutes familiarising yourselves
  individually with the scenario and the task
• Spend 45 minutes working together in small groups
  on the task
• Spend remaining time sharing main conclusions