Introduction to Research Supervision

1
Introduction to Research Supervision

• Margaret KileyCEDAM

2
Program

• Supervision
• Supervisory relationships
• Supervisory contexts
• Supervisor roles and responsibilities

3
ENVIRONMENT/CONTEXT Candidates and supervisors
interact and learning within a research learning
environment
discipline
CANDIDATE CHARACTERISTICS e.g. gender, age,
enrolment, previous academic /research
experience, motivation, intellectual capacity,
research topics, conceptions of research
OUTCOMES Timely progression and completion
Career prospects Publications Quality
thesis Research and generic skills
Research understandings Changed world view
Changed perception of self as researcher
and learner Becoming a researcher
Contribution to field of research Social
Value
institutional protocols
LEARNING AND RESEARCH EXPERIENCES e.g.
modification/development of expectations,
conceptions, approaches to research supervision
research culture
ASSESSMENT AND EXAMINATION
government policies
SUPERVISOR CHARCTERISTICS e.g. previous doctoral
experiences, gender, age, previous experience
supervising/ examining, conceptions of research
global developments
university policies
4
Supervision rather than Supervisor

• Research suggests that there are a range of roles
  that need to be filled in terms of research
  supervision, in addition to knowledge of the
  topic and methodology, these are
• Mentor
• Coach
• Facilitator of candidature
• Sponsor
• From Pearson, M Kayrooz, C. (2004). Enabling
  critical reflection on research supervisory
  practice. International Journal for Academic
  Development, 9(1), 99-116

5
Mentor

• The mentoring role requires specific subject
  expertise and includes mentoring students so
  they can complete the research project itself,
  but also mentoring the intellectual development
  of the student, i.e.
• Encourages publishing
• Encourages networking
• Helps with seminar and conference presentations
• Assists with career goals

6
Coach

• The coach role involves helping candidates
  develop their research expertise while they are
  actually doing their research project. The
  coaching role often is performed by a range of
  people. This role includes
• Helping students with identifying the research
  question and theoretical framework
• Helping plan and refine the project
• Advising on critical aspects of research
• Being directive when needed

7
Progressing/facilitating Candidature

• The progressing the candidature role can be
  thought as facilitation-related functions, this
  includes
• Monitoring progress
• Periodically reviewing supervision arrangements
• Negotiating availability and initiating contact
• Devoting sufficient time to the student

8
Sponsor

• The sponsor is one who, for example, will
• Ensure candidates have access to basic resources
• Ensure, or advise on how, students can access
  funding for conferences, field work etc
• Keep students current with policies procedures
• Identify administrative procedures that students
  need to meet
• Provide access to expertise and full
  participation in the research practice
  including alternative sources of expertise.

9
Can/should one person do all this?

• While there are a few outstanding people who can
  fulfill all of those roles, generally we find
  that one person is strong in one area and not so
  strong in another.
• In your group, discuss
• Which of the roles reflect your strengths?
• Which roles would you need someone else to
  fulfill?
• What can/should you do about it?
• How can you encourage your candidates to seek out
  people who can fulfill the four roles?

10
Tools

• Expectation Scale
• Learning needs analysis
• To assist students identify strengths that they
  bring to their candidature and where they need to
  develop new skills and knowledge
• Supervisory alignment
• To assist students and supervisors gain a sense
  of where the other is situated
• Memorandum of Understanding
• Helpful for even just discussing the aspects of
  supervision without necessarily turning into an
  MoU

11
Expectations

• Expectation scale
• Read and decide where you fit on the scale for
  each of the points
• Discuss with the group

12
Communication Techniques

• In groups, discuss techniques that you have used,
  your supervisor used, or you know from others
  that have helped communication (st/su, st/panel,
  st/st, st/other) e.g.
• Agendas for, and Notes from, individual and panel
  meetings
• Memorandum of Understanding
• Email/web-based discussion
• Group meetings (with different candidates getting
  practice at chairing, noting etc)
• Regular meeting times/Open-door policy/Meetings
  as needed
• Meetings with others in the Centre/disciplines

13
The hard facts

• Code of Practice
• HRD Enrolments, progression and completion (RTS)
• Evaluative data re the HDR experience
  supervision

14
Research Training Scheme

• The Government funds a certain number of RTS
  places to universities annually. Some places are
  high cost e.g. some sciences, others low cost
  e.g. humanities
• Funding is determined by students enrolled i.e.
  reaching target, student completions, and other
  research income e.g. from staff research grants
  and publications
• Funding is calculated on a four-year candidature
  (FTE) i.e. if a student takes longer than four
  years then, the University is not being paid for
  supervising that student.

15
Graduate Destinations (2005) within University
Type
16
Postgraduate Research Experience Questionnaire
(PREQ)

• Developed by Graduate Careers Council of
  Australia in 1998
• Distributed by each University after graduation
  with results collated nationally by GCCA and then
  sent to each university
• 28 Statements clustered into six scales
• Supervision (1, 7, 13, 21, 24)
• Skill development (6, 10, 14, 20, 26)
• Intellectual climate (5, 9, 16, 22, 23)
• Infrastructure (3, 8, 12, 18, 27)
• Thesis Examination (2, 15, 25)
• Goals and Expectations (4, 11, 19)
• Overall Satisfaction (28)

17
PREQ 2004 (National)

• Supervision Scale (Agree 45)
• 2002 70.8
• 2003 72.7
• 2004 72.7
• Skill Development (Agree 45)
• 2002 89
• 2003 89.6
• 2004 91.1
• Intellectual Climate (Agree 45)
• 2002 54.7
• 2003 56.3
• 2004 57.7

• Infrastructure Scale (Agree 45)
• 2002 65.8
• 2003 67.2
• 2004 68.5
• Thesis Examination Scale (Agree 45)
• 2002 75.0
• 2003 75.1
• 2004 76.7
• Clarity of Expectations (Agree 45)
• 2002 87.5
• 2003 89.3
• 2004 90.1

Overall Satisfaction 2002 80.8, 2003 82.3,
2004 83.8
18
What does this instrument say about the
postgraduate research experience?

• In groups look through the statements that
  students are asked to consider in the PREQ.
• What sort of picture are they implying of the
  research experience?
• How does that relate to your own experience?
• What might it mean for you as a supervisor?

19
Intellectual Climate

• From the PREQ results we can see that one of the
  scales which is consistently low is Intellectual
  Climate or research culture
• In pairs work through the sheet in your folder
  titled Developing a vibrant research culture
  among your postgraduate students
• How might you contribute to, and encourage your
  students to also contribute to, a positive
  research culture in your discipline/school?

20
Evaluation Strategies

• Supervision is very difficult to evaluate given
  the small number of students involved and lack of
  anonymity
• On the other hand we need some sort of feedback
  on performance for both formative (improvement)
  and summative (going for promotion) purposes
• Consider the options that are available at
  Newcastle
• Look at the Alternative Evaluation Strategies
  sheet and the case study and discuss in groups
  whether any of those ideas would work for you.

21
Stages of Candidature

• Recruitment and selection
• Framing the candidature e.g. needs analysis,
  expectations, topic selection/refinement,
  establishing the panel, literature review and
  methodology
• Guiding and monitoring progress e.g. ensure
  formal requirements met, writing, development of
  networks, feedback on progress
• Completing e.g. when to stop, feedback,
  nomination of examiners, support during and after
  examination
• The post graduation phase e.g. preparing a
  research and publication plan

22
Case Study

• In your folder you have Case Study 1(a) and 1 (b)
• In your group work through one of the case
  studies and discuss the questions at the end

23
Case Study (cont)

• Now work through the other case study in the
  folder
• Can you see how the misunderstandings might have
  developed?
• How might they have been avoided?

24
Supervisory Panels

• Discuss how you might work with a candidate on
  the task of constructing a panel/committee taking
  into account the various roles (Mentor, Coach,
  Facilitator, Reflective Practitioner and Sponsor)
  and the different stages of candidature and the
  related responsibilities. (Supervisory Framework
  might be useful as a guide)

25
The Proposal Seminar

• Many universities in Australia only allow
  students an interim enrolment until they have
  successfully completed their proposal seminar
• There is evidence to suggest a correlation be a
  high quality research proposal and successful PhD
  submission and completion
• Who at Newcastle is responsible for organising
  the the seminar?
• What guidance is the candidate given?

26
Feedback on work

• One of the most common complaints from candidates
  is the lack of timely and useful feedback from
  supervisors
• These complaints come through in national surveys
  e.g. the Postgraduate Research Experience
  Questionnaire (PREQ)
• What makes for useful feedback?
• What is timely?

27
In small groups

• How do supervisors give feedback to research
  candidates? E.g. did your supervisor give you
  verbal, hand written on written work that has
  been presented, via email etc
• How often do you think a supervisor should
  provide feedback and how can they make the time
  to do it?
• What advice can supervisors give to candidates
  about submitting work for feedback e.g. suggest
  that candidates give them drafts to read as they
  are about to head off on a long flight?
• What do supervisors expect candidates to do with
  the feedback? e.g. does the supervisor expect
  them to act on it or is it for advice only?

28
Monitoring Progress

• Monitoring progress has been shown to be
  critical in candidature.
• Most Australian universities have a system (e.g.
  Annual Progress Reports) where the supervisory
  panel/committee discusses with the candidate
  their progress over the past 12 months and plans
  for the next 12 months
• Reflecting on, and discussing progress, have been
  shown to have a positive affect on progress. It
  can also be the time when changes are made to the
  panel.
• What happens at Newcastle?

29
Examination

• The Supervisory Panel is responsible for
  proposing names of potential examiners to the
  Head of School (or equivalent)
• Encouraged to discuss potential names with the
  candidate, but generally candidates do not to
  know final names
• Generally 50 of all Australian dissertations are
  sent overseas
• The aim is to find examiners who are
  knowledgeable in the area and who will give a
  fair and balanced opinion
• The mean time for examination of
• accept as is theses is 0.35 of a year
• minor revision 0.39 of a year (although 17 took
  more than six months)
• major revision 0.53 of a year and
• revise and resubmit 1.39 years. (Courtesy Sid
  Bourke)

30
Experienced Examiners report that they

• Expect the student to pass as they open the
  thesis
• Are very reluctant to fail a student with most
  experiencing considerable distress if they do so
• Come to a decision about the quality of a PhD by
  about the end of Chapter 2
• Have a formative rather than summative view of
  thesis examination
• Believe that there is a risk attached to sending
  theses to inexperienced examiners
• Are reluctant to take much notice of
  institutional criteria when examining
• See Nobel Prize paper for more detail

31
Experienced Examiners appear to

• Be fiercely independent in their views
• Hold varying views about the purpose of the PhD.
  (Is it the thesis or the student being
  examined?)
• Consider professional duty as the main reason for
  examining, followed by the fact that they are
  going to be needing examiners for their own
  students!
• Devote considerable time to examining each thesis
  
• Have surprisingly broad approaches to
  methodology/ paradigm
• Demonstrate few discipline differences in their
  responses, other than regarding publications

32
Inexperienced Examiners

• Have a high level of confidence in their ability
  to examine (which is not always reflected in what
  they say in response to other questions)
• Frequently talked about experience from
  supervising examining Honours students and
  theses
• Adopt a similar approach to the actual process of
  examining as do their more experienced
  colleagues, although they are more likely to
  focus on the steps or components of a PhD
  rather than the whole
• See their role as maintaining standards and
  performing their summative assessment role
  correctly

33
Inexperienced Examiners

• At a surprisingly high rate, wanted to fail first
  thesis or said it was awful
• Are more prone, than experienced examiners, to
  follow institutional criteria.
• Felt (some of them) that they were being examined
  too
• Suggest that one of their main difficulties is
  their inability to benchmark
• Seem to have very high expectations of the
  supervisors of the thesis being examined

34
Is there a difference?

• From work of Trafford (2003) from 130 vivas it
  was possible to determine that
• Experienced examiners tended to ask questions
  that can be defined as Defending doctorateness,
  contributing to knowledge, critique of research,
  synthesizing concept
• Inexperienced examiners tended to ask more
  technical questions
• Trafford, V. (2003) Questions in doctoral vivas
  Views from the inside, Quality Assurance in
  Education 11(2) pp 114-122

35
Traffords Categorisation
Innovation and DevelopmentHIGH
C. Questions generally related to issues such as
research question, choice of topics, location of
study
D. Defending doctorateness, contributing to
knowledge, critique of research, synthesizing
concept
Scholarship Interpretation
A. Types of questions include resolving research
problems, content, structure
B. Implications, awareness of, and familiarity
with wider literature
LOW
36
Strategies for examining

• Different examiners approach the task
  differently, but most
• Begin by reading the Abstract, Introduction
  Conclusion to gauge the scope of the work and
  whether what candidates say they are going to do
  is actually done
• Look at the references to see what sources have
  been used and whether they need to follow up on
  any of them
• Then read from cover to cover taking detailed
  notes, finally go back over the thesis to check
  whether their questions have been answered or
  whether their criticisms are justified

37
The reports demonstrate

• A less than ideal thesis has
• Too much detail with lack of analysis
• Lack of confidence, energy engagement by the
  candidate
• Lack of argument and rigour
• Shoddy presentation (typos etc)
• Lack of critique of own analysis/ sweeping
  generalisations based on opinion rather than
  analysis
• Inadequate or poorly expressed methodology scope

• A good thesis has
• Critical analysis argument
• Confidence a rigorous, self-critical approach
• A contribution to knowledge
• Originality, creativity a degree of risk taking
• Comprehensiveness scholarly approach
• Sound presentation structure
• Sound methodology

38
Resources

• Papers from all seven biennial Quality in
  Postgraduate Research conferences
  http//qpr.edu.au
• Australian University consortium For Improving
  Research Supervision Training (fIRST)
  http//www.first.edu.au (you will need a Username
  and Password from University Contact)
• Australian Deans and Directors of Graduate
  Studies http//www.ddogs.edu.au/cgi-bin/index.pl
• SORTI web site at the University of Newcastle has
  information on examining theses particularly in
  the performing/ visual arts http//www.newcastle.e
  du.au/centre/sorti/publications.html