June 11th and 12th Galway

1
June 11th and 12th Galway Prof. Kathy
Isaacs Prof. Liz McDowell Prof Phil Race Prof
Ferdinand von Prondzynski Dr. Janet Parker Dr.
Kelly Coate
Designforlearning.eventbrite.com
2
Towards Evidence-based policy in Higher Education
Large scale staff surveys of teaching,
professional interests and the role of higher
education
Dr. Iain Mac Labhrainn, CELTFebruary 26th 2009
3
Preamble

• Cultural loading of language 
• Evidence Based - what kind of 'evidence
• League tables?
• simply 'measurable' or intrinsically valuable?
• interpretation
• qualitative plus quantitative 
• contextual 
• intrinsic value system 
• Aim?
• plan that is realistic, operational and
  effective 
• beneficial to all constituents 

4
context

• Request to write and contribute to a range of
  strategies, mission statements and policy
  documents
• Wide reading
• Particular style
• Incredible similarity
• Frequent jargon, buzz-phrases
• Lack of clarity / plain English
• Can I avoid writing like this easy though it is?

5
Prof. Harry Frankfurt Professor Emeritus of
Philosophy at Princeton University
6
Frankfurt Test

• Test to apply to all policy documents and
  strategic planning
•  Subtle distinction between BS and lying - BS is
  much more dangerous to the truth!
•  
• Applies equally to managerialism as to
  'McDonaldisation' arguments 

7
Warning signs ?

• semantic density
• frequency of jargon terms with short half-life
  (or which have appeared in dictionary in last 10
  years)
• alignment with management consultancy fashions
• pseudo-scientific terminology
• extent and type of evidence base
• statistically valid
• peer-reviewed
• self-consistent
• Authorship/professional expertise

8

• Bullshit is unavoidable whenever circumstances
  require someone to talk without knowing what he
  is talking about. Thus the production of bullshit
  is stimulated whenever a persons obligations or
  opportunities to speak about some topic exceed
  his knowledge of the facts that are relevant to
  that topic. This discrepancy is common in public
  life, where people are frequently impelled
  whether by their own propensities or by the
  demands of others to speak extensively about
  matters of which they are to some degree
  ignorant. p63

9
?
10
Academic Staff Survey gt Strategic planning
"It'll never work. No-one will answer. It's too
big. No-one has the time."
11
(No Transcript)
12
From Anecdote to Pie-Chart
Dr. Iain Mac Labhrainn, CELTNovember 13th , 2008
13
Built on earlier, validated surveys

• The HERI Faculty Survey a triennial survey of
  academic staff across the US.
• Higher Education Research Institute, University
  of California, Los Angeles
• The Approaches to Teaching Inventory (ATI) of
  Trigwell Prosser.
• Trigwell, K and Prosser, M. (1996b) Changing
  approaches to teaching a relational perspective
  Studies in Higher Education, Vol. 21, No. 3, 275
• The Impact Of Training Of University Teachers on
  their Teaching Skills, their Approach to Teaching
  and the Approach to Learning of their Students,
  Gibbs, G. Coffey, M., Active Learning in Higher
  Education, Vol. 5, No. 1, 87-100 (2004)

14

• The Perceptions of the Teaching Environment
  Inventory
• Prosser, M. and Trigwell, K. (1997a) Perceptions
  of the teaching environment and its relationship
  to approaches to teaching, British Journal of
  Educational Psychology, Vol. 67, pp. 25-35
• Patersons survey on the role of higher education
  in society, which explores issues of diversity,
  access and civic responsibility.
• Paterson, L. (2003) The survival of the
  Democratic Intellect academic values in Scotland
  and England, Higher Education Quarterly, Vol. 57,
  pp. 67-93

15
Survey instrument

• Developed with Elaine Keane
• Paper-based, scanned
• Six sections
• Background/demographics
• Teaching Approaches Context (including
  perceptions of student engagement)
• Teaching Assessment Methods
• Evaluation
• Professional Activities
• The Purposes of Higher Education
• Also asks about use/satisfication with
  services/resources, attitudes and working
  environment/conditions

16
(No Transcript)
17
(No Transcript)
18
Deployment Strategy?
19
Id like your honest, unbiased and possibly
career-ending opinion on something
20
Deployment strategy

• Completion time 40-50 minutes
• Traditional postal not feasible
• Maximise return
• Maximise reflection/concentration/focus
• Contextualise
• Departmental meetings/scheduled sessions
• Independence Trust key issues

21
Results
22
(No Transcript)
23
Basics

• 277 / 450 respondents
• Spread across all Faculties
• Range of levels of seniority
• Division by gender representative
• Self-consistent results in individual sections
• Identifiable trends and factors

24
Approaches to Teaching
25
Approaches to Teaching
Intention
Strategy
Information Transfer (IT)
Teacher Focus (TF)
Conceptual Change (CC)
Student Focus (SF)
ITTF CCSF ITSF CCTF
26
Potential Factors

• Academic Discipline
• Class size
• Gender
• Age/Experience/Status
• Professional Interest/Priority

27
(No Transcript)
28
CCSF
29
CCTF
CCSF
30
ITTF
CCTF
31
Summary

• CCSF is the single largest cluster overall
• Most prevalent in Arts
• ITTF most prevalent in Science Engineering
• CCTF strongly apparent also
• i.e. with intention to focus on concepts but in
  practice use teacher-centred approaches
• Class size cited as most significant issue for
  this group

32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Level of course U/G vs P/G
38
Level of course U/G vs P/G
39
Level of course U/G vs P/G
CCSF
40
(No Transcript)
41
Summary

• Time pressure identified as the strongest
  negative factor in attitude towards teaching
• Those with TF rate student aspects as more of an
  issue (attendance, engagement, preparedness,
  skills, etc)
• CCSF use wider range of teaching methods in both
  small and large classes
• ITSF prominent amongst applied language teachers
• Those with more research orientation, more likely
  to be TF (and IT)
• Overall CC is the most common intention

42
Strategic Priorities?
43
(No Transcript)
44
Values Purpose of HE

• To foster a capacity for independent critical
  thinking among students
• To conduct basic, theoretical and applied
  research
• To define new directions in research and
  scholarship
• To facilitate lifelong learning
• To pursue knowledge as a value in itself
• Civic engagement
• Direct economic benefit/role ?

45
Learning, Teaching Assessment
StrategyUniversity Strategic PlanResearch
StrategyCELT Operational PlansCollege
StrategiesMA in Academic Practice

•  

46
Applications?

• LTA Strategy
• CELT Operational Plan
• Review of Training
• University College Strategic Plans
• Review of AV/classroom technologies
• Review of Learning Technology
• CKI Strategy

47
Training Materials

•  

48
LTA Strategy 2009-2011

•  

49
CELT operations

• Training programmes focus
• Challenge of real Academic Practice
• Issues of promotion and reward
• Flexibility
• Linkages with disciplinary contexts/needs
• Using MA projects to focus on specific issues
• Technical support/technology adoption
• Student feedback
• Curricular reform
• Research/Teaching synergies
• Civic Engagement

50
Some Conclusions
51
Reflection (Survey)

• Considerable effort required in developing,
  deploying and processing surveys of this type
• Provides invaluable information for strategy and
  operational planning
• Reveals extent of willingness to improve and
  innovate
• Baseline input for institutional strategic plans
• Values staff opinions and needs in prioritisation

52
(No Transcript)
53

• Systematic, large-scale staff survey
• Interviews and focus-groups
• Student feedback
• Feedback from range of constituencies 
• public perception
• employers
• government
• alumni
• Partners (international)
• Institutional Research Data Comparative
  Statistics
• Analysis of individual professional artefacts
• Reflective journals
• Teaching Portfolios
• Research proposals
• MA/PhD research
• Government policy frameworks
• International literature review

54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
Reflections (Strategy/Policy)

• Do not argue yourself out of getting good data!
• Inputs from range of perspectives
• Dissensus is valuable 
• Prevent academic critique from sinking into
  cynicism 
• Contextual awareness
• Honesty and transparency
• Self-awareness professional limits
• Consultants lead to formulaic outcomes
• Deliberation
• Evaluation and critical review
• Recognition of risk 
• Value of creative approaches

59
June 11th and 12th Galway Prof. Kathy
Isaacs Prof. Liz McDowell Prof Phil Race Prof
Ferdinand von Prondzynski Dr. Janet Parker Dr.
Kelly Coate
Designforlearning.eventbrite.com
60
(No Transcript)
61
Contact

• iain.maclaren_at_nuigalway.ie