Evaluation of strategic funding to develop graduate employability: year 1 findings

1
Evaluation of strategic funding to develop
graduate employability year 1 findings

• Presentation to SHEEN Meeting
• Patricia Ambrose Sheila Sim
• 20 November 2008

2
Presentation

• Key findings from HEI evaluation reports
• Engaging at strategic level
• Engaging academic staff
• Engaging learners and student bodies
• Engaging employers and other external
  stakeholders
• Evidence of success
• Features of good practice
• HEIs priorities for year two
• Value-added of SFC funding
• Sustainability
• Support from SHEEN and other agencies
• Recommendations

3
Key findings engaging at strategic level (1)

• Priorities for HEIs in year 1
• Restatement of original aims objectives
• Appointment/secondment of dedicated staff
• Developing enabling structures/reviewing policies
  strategies
• Scoping, mapping, auditing of existing activities
  materials
• Communication/awareness raising
• Piloting new approaches and materials
• Some specific areas of focus include
• PDP
• Work placements/WRL
• Building links between employability staff and
  academics
• Volunteering
• Graduate attributes

4
Emerging findings strategic issues (2)

• Links to HEIs overall strategies for
  employability
• 6 HEIs mention specific inclusion in strategic
  plan
• 9 HEIs incorporate employability within their
  learning teaching strategy or equivalent
• 7 HEIs mention a specific strategy for
  employability ( 1 in development)
• 4 HEIs currently reviewing in context of their
  strategic plan or other strategy (LT or
  employability)
• Evidence of wider embedding within
  decision-making and operational structures
• Annual programme monitoring review
• Embedding into module and programme descriptors
• Employability sub-cttes reporting to LT
  committees

5
Key findings engagement with academic staff (1)

• Significant focus of effort for most HEIs
• Variability across and within HEIs re. level of
  staff awareness and engagement (reflecting
  diversity of sector, mission and subject foci)
• Importance of relationship building
• People at different stages in the employability
  journey
• Winning hearts and minds
• Scoping discussions/staff interviews
• More regular communication
• Better access to resources
• Barriers
• Variable levels of awareness and confusion/lack
  of understanding of terminology
• Initiative fatigue/good at this already
• Employability/TL of lower priority than research
• More challenging for non-vocational subjects
• Lack of staff awareness of discrepancies between
  their own and student perceptions

6
Key findings engagement with academic staff (2)

• Staff development
• Mix of mandatory and voluntary sessions
• Links to induction, PG Cert for HE, plus ad hoc
  events, conferences, away days, expert seminars
• Toolkits for implementing new approaches (e.g.
  PDP)
• On-line message boards/mailing lists
• Communication and awareness raising
• Regular bulletins/e-zines
• Meetings/ongoing dialogues with academic
  depts/faculties/ presentations to relevant
  committees
• Structured meetings with key staff
• Provision of bespoke advice/consultancy and
  support
• Cross-pollination of expertise (e.g. careers
  advisors joining departmental industry liaison
  boards)
• VLE site/development of specific web pages
• Fairs/wider events
• Filming sessions for streaming via web
• One HEI mentioned working with corporate
  marketing team to publicise good practice
  examples

7
Key findings engagement with academic staff (3)

• Active involvement of staff
• Use of academic champions (3 HEIs, plus 2 others
  considering)
• Re-design of programmes, including embedding of
  employability into modules
• Development of new resources/materials
• Using feedback from students to identify gaps in
  current provision
• Identifying new and innovative approaches
• Maximising the benefits of staffs existing links
  with employers and ensuring these feed back into
  teaching and research

8
Key findings engagement with learners/student
bodies (1)

• Variability across HEI responses some have more
  well developed student engagement than others
  (but all planning to address this issue)
• Some evidence of active involvement in policy and
  materials development (e.g. students help to
  choose e-tool for PDP, evaluating pilots, SU
  developing toolkit to assist with student
  engagement)
• Co-curricula (e.g. recognising and accrediting
  student to student mentoring, credit-bearing
  module on internship etc.)
• Improving access (entitlement) to work
  placements, exchanges, other opportunities such
  as volunteering/community engagement
• Using survey or other feedback to evaluate
  existing provision and inform new developments
• Student members on key committees
• Identifying target groups (international
  students)

9
Key findings engagement with learners/student
bodies (2)

• Communication and awareness-raising
• Electronic feedback on new services
• Student careers survey
• Use of focus groups and some one-to-one feedback
  meetings
• Using existing student networks to generate ideas
  on how to market PDP and other employability
  initiatives
• Use of VLE/website
• Targeted events (Future Focus week, stands in SU)
• On-line competitions
• Student ambassadors to promote careers services
• Dissemination via student representatives
• Use of alumni (surveys, mentoring, case studies,
  guest speakers, provision of work placements)
• Developing more user-friendly terminology (e.g.
  Future Focus not PDP)
• Links between employability co-ordinator/team and
  SU

10
Key findings engagement with employers other
external stakeholders (1)

• Less focus overall on this strand of activity
  (with some exceptions)
• Some HEIs identifying this strand as a priority
  for year 2
• Building the information base
• Consolidating and widening range of contacts
• Meetings with key organisations, community groups
  etc
• Generating more and better placement opportunities

11
Key findings engagement with employers other
external stakeholders (2)

• Some specific activities at individual HEIs
• New fora (Recruiters Club, Employer Link one stop
  shop, Employer Forum)
• Speed networking event
• Dragons Den activity
• Involving employers in approach to PDP
• Helping with validation of new credit-bearing
  modules
• Research on work placement to inform framework
  for future developments
• More focus on not-for-profit and sustainable
  development sectors in response to student demand
• Developing Customer Relationship Management
  database to record and co-ordinate contacts
• Plan to involve employers in staff training
• Using existing academic staff links

12
Evidence of success/effectiveness

• Too early for significant feedback several HEIs
  are starting to think about KPIs and more
  systematic approaches to gather evidence and
  review achievements against aims and objectives
• Mix of measures
• some more quantitative/outputs-based
• No. of placements offered and jobs secured
• Improvements in graduate employment rates
• Levels of attendance at events
• Usage of new materials
• No. of subsequent follow-up contacts
• others more qualitative/outcomes-focused
• Enabling, process and outcomes indicators
• Short-, medium- and longer-term objectives
• Ways of tracking perceptual and behavioural
  change, including
• Setting benchmarks
• Pre- and post-activity questionnaires
• Feedback from academic champions and employers
• Case studies
• Team established to review activities across the
  institution

13
Features of good practice

• Serious engagement across strategic and
  operational levels of the HEI
• Thorough scoping/auditing of existing
  activities/resources gaps and priorities
  identified
• Careful piloting (drawing on robust evidence
  base)
• Structured opportunities for staff and students
  to contribute
• Recognition of importance of (tailored) staff
  development
• Commitment to offering students better access
  (entitlement) to range of opportunities and
  services
• Bespoke provision for key target groups
• Improving institutional information base on
  employer, alumni and community links
• Range of communication and awareness-raising
  materials/activities
• Some evidence of critical reflection and thinking
  seriously about KPIs and success measures

14
HEIs priorities for year two

• No significant changes to plans anticipated
• More of the same across the range of activities
  identified
• Greater emphasis on
• Own evaluation/success criteria
• Evaluating and rolling out pilot initiatives
• Employer engagement
• Continuing to raise awareness and enhance
  resource base
• Building in external perspectives
• Sustaining and enhancing progress to date

15
Value-added of SFC funding

• Appointment/secondment of dedicated staff with
  time to devote specifically to this area
  catalyst for enhancing existing activities and
  enabling new ones
• More concentrated development of initiatives and
  swifter implementation
• Enabling the trialling of different approaches
  and more bespoke or targeted solutions
• More strategic and embedded approach to
  employability across the institution
• Development of new tools
• More consistent and high quality offer to
  students
• Pump-priming to build a more sustainable
  framework for the future
• Complementing careers management with a more
  pedagogic approach
• Enabling greater critical perspective on practice
  and commissioning of specific RD to address gaps
• Greater articulation/synergy between
  employability and other enhancement themes
• Facilitating collaborative working

16
Sustainability

• Embedding approaches via
• Overall strategic plan
• Operational infrastructure
• Curricula developments
• Materials/resources which are easy to maintain
  and update
• Funding is enabling development and
  implementation of approaches which are then
  designed to roll forward as part of ongoing
  roles/responsibilities for staff across the
  institution

17
Support from SHEEN and other agencies (1)

• Positive feedback on SHEEN
• Access to good practice
• Employability co-ordinators group is invaluable
• Networking and informal support at practitioner
  level
• Stimulating ideas on best models for delivery
• Well-regarded events
• Continuing to raise profile of employability at
  the sector level
• Other agencies
• Value support and advice from HEA
• HEA subject centres are good source of case
  studies/good practice
• Quality of resources and advice provided by
  HEA/QAA etc. help to convince academic staff of
  the validity of this area of work

18
Support from SHEEN and other agencies (2)

• Improvements and issues
• How could HEA and other agencies bring together
  work on employability and other enhancement
  themes (research and teaching linkages, graduate
  attributes) as many ideas overlap?
• Should the network move over time from
  inter-institutional support to greater
  cross-institutional working, commissioning of
  research and more practice-oriented
  collaboration?
• External bodies could provide better information
  on links between different policies which impact
  on employability and also clarify remits of
  various organisations in this area?
• Agencies want a more structured, formal network
  while practitioners value a more flexible,
  informal approach?

19
Recommendations

• for the SFC and other partner organisations (the
  HE Academy, QAA Scotland, etc)
• bring together work on employability and the
  other enhancement themes more effectively
• consider and consult on the future role and
  longer-term sustainability of SHEEN
• provide further support to HEIs on the
  development of KPIs and success criteria (with
  expert input as appropriate)
• for HEIs
• focus further on how best to monitor and evaluate
  the success of their own activities
• engage in more critical self-reflection during
  year two
• how can funding be best used to support all
  teaching and learning provision within
  institutions
• particular needs of international students
• for the evaluation team
• revise the evaluation framework in the light of
  feedback, with a view to making the exercise as
  useful and streamlined as possible
• assist the SFC in undertaking its mid-term review
  of the initiative in Spring 2009

20
Questions/issues for discussion

• Improvements to the evaluation framework
• scope of activities to be covered 
• purpose of Part 3
• changes to key evaluation questions
• Further support for HEIs in developing KPIs and
  success criteria (possible seminar?)
• Other questions/issues and comments

21
Contact
Patricia Ambrose Sheila Sim Associate
Director Senior Consultant SQW Consulting SQW
Consulting t. 020 7307 7147 0131 243 0730 e.
pambrose_at_sqw.co.uk ssim_at_sqw.co.uk w.
www.sqw.co.uk