Michel P

1
How to assess the effectiveness of ICT use in
schools and classrooms ?

• Michel Pérez, Inspecteur général
• de lEducation nationale (France)

2
Summary

• Contributing to the conception and the testing of
  a European Framework for the evaluation of ICT in
  education (EUN projects P2P and P2V)
• Assessment of ICT use in France
• how the French general inspectorate (IGEN)
  contributes to the promotion and the assessment
  of ICT use
• how the French ministry of education tries to
  assess innovation (ICT-related or innovation in
  general)

3
Why inspectors need to assess the use of ICT?

• Digital devices are more and more present in the
  teaching process.
• Local authorities investing in ICT want to know
  what use is made of the equipment and
  installations.
• New tools and systems appear every day on the
  market VLEs (Virtual Learning Environments),
  interactive whiteboards, laptops, tablet
  computers, one-to-one computing, personal
  devices, etc.
• We know the benefits of computers in learning
  autonomy, self-assessment, personalization of
  learning, diversity and remote accessibility of
  contents, collaboration and sharing, continuity
  of monitoring, etc.    
• BUT
• We are not sure that an appropriate use is made
  of the equipments.
• It is currently impossible to accurately assess
  the gain provided by digital tools and measure
  their contribution to the improvement of
  students' skills - (cf. André Tricot, Matt
  Richtel Classrooms of the Future, Stagnant Scores
  - New York Times 2011).
• Efficiency mainly depends on the learning context
  and the quality of the teacher is a key element.

4
Framework for the evaluation of ICT in
educationA EUROPEAN SCHOOLNET program

• PEER-TO-PEER (P2P) PROJECT FOR INSPECTORATES -
  2006
• PEER-TO-VALORIZE (P2V) PROJECT - 2007/2008

5
(No Transcript)
6
History and aims of the project

• The framework was developed as part of the
  P2P-Inspectorates project.
• Its aim was to bring together standards and
  indicators as used by several education
  inspectorates throughout Europe, revise and
  reorder these criteria and present them as one
  shared model.
• The framework could also be used by individual
  schools to self-assess their ICT use and by
  groups of schools to carry out peer evaluations.
• P2P-Inspectorates was a cooperation between
  Ofsted (Office for Standards in Education,
  England), Skolverket (Swedish National Agency for
  Education, Sweden), HMIE (Her Majesty's
  Inspectorate of Education, Scotland), Department
  of Education and Science (Ireland), IGEN
  (Inspection Générale de lÉducation Nationale,
  France) and Inspectie van het Onderwijs (The
  Netherlands).
• P2P and P2V were co-financed by the European
  Commission. The inspectorates work packages of
  P2V and P2P were developed under the auspices of
  SICI (http//www.sici-inspectorates.eu/) and were
  coordinated by The Dutch Inspectorate of
  Education.

6
7
History and aims of the project

• P2V project ("peer learning to valorize" which
  succeeded P2P) has enabled the development of a
  new framework in an evaluation toolkit containing
  instructions, forms, comments and additional
  forms of assessment.
• The P2V framework has been used by inspectorates
  in several European countries to assess the use
  of ICT in primary and secondary schools.
• The "toolbox" (P2V Work package 6 - entitled
  "Inspecting and Evaluating the use of ICT in
  schools in Europe was presented by Bert Jaap Van
  Oel (Dutch inspector and project coordinator).
• Partners in the P2V project were the
  inspectorates of Scotland, Sweden, France,
  Catalonia, Lithuania and the Netherlands.

8
A quick guide to the framework

• The framework is accompanied by a number of
  support documents that help its user in gathering
  relevant information
• a comprehensive user guide that helps the users
  of the framework (inspectors or schools) towards
  an accurate evaluation
• a school questionnaire, which is sent to the
  school before the visit and provides useful
  information about the schools ICT policy,
  hardware, etc.
• a lesson observation form, which is used to make
  notes during the lessons visited
• a guidance sheet for the meeting with teachers
• a guidance sheet for the meeting with students
• a guidance sheet for the meeting with school
  management/ICT leadership

9
The content of the framework

• The framework consists of three main themes
• Conditions, Use and Outcomes.
• - There are a number of quality areas (eight in
  total) within each theme,
• - Quality indicators with corresponding evidence
  pointers are identified within each area.
• Conditions
• C1. Leadership,
• C2. Infrastructure and access,
• C3. Curriculum planning,
• C4. Quality assurance and improvement
• Use
• U1. Pupil use,
• U2. The teaching process,
• U3. Administrative use
• Outcomes
• O1. Impact on learning and standards
• When using the framework, evaluators can indicate
  which evidence they found by checking the
  appropriate boxes and can accordingly assign a
  score to each Quality Indicator.

10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
A short review of the results

• The last review meeting held in Brussels in
  October 2008 helped to show different lines of
  thought.
• It should first be noted that ICT have
  transformed radically and irrevocably the way we
  teach and learn. It is therefore appropriate to
  examine carefully these teaching tools and the
  modes of learning.
• Reflection on the methodology of visits
• The one-day (or more) visits for school
  assessment are conducted in several European
  countries as part of the evaluation of schools
  (which is not the norm in France).
• However, in France the grid could be filled at
  the time of inspection of teachers and might be
  subject to analysis (a self-assessment grid is
  included in the toolbox ).
• At the end of the visit, a report could be made
  to the school management team extended to
  representatives of the school board this report
  could be an opportunity for fruitful dialogue and
  would identifying areas for improvement.

14
A short review of the results

• The analysis of the results of evaluations of the
  use of ICT in schools visited by country shows
  the diversity of situations that depend on
  various factors
• the size and equipment of schools,
• the educational culture of the school and,
  concretely, whether or not the teachers work
  together in a dynamic way,
• the teaching authority in the school, which is
  a factor of coherence,
• the use of ICT by students for free access and
  personal work (no program involving one-to-one
  computing was detected).

15
Updating the framework

• The ICT framework was intended to be applied
  during schools visits however its administration
  is cumbersome and does not allow for instant
  reaction.
• The framework could evolve into an online tool
  allowing
• - a self-diagnostic evaluation of the use of ICT,
  showing the strengths and weaknesses of the
  school
• - a dialogue with external assessment in order to
  define tracks of progress

16
Some assessment activities for the use of ICT in
France
17
IGENs activities for the assessment of
innovative uses of ICT

• Permanent monitoring within each specialty
  group of general inspectors
• Monitoring mission within IGENs ICT committee
  (cellule TIC)
• Counselling and advising DGESCO experts (DGESCO
  General Directorate of School Education)
• Supporting reforms and action plans, such as the
  DUNE action plan (Développement des Usages du
  Numérique à lEcole) or the Ecoles Numériques
  Rurales action plan (Digital Rural Schools)
• Accompanying innovative experiences, the digital
  textbook, for example
• Audits and reports at the request of Minister and
  local authorities

18
An experimental online tool for self-assessment
of schools in France
19
Online school self-assessment

• OAPE Overview
• This site is designed as a tool to help schools
  in the monitoring of their educational activity.
• It includes a set of questions that focus on
  current practices involving several actors.
• It is used by the head teacher, who is free, if
  he wishes to do so, to share the access to OAPE
  with other members of the school community.
• It consists of four types of tabs
• 5 tabs for the diagnosis (corresponding to the
  5 pillars of the schools characteristics and
  specificities)
• - 1 tab to summarize the various results of the
  diagnosis
• - 1 tab for the radar chart, a diagram which
  provides an immediate and global visualisation of
  the diagnosis
• - 1 tab for resources.
• With this tool, the head teacher can identify the
  pillars of the schools specificities that are
  heavily invested and conversely the pillars
  that are less so.

20
(No Transcript)
21
(No Transcript)
22
Experimental audit protocol for the assessment
of a (regional) VLE

• With the participation of IGEN, an audit protocol
  was recently devised in the region of Nantes
  (Académie), with a view to
• verifying how the regional VLE is developing in
  the schools
• assessing the effective use of the VLE in
  schools
• this protocol will be applied in order to
• help the schools in their self-assessment
  approaches
• enter into a process of continuous improvement
• for local authorities
• to make an assessment and provide a tool for
  monitoring progress,
• to manage the evolution of uses by bringing to
  light the points assessed.

23
The stages of the protocol

• Collection of information
• Centralised data collected on the VLE
• Two questionnaires a questionnaire for head
  teachers, a questionnaire for teachers. No
  questionnaire for students.
• Visit that comprises verification, interview(s),
  instant / on-the-spot debriefing
• Written audit report
• Summary of recommendations

24
A BRIEF CONCLUSION

• Effective use of ICT in teaching must obviously
  be checked ...
• to support teachers innovative activity,
• to help them make a fair use of digital devices
  in their teaching activity,
• to promote substantial progress in the learning
  process.
• Inspectors need to have some efficient tools to
  aid in the evaluation of the use of ICT in
  schools.
• We need to provide a shared tool, fruit of a
  collaborative work.
• It should be possible and relevant to develop
  this framework into an online tool for
  self-diagnosis and remote dialogue preparing the
  evaluators visit.