Workshop on

1

ICT4D ME Framework in the UN MDG Context Based
on the InfoDev/World Bank Handbook

• Workshop on
• Assessing the Effects of ICT in Education
• Ispra, Italy April 28, 2009
• DRAFT VERSION

Prof. Dan Wagner International Literacy
Institute University of Pennsylvania
2
Main points

• Introduction to UN MDG perspective
• MDGs as a frame of reference for OECD?
• The InfoDev/World Bank Handbook
• Pro-equity approaches to ME
• Conclusions

3
1. Millennium Development Goals for 2015 What
they are

4
2. Millennium Development Goals Relevant for
OECD?

• By global perspective, OECD is implicated
• But also due to key features, such as
• Need for evidence-based decision making
• Concern about cultural appropriateness and
  diversity
• Access and content
• Learning and achievement
• Cost
• Concerns about equity

5
2. More on ICT and equity

• Digital divide is decreasing between countries,
  but
• Digital divide is increasing within countries.
• Who are the poorest within countries?
• Illiterate low-literate persons
• Ethno-linguistic minorities
• Migrants/Refugees
• People with disabilities and special educational
  needs
• How many people? More than half the population of
  poor LDCs
• How many in OECD countries?

6
2. Difficulties of Monitoring and Evaluation in
ICT4D contexts

• ICT is seen a bridging major divides (e.g.
  digital divide), but mostly doesnt
• ICT still reaches mainly the most-reachable,
  where there is an installed base of
  infrastructure (electricity grid, phones)
• Among the poor, there are very diverse end-user
  skills and literacy levels
• Teachers and learners are relatively isolated and
  dispersed
• Lack of culturally-relevant (local language)
  resources
• e-Learning is often about e, and less about
  learning
• Origin of the InfoDev Handbook How can ME
  provide an empirical base for effective ICT4D?

7
3. Monitoring and Evaluation of ICT in Education
A Handbook for Developing Countries

• Principal authors Dan Wagner (ed), Bob Day, Tina
  James, Robert B. Kozma, Jonathan Miller Tim
  Unwin
• Advisors Andrea Anfossi Boubakar Barry
  Mohammed Bougroum Enrique Hinostroza Shafika
  Isaacs Daniel Kakinda Meng Hongwei Edys
  Quellmalz Pornpun Waitayangkoon
• Preface Michael Trucano, InfoDev/WorldBank

8
3. Handbook for ME of ICT4D The Model

• Similar to this workshop macro, meso and micro
• Outcomes may be seen at either/both student and
  teacher levels
• Outcomes can include ICT-related
  skills/attitudes, learning achievement, job
  outcomes, etc.

9
3. Handbook for ME of ICT4E Chapter example

• Monitoring and Evaluation of ICT for Education
  Impact A Review. Bob Kozma
• Some examples of evaluation results
• Research evidence shows that simply putting
  computers into schools is not enough to impact
  student learning.
• Combinations of input factors can best influence
  impact, including national policies and programs
  related to changes in curriculum, pedagogy,
  assessment, and teacher training.
• Policymakers in developing countries need to
  address the barriers to ICT use, such as lack of
  skilled support staff and adequate
  infrastructure.

10
3. Handbook for ME of ICT4E Chapter example

• Core Indicators for Monitoring and Evaluation
  Studies in ICTs for Education. Bob Kozma Dan
  Wagner
• In order to understand the outputs, inputs must
  also be measured, such as ICT resources, teacher
  training, pedagogical practices, etc.
• Common types of data collection
• Questionnaires
• Face-to-face interviews
• Workshops / Focus groups
• Content analysis of materials
• Infrastructure audits
• Checklists

• Software analysis
• Self-assessment
• Direct assessment
• Classroom observations

11
4. Pro-equity approaches to ME

• Pro-Equity Approaches to Monitoring and
  Evaluation Gender, Marginalized Groups and
  Special Needs Populations, Dan Wagner
• Who is excluded from surveys, and from core
  indicators?
• Access-only approaches versus content approaches
  Same or different?
• Requires demographic breakdown of the
  intervention sample by gender, language,
  ethnicity, age, location, and income relative to
  the rest of the national population
• Evaluation and assessment results must allow bias
  towards marginalized groups in order to obtain
  reliable results.

12
4. Status of inequality in literacy and ICT
Literacyis similar across the globe
Youth and Adult Illiteracy Rates (15 years and
older, 2000)

• Large inequalities by gender, ethnicity and
  language
• In poor countries, more the half the population
  is illiterate or low-literate
• 100-200 million children are out-of-school youth

TechnologicalIlliteracy
Traditional Illiteracy
OECD countries
Africa
Arabregion
Latin America
East Asia Oceania
South Asia

13
5. Conclusions Part 1

• Pro-equity Direct implications for the Workshop
• Domains. Information gathering will vary at all 3
  levels policy, institutional individual
• Variables. Need to develop additional indicators
  (e.g. language, disability, in addition to
  gender and region
• Methods. May have to use culturally and
  psychologically appropriate tools, such as
  ethnography, sociolinguistics, cognitive
  psychology
• Evaluation/Impact. Impacts may be different for
  different populations.
• Sources. May need to look beyond OECD countries

14
5. Conclusions Part 2

• Questioning differently
• Does a truly global approach have implications
  for a regional/SES country approach?
• Is a pro-equity pro-poor approach worth special
  thinking about ICTs and use in cultural context?
• Do we need to think more about general skill
  levels (e.g. literacy) as playing a role in ICT
  impacts?

15
To obtain an electronic copy of the Handbook

• Online at www.INFODEV.org
• Online at www.LITERACY.org