PROGRAM EVALUATION

1
PROGRAM EVALUATION
AÇEV
HAMBURG-FEBRUARY 2007
2
AÇEP Mother-Child Education Program
AÇEVs scientifically-proven, home-based,
low-cost preschool and parent education program
implemented since 1993

• Beneficiaries
• 200,000 mothers and children in 78 provinces of
  Turkey
  
• 900 teachers trained
• Yearly target 45,000 mothers and children
• Aim of Program
• To empower mothers by supporting them in
  their parenting roles and providing the tools
  necessary for fostering the cognitive development
  of their children before they enter school.
• Target Group
• Socio-economically disadvantaged mothers of
  6 year olds who do not go to preschool.
  

• Main Components and Duration
• Mother Support Program25 weeks-long
• Implemented in the form of weekly 3 hour
  group discussion meetings, facilitated by an ACEV
  trained group leader
  
• Cognitive Training Program (for the child)25
  weeks-long
  
• Implemented at home daily by mother and
  child using worksheets, storybooks and other
  materials
  
• Home visits3-4 times during the 25 weeks
• Conducted by group leader to ensure quality
  and provide one-on-one support to mother at home

3
OVÇEP Preschool Parent-Child Education Program
AÇEVs scientifically-proven, center-based
preschool education program which brings the
preschool and the parent together, implemented
since 1999

• Beneficiaries
• 18,000 mothers and children in 26 provinces of
  Turkey
  
• 102 teachers trained
• Yearly target 5,000 mothers and children
• Aim of Program
• To support the preschoolers literacy and
  numeracy skills and to ensure the educational
  support provided to the child is continuous and
  complementary, through strengthening the
  school-family collaboration.
• Target Group
• Children attending state-run preschools and
  their parents.
  

• Main Components and Duration
• Cognitive Training Program (for the classroom)28
  weeks-long
  
• Implemented in class daily by teacher and
  preschoolers using worksheets, storybooks and
  other materials
  
• Cognitive Training Program (for the home)25
  weeks-long
  
• Implemented in the home daily by the parent
  and the child using a second set of worksheets,
  storybooks and other materials
  
• Parent Support Program9 sessions
• Implemented in the form of monthly 3 hour
  group discussion meetings, facilitated by the
  ACEV trained preschool teacher

4
Why do we evaluate our programs?

• to evaluate how far the program has achieved its
  goals.
  
• to determine the short and long term impacts of
  the program.
  
• to have regular recording and monitoring system
  in order to gather information from the field
  which we implement the program.

5
Ongoing Evaluation
6
Ongoing Evaluation

• We evaluate all the participant mothers per
  year.
  
• We use pluralistic perspective evaluation (Group
  leader, Mother, Child)
  
• Qualitative ve quantitative research methods used
  together
  
• All componants of the program is evaluated (Group
  meetings, Cognitive training program
  implementation, Home visits)

7
Impact Evaluation

• Effects on children
• Pre-literacy and pre-numeracy skills
• Environmental differences
• Gender differences
• School success
• Teachers perception of the child
• Effects on mother child interaction
• Effects on mothers

8
Impact Evaluation
9
Impact Evaluation

• Quasi experimental design is used.
• The duration for preparation phase of the
  research took nearly 1 year.
  
• Short and long term impacts have been assessed.
• Sample was constituted of 177 children (85
  experiment group, 92 control group)
  
• Control group was matched with the experimental
  group on certain characteristics.
  
• Research has been carried out in four different
  regions, in four different provinces.
  
• Pluralistic perspective evaluation has been
  carried out (mother, teacher, child)
  
• Trainer seminars are also regularly evaluated per
  year.

10
DESIGN OF THE EVALUATION RESEARCH

• AT THE END
• OF PRIMARY
• SCHOOL
• LITERACY and
• NUMERACY
• SKILLS
• INSTRUMENT
• SELF CONCEPT
• MOTHER
• INTERVIEW
• TEACHER
• INTERVIEW

• AT THE BEGINING
• OF MOCEP
• PRE-LITERACY and
• PRE-NUMERACY
• SKILLS
• INSTRUMENT
• SELF CONCEPT
• MOTHER
• INTERVIEW

• AT THE END
• OF MOCEP
• PRE-LITERACY and
• PRE-NUMERACY
• SKILLS
• INSTRUMENT
• SELF CONCEPT
• MOTHER
• INTERVIEW

MOTHER- TRAINED
NON-TRAINED
11
LIST OF SKILLS IN BOTH INSTRUMENTS

• Pre Literacy Skills
• Visual Recognition
• Visual Discrimination
• Sequential Knowledge
• Ability to follow verbal direction
• Pencil Control
• Capacity to copy letter
• Visual attention
• Recognition of letter
• Visual retrieval
• Syntax memory
• Spatial concepts
• Knowledge of prepositions
• Visual memory
• Discrimination by category
• Listening comprehension

• Pre Numeracy Skills
• Recognition of Shapes
• Counting
• Visual Matching
• Awareness of Correspondence
• Auditory attention
• Visual Counting
• Sequential Counting
• Knowledge of numbers
• Visual Recognition and Discrimination of numbers
• Addition
• Subtraction

12
EFFECTS OF MOTHER CHILD EDUCATION PROGRAM

• At The End of The Program

13
CHANGE IN PRE LITERACY AND PRE NUMERACY SKILLS
14
CHANGE IN PRE-LITERACY AND PRE-NUMERACY SCORES
ACCORDING TO ENVIRONMENTAL STIMULATION
15
SEX DIFFERENCES IN THE MOTHER-TRAINED GROUP
60
50
40
GIRLS
30
CHANGE SCORES
BOYS
20
10
0
PRE-LITERACY
PRE-NUMERACY
16
SEX DIFFERENCES FOR THE NON-TRAINED GROUP
20
18
16
14
GIRLS
12
BOYS
10
CHANGE SCORES
8
6
4
2
0
PRE-LITERACY
PRE-NUMERACY
17
EFFECTS OF MOTHER CHILD EDUCATION PROGRAM

• At The End of Primary School

18
LITERACY AND NUMERACY SKILLS
19
PASSING GRADES
20
DIFFERENCES ACCORDING TO ENVIRONMENTAL
STIMULATION INDEX
100
90
STIMULATING ENVIRONMENT
80
70
NON-STIMULATING ENVIRONMENT
60
50
MEAN SCORES
40
30
20
10
0
LITERACY
NUMERACY
LITERACY
NUMERACY
MOTHER-TRAINED
NON-TRAINED
21
DIFFERENCES ACCORDING TO STUDY ENVIRONMENT AT HOME
100
90
80
SUITABLE ENVIRONMENT
70
60
NON SUITABLE ENVIRONMENT
50
MEAN SCORES
40
30
20
10
0
LITERACY
NUMERACY
LITERACY
NUMERACY
MOTHER-TRAINED
NON-TRAINED
22
DATE CHILD BEGAN TO READ
25
20
15
NUMBER OF CHILDREN
10
5
0
NOV
DEC
JAN
FEB
MAR
APR
TRAINED
NON-TRAINED
23
SCHOOL READINESS OF CHILD ACCORDING TO MOTHER
24
MOTHER SHOWED AN EXTRA EFFORTFOR HER CHILDS
SUCCESS AT SCHOOL
25
TEACHERS EVALUATION on SCHOOL READINESS
3.9
3.8
3.7
3.6
MEANS
3.5
3.4
3.3
3.2
COGNITIVE
SOCIAL
MOTHER-TRAINED
NON-TRAINED
26
TEACHERS EVALUATION OF CHILDREN
4.5
4
3.5
3
2.5
MEANS
2
1.5
1
0.5
0
APPROPRIATE
ATTENTIVE
CREATIVE
CURIOUS
BEHAVIORS
MOTHER-TRAINED
NON-TRAINED
27
MOTHERS INTEREST IN SCHOOLING
4.6
4.5
4.4
MOTHER-TRAINED
4.3
NON-TRAINED
4.2
MEANS
4.1
4
3.9
3.8
3.7
3.6
ACCORDING TO MOTHER
ACCORDING TO TEACHER
ATTENDANCE AT SCHOOL MEETINGS