Improving impacts of classrooms: Professional Development and Classroom Observation

1
Improving impacts of classroomsProfessional
Development and Classroom Observation
Robert C. Pianta, Ph.D. University of
Virginia CASTL
2
Questions and challenges for policy, research,
and training in education

• What are offered to students in classrooms?
• Do interactions with teachers and experiences in
  classrooms matter for students?
• Can observation leverage improvements quality,
  and effectiveness of teachers/teaching?
• Measure, validate, improve teacher quality at
  scale through standardized observation

3
Results of large-scale observational studies

• National-level studies
• National Center for Early Development and
  Learning (NCEDL)
• NICHD Study of Early Child Care
• Up to 1,000 settings observed at preschool,
  K, 1, 3, 5 ? more than 4,000 classrooms
• Largest set of systematic standardized classroom
  observation in U.S. schools
• All teachers credentialed/certified

4
Observations and large-scale applications

• Trade-offs and decisions
• Multiple versus single occasions
• Length of the window
• Time of day / content of instruction
• Unit of analysis global or micro
• Classroom-level or child-level
• Training demands and reliability
• Applicable across diverse settings
• No system can address every concern
• Ultimate criterion is link to child outcomes

5
Describing opportunities to learn Counting
behaviors, activities, practices

• Vast majority of interaction/activity is whole
  group or individual seatwork
• Few, if any, social or instructional interactions
  between teacher and individual child
• Mostly literacy activities
• Exceptional variation within and across grades
• Consistent patterns from pre-k to 5th grade

6
How do students spend time?

• High levels (30) of business/routine activity
• pk-5 managing materials, routines
• High levels of basic skills focus
• 71 in pk-1 141 in 3-5
• Ratio of listening, sitting, watching Doing
• 10 pk-1 1,3,5

(NICHD ECCRN 2002, 2004)
7
Rating interactions What is the CLASS?

• Derived from developmentally-informed analysis of
  settings and putative impacts on broad-based
  outcomes.
• Focus on dimensions of teacher-student
  interaction in 3 domains - a theoretical claim
  about latent structure of classroom settings
• Emotional Support
• Organization / Management
• Instructional Support

8
DOMAINS
DIMENSIONS
INDICATORS
9
Classroom ratings CLASS PK-5

• Positive climate
• Negative climate
• Teacher sensitivity
• Regard for student perspectives
• Effective behavior management
• Learning formats/engagement
• Productivity
• Concept development
• Evaluative feedback
• Language modeling

Emotional Support
Organization/ Management
Instructional Support
10
What is the quality of the classroom setting?

• Positive emotional climate
• Productivity
• Quality of feedback

11
Profiles of classroom quality First grade
7
6
5
4
Quality
Emotional
3
Instructional
2
1
0
Positive emotional climate Low academic demand
Very positive emotional climate High
instructional quality
Mediocre, Low academic demand
Negative emotional climate, Low
instructional quality
31
23
29
17
12
Associations with structural/selection factors

• Exceptional variability within and across grades,
  including across the same grade/curriculum.
  Stability across grades is low 15
• Little to no association of observed interactions
  with
• Teacher experience or training
• Teacher salary
• Small associations with structure and selection
  (.10 - .20)
• Class size larger classes more structured
  smaller classes more social and higher
  instructional quality
• Family income/education related to more positive
  ratings
• Students needing access to stable high-quality
  instruction do not typically receive it 10
  rate for low achievers

13
Classroom interactions and childrens social and
academic performance

• Designs that isolate effects for instructional
  and emotional inputs controlling for other
  influences in growth models
• Family and demographic factors
• Childs prior performance
• Structural features of schooling
• Primarily small main effects (/- .10)
• Instructional and emotional quality predict more
  positive achievement and social outcomes
• Larger effects on more proximal outcomes (e.g.,
  child engagement)
• More instruction in literacy and math also
  predict to those outcomes
• Stronger effects for different groups of children
• Low maternal education
• Adjustment problems in K
• Poor

14
Pre-k quality and growth in child outcomes

• Where should we focus attention in policy,
  program development, and teacher preparation?
• Predicting achievement growth during preK from
• Structural features (teacher ed., curriculum,
  etc.)
• Observed interactions (ECERS, CLASS)
• No association of structure with outcome, singly
  or in combination (e.g., NIEER index)
• Instructional and Emotional Supports (CLASS)
  predict positive changes in literacy, language,
  and math skills small effect sizes persist
  into kindergarten

15
Gains in Gr. 1 achievement in instructionally
supportive classrooms
107
106
105
Standardized tests of achievement adjusted
104
103
High educ.
102
101
Low educ.
100
99
98
Low
Moderate
High
1st Grade Instructional Support
16
Gains in Gr. 1 achievement in emotionally
supportive classrooms
107
106
Kindergarten adjustment problems
105
104
Standardized tests of achievement adjusted
103
No problems
102
101
Multiple problems
100
99
98
Low
Moderate
High
1st Grade Emotional Support
17
Implications of our work P-5

• Redefine teacher quality in terms of
  performance/interactions in classroom
• Strive toward moving instructional dimensions and
  implementation up the scale of quality
  interactions
• Develop training and support approaches that
  address teachers interactions with children
• Approach these goals systematically and
  scientifically with standardized, validated
  observations at core
• A science of teaching and teacher-training that
  relies on direct and validated observation

18
Measurement issues/directions

• Develop extension of CLASS for grades 6-12
• Ecometrics Partitioning variance related to
  rater, time, day, season, window, consider
  alternative units of analysis.
• Global features more stable, valid. Reliability
  improved through adding raters in the system.
  Implications for scalability and for research on
  intervention effects, etc.
• Informant-report version empirical tests of
  content-knowledge hypotheses

19
Standardized observation Support for
high-quality interactions
Professional development/ training
Observational Assessment
Social and academic outcomes for children
Resource allocation
CLASS Instructional Organization Social
Improved teacher outcomes
Evaluation
Curriculum
20
(No Transcript)
21
Focus on interactions in classrooms

• Teacher-child interactions and relationships are
  the means by which the curricula are implemented
• MTP uses CLASS as the basis for defining
  high-quality implementation and as the
  target/focus of professional development
• Goals of MTP are to
• Increase teachers observation skills in
  identifying interactive behaviors and cues
  related to CLASS
• Identify childrens differential responses to
  teacher behaviors
• Increase teachers skills in identifying
  alternative responses to children's cues
  create opportunity

22
MTP support for teachers
Two forms of web-based support for quality
implementation
MTP Activities
Teacher-child interactions and relationships
CLASS
Student growth in language, literacy and social
relationships
23
Project design

• 240 Participating PreK teachers statewide - VPI
• 3 Conditions of participation / levels of support
• Consultancy plus website plus curriculum
• Website access plus curriculum
• Materials/curriculum only
• All conditions
• Receive iBook laptops
• Receive MTP activities
• Complete surveys and assess 4 randomly selected
  children

24
Quality teaching videos PK-3
25
CLASS examples PK-3
26
MTP consultation cycle
27
Prompts
Intended to identify positive aspect of teachers
instruction in relation to a class dimension

• This clip is a nice demonstration of concept
  development. You ask the children why the girl
  in the book is happy. You receive one answer and
  then go on and get ideas from two other children.
  What other strategies do you use to promote
  concept development?

28
Prompts
Intended to help a teacher identify CLASS
dimensions and examples in her teaching
interactions

• Here you quickly and effectively redirect the
  children back to the story. As you watch this
  clip, tell me what you are doing to help the
  children remember the rules.

29
Prompts
Feedback on how the teacher implements lessons

• Some of the other children are
• not engaged in this lesson.
• What kinds of learning formats
• might you have used to draw
• all of the children in?

30
MTP-S Prompts Feedback for teachers
31
Effects of MTP support on interactions

• Examine both effects of condition on outcome (web
  vs consultation), treatment on treated, and
  moderation with regard to classroom demands.
• Teachers receiving consultation show greater
  increases in quality of instructional
  interactions early career teachers who view
  CLASS videos show gains in interactions with
  children effects seem attributable to video
  review
• Consultation moderates poverty effect

32
Changes in sensitivity for teachers in the MTP
consultation and web only study conditions
33
Changes in language stimulation for teachers in
the MTP consultation and web conditions
34
Moderating effects of study condition on the
association between classroom poverty and changes
in teacher sensitivity
35
Moderating effects of study condition on the
association between visiting video pages and
changes in teacher sensitivity
36
Effects of MTP support on child outcomes

• Examine effects of condition and treatment on
  treated with Consultancy, Web, and Activities
  groups.
• When teachers participate in consultation,
  children show greater gains in tests of early
  literacy

37
(No Transcript)
38
(No Transcript)
39
Associations between Teachers Exposure to the
MyTeachingPartner Consultancy, Language and
Literacy Activities and Web-Site, and Childrens
Development of Language and Literacy
Skills during Pre-K
Definitional Vocabulary Definitional Vocabulary Phonological Awareness Phonological Awareness Print Knowledge Print Knowledge PALS Total PALS Total
Level-1 B SE B SE B SE B SE
Child Characteristics
Pre-test 0.45 0.03 0.53 0.03 0.48 0.02 0.49 0.02
Boy -0.06 0.17 -0.92 0.27 -1.19 0.39 -1.33 0.64
Non-English -0.66 0.34 -0.26 0.52 0.46 0.70 0.51 1.18
Maternal Education (years) 0.11 0.05 0.30 0.08 0.13 0.11 0.12 0.18
Poverty -0.24 0.24 -0.07 0.40 -0.21 0.47 -0.58 0.83
Second Year of Intervention 0.55 0.17 0.62 0.28 0.50 0.35 1.62 0.59
p.05. p.01. p.001.
40
Associations between Teachers Exposure to the
MyTeachingPartner Consultancy, Language and
Literacy Activities and Web-Site, and Childrens
Development of Language and Literacy
Skills during Pre-K (continued)
Definitional Vocabulary Definitional Vocabulary Phonological Awareness Phonological Awareness Print Knowledge Print Knowledge PALS Total PALS Total
Level-2 B SE B SE B SE B SE
Classroom Characteristics
Mean Pre-test Score 0.17 0.06 0.07 0.08 0.12 0.08 -0.11 0.06
Proportion in Poverty 0.04 0.60 0.44 1.06 0.42 1.72 0.53 2.80
Proportion Non-English -0.35 0.63 -0.90 1.01 -3.00 1.67 -10.2 2.70
Mean Mothers Education -0.05 0.16 0.30 0.28 -0.04 0.47 -0.27 0.79
Teacher Characteristics
Advanced Degree 0.07 0.22 -0.44 0.36 -0.32 0.62 0.03 1.05
Major in ECE 0.33 0.21 0.59 0.35 0.59 0.60 2.69 0.99
Years Teaching PK -0.00 0.01 0.02 0.02 0.01 0.04 0.06 0.06
Intervention Components
GT 20 hours/0 hours 1.03 0.33 -0.12 0.54 0.88 0.95 2.45 1.50
GT 20 hours/LT 20 hours 0.76 0.31 -0.22 0.57 1.40 0.90 2.23 1.54
Hours per week MTP-LL Activities 0.22 0.14 0.67 0.24 0.44 0.38 1.41 0.64
Low Web Use 0.42 0.32 -0.33 0.53 1.09 0.94 1.98 1.52
Medium Web Use 0.08 0.29 -0.54 0.48 0.41 0.82 1.53 1.34
p.05. p.01. p.001.
41
Moderating Effects of Teachers Years of
Experience on the Association Between Exposure to
the Consultancyand Childrens Development of
Print Knowledge
42
Standardized observation of interactions

• Is feasible, reliable and valid. A scalable
  language and lens for classroom settings
• Three domains Emotional, Organizational,
  Instructional appear valid across grades
• A lever for research on teacher professional
  development and preparation to increase setting
  quality and child outcomes
• Implications for accountability systems, teacher
  quality, research on teacher ed.