RePresenting Teaching: Perspectives on the Illusive Notion of Quality

1
(Re)-Presenting Teaching Perspectives onthe
Illusive Notion of Quality

• UMCP/MCPS Partnership Study of
• High Quality Teaching in
• Mathematics Reading
• Supported by a grant from
• Interagency Education Research Initiative
• a combined effort of
• the USDE, NIH, and NSF

2
Background of the HQT Study

• Assumption importance of teachers and teaching
• Context
• lack of foundational skills
• 4th grade slump
• achievement gap

3
Research Focus

• What do teachers do to
• help students achieve above expectations in
  reading and mathematics?
• close the achievement gap?
• How do they change their pedagogical practices?
• What is the influence of education policies and
  organizational factors?
• What is the correspondence between high quality
  teaching constructs and student learning?

4
Description of Study Site Participants

• Montgomery County Public Schools
• 19th largest school system in US
• 136,000 students
• Over 100 different languages represented in ESOL
  population
• 1/3 of schools have poverty enrollments over 40
• Participating Schools and Teachers
• 76 4th 5th grade teachers
• 18 schools (30-85 FARMS enrollments higher
  than expected student performance)

5
Data Sources for Symposium Papers

• Observation Instrument
• Time sampling protocol focuses on Instructional
  Practices. Its completed by trained observers,
  entered on lap top. Over 10,000 episodes for
  reading and for mathematics.
• Daily Log
• Focuses on Curriculum Coverage. Completed by
  teacher, data on time and content for one student
  from class roster entered into PDA. Over 3,300
  daily log entries in reading and in mathematics
  (average length 74 class minutes per entry)
• Principal Interviews
• With 16 principals focused on support for
  high-quality teaching and student learning in
  Grades 4 5 and how the achievement gap was
  addressed.

6
Time Sampling Coding Screen
7
(No Transcript)
8
Representing Reading Teaching Through Classroom
Observations and Daily Logs

• Marilyn Chambliss, Robert Croninger,
• John Larson, Anna Graeber
• and Linda Valli

9
What is Good Reading Instruction?

• Promoting IRA NCTE Standards?
• Matching state and school district goals and
  prescriptions?
• Practices that produce high standardized test
  results?
• Practices of nominated high-quality teachers?
• How is it studied and measured?

10
Synthesis of Findings High quality reading
instruction

• Allows students to have choice about interesting
  challenging text of a variety of genres and
  encourages them to respond personally to those
  texts
• Promotes dialogue about reading and writing
• Creates a balance between comprehending and
  reasoning about texts and necessary skill
  instruction

11
High Quality Teaching in Reading Math Common
Constructs

• How teachers understand and represent subject
  matter
• The existence of a classroom discourse community
• Level of cognitive demand of activities and
  context

12
Most Frequent Teacher/Student Activity

• Teacher
• Manages Activity
• Poses Low Order
• Listens to, Watches
• Reads Student Work

• Student
• Management
• Responds Simple Answer
• Listens, Watches

13
Least Frequent Teacher/Student Activity

• Teacher
• Extrinsic Reward
• Posts Outline
• Redirects Conversation

• Student
• Writes Outline
• Performs Speech (Extemporaneous)
• Questions Another S
• Performs Play
• Views Video, DVD

14
Most and Least Frequent Content

• Most
• Management/Activity
• Story/Poetry Elements, Text Design
• Reading Text

• Least
• Fluency
• Conventions
• Writing about Genre

15
Cognitive Demand

• The degree to which a teacher
• presses for or evokes reasoning, reflection on
  learning, and higher order thinking, and
• Engages students in demanding content using
  challenging genres.

16
Teacher Activity Cognitive Demand

• Low Demand
• Poses low order task, question, problem
• Elaborates low order task, question, problem
• Manages an activity if students are in
  transition, socializing

• High Demand
• Requests student reflection on learning
  alternative answer student self-assessment
  elaboration of student response attention to a
  response
• Poses high order task, question, problem
• Elaborates high order task, question, problem
• Manages an activity if students are on task

17
Student Activity Cognitive Demand

• Low Demand
• Responds with or states simple answer/statement
• Writes exercises, brief writing, or notes
• Views video
• Illustrates

• High Demand
• Responds with or states hypothesis, prediction
  explanation, justification alternative answer/
  statement elaborated answer/statement
• Writes extended writing, graphic organizer,
  outline
• Performs preplanned student composed speech,
  extemporaneous speech, author composed play

18
Reading Content Cognitive Demand

• Lower Demand
• Reading comprehension literal response
• Fluency
• Decoding
• Spelling
• Conventions
• Viewing, listening
• Illustrating

• Higher Demand
• Reading comprehension genre, theme or main idea,
  expository text design, poem elements, personal
  response, strategy
• Writing related to reading genre, theme or main
  idea, expository text design, poem elements,
  strategy
• Writing unrelated to reading prewriting,
  composing, editing, revising

19
Reading Observation Part Scores for T/S
Interaction

• Higher Cognitive Demand
• Teacher 10 Student 10
• Lower Cognitive Demand
• Teacher 17 Student 19
• Any Instruction
• Teacher 65 Student 86

20
Reading Observation Part Scores for Content

• Higher Content 30
• Lower Content 10
• Any Reading 46
• Any Writing 17
• Any Usage/Conventions 6

21
Cognitive Demand T/S Interaction

• Higher cognitive request with higher student
  response (r .53)
• Higher cognitive request with lower student
  response (r .22)
• Lower cognitive request with lower student
  response (r .57)

22
Cognitive Demand and Classroom Organization

• Ts request higher or lower in mixed
• (r. . 23, r. .25)
• Ss respond higher or lower in mixed
• (r .22 r. .20)
• Higher content in independent
• (r. .16)
• Lower content in whole group
• (r. .15)

23
Cognitive Demand in Log Data

• Percent Class Time
• Exposition 32.3
• Narrative 35.2
• Poetry 3.5

24
Time Spent on Various Genres

• Most Time
• Exposition
• Newspaper articles
• Essays
• Trade books
• Narrative
• Realistic Fiction
• Historical Fiction
• Science Fiction

• Least Time
• Speeches
• Letters
• Journals
• Plays
• Mythology
• Mysteries

25
HQT Reading Instruction

• Promotes a balance of exposition and narrative
  chosen by the teacher to be personally relevant
  to students lives.
• Promotes dialogue about reading primarily in
  teacher-led small groups.
• Creates a balance between comprehending and
  reasoning about texts and necessary skill
  instruction with the majority of the instruction
  being neither higher nor lower cognitive demand.

26
Synthesis of Findings High quality reading
instruction

• Allows students to have choice about interesting
  challenging text of a variety of genres and
  encourages them to respond personally to those
  texts
• Promotes dialogue about reading and writing
• Creates a balance between comprehending and
  reasoning about texts and necessary skill
  instruction

27
Representing Mathematics Teaching Through
Classroom Observations and Daily LogsSome
Views of Cognitive Demand

• Anna Graeber, Robert Croninger, John Larson,
• Linda Valli and Marilyn Chambliss

28
Perspective of Cognitive Demand

• Important aspect of instruction
• Defined in numerous ways in mathematics education
  literature
• In this paper, the terminology, higher and lower
  cognitive demand is used
• Each perspective gives some insights

29
Time Spent on Math

• District guidelines 60 minutes
• Sample average 66 minutes
• (Daily log data)
• National average, (Mahlzahn, 2001)
• Grades 3-5 61 minutes

30
Classroom Organization

• Whole Class 54
• Independent Work 25
• Small Group (12) and
• Mixed Group Indep. 20

31
Major Coding Categories Teacher Activity

• Requests (H)
• Poses (H,L)
• Elaborates on (H, L)
• Responds/States
• Models
• Defines
• Posts
• Lectures
• Reads Aloud from Text
• Listens/Watches
• Manages (Activity, Materials, Behavior)
• No Obvious Instruction

32

Percent of Lesson Time in Various Teacher
Activities

33

Percent of Lesson Time Teacher Calls for Higher
and Lower Order Cognitive Responses

• Requests, Poses or Elaborates on Problem,
  Question or Task that is High Order -- 11
• Poses or Elaborates on Lower Order Question or
  Routine Task --20

34
Major Coding Categories Student Activity

• Responds/States (H,L)
• Works on (H,L)
• Formal Assessment
• Asks a Question,Reads from Text,Writes on Board
• Listen/Watch
• Management
• Mixed
• No Apparent Academic Behavior

35
Percent of Time for Various Student Activities
36

Percent of Lesson Time Students Productions Are
of Higher and Lower Order

• Responds with conjecture, explanation, or
  alternative method or does extended writing or
  works on problem or task--17
• Responds with simple answer or works on routine
  exercises --33

37

Mathematics Content Coding Categories

• Conceptual
• Procedural
• Linking Conceptual and Procedural
• Learning Strategies - Conceptual , Procedural
• Management
• Mixed
• Non-Instructional

38
Percent of Time on Various Mathematics Content
39
Teacher Calls for Higher Order Student
Production of Higher Order Over Time

40
Mathematics Lesson Content Over Time

41
Convergences Across Classroom Observations
A Multilevel Exploration of Teachers Requests
Students Oral Responses
by Robert Croninger, John Larson Clare VonSecker
42
Background for Exploration

• One representation of high-quality teaching is
  the extent to which teachers deeply engage
  students in subject matter.
• A form of engagement is the extent to which
  teachers solicit (and receive) cognitively
  demanding oral responses from students.
• What do classroom observations in reading and
  mathematics say about this form of high-quality
  teaching?

43
First Set of Questions

• Does the quality of students oral responses
  (higher v. lower cognitive demand) vary across
  lessons, classes, schools?
• What characteristics of instructional practices,
  classes schools might help to explain any
  variation that exists in the quality of students
  oral responses?

44
Second Set of Questions

• Does the quality of students responses vary with
  the quality of teachers requests (higher v.
  lower for cognitive demand)?
• Does the effectiveness of teacher requests (i.e.,
  the covariance of requests and responses) vary
  between classes schools?
• What characteristics of classes schools might
  help to explain any variation that exists in the
  effectiveness of teachers requests?

45
Analytic Sample

• Reading observations
• 412 observed lessons (7 per class)
• 62 4th 5th grade classes (4 per school)
• 17 elementary schools
• Mathematic observations
• 434 observed lessons (7 per class)
• 66 4th 5th grade classes (4 per school)
• 18 elementary schools

46
Dependent Primary Independent Variables

• Dependent variable
• Higher v. lower cognitively demanding responses
  by students.
• Primary independent variable
• Higher v. lower cognitively demanding requests by
  teachers.

47
Explanatory Control Variables

• Average instructional practices across lessons,
  classes or schools (e.g., use of more v. less
  demanding lesson content, more v. less
  instructionally focused management activities,
  organization of students for instruction).
• Composition of classes schools (e.g., average
  proportion of at-risk students per class
  (free-reduced price lunches, English-language
  learners, special education students), school
  average of proportion of at-risk students per
  class).

48
Analytic Strategy

• Use multilevel modeling (HLM) to
• Partition variance in student responses between
  lessons, classes schools.
• Partition variance in the effectiveness of
  teacher requests between lessons, classes
  schools.
• Develop class-level school-level models to
  explain variability at each level.

49
Quality of Student Responses

• Reading

• Mathematics

50
Variability Between Lessons

• The more cognitively demanding the requests by
  teachers, the more cognitively demanding the
  responses given by students during lessons.
• Average effects of teacher requests similar in
  reading mathematics, .48 SD .46 SD
  respectively.

51
Variability Between Classes

• More cognitively demanding responses in classes
  where teachers requested more cognitively
  demanding responses across lessons (.18 SD,
  reading .13 SD, mathematics).
• Less demanding responses in mathematics classes
  with higher levels of procedural content (v.
  conceptual linking) across lessons (-.21 SD).
• Less demanding responses in reading classes with
  higher proportions of English-language learners
  (-.24 SD) less demanding responses in
  mathematics classes with higher proportions any
  at-risk students (-.12 SD).

52
Variability Between Schools

• More cognitively demanding responses in schools
  where more cognitively demanding content focus of
  lessons across reading classes within schools
  (.14 SD).
• Less demanding responses in schools with higher
  proportions of any at-risk students across
  mathematics classes within schools (-.12 SD).

53
Effects of Teacher Requests

• Effects of teacher requests vary significantly
  between classes schools in reading
  mathematics.
• Estimate of potential range of effects across
  classes is similar for reading mathematics,
  roughly 0 to .95 SD.
• Estimate of potential range of effects across
  schools is also similar, roughly .19 SD to .75
  SD.

54
Variability Between Classes

• Effects of more cognitively demanding requests
  are greater in reading classes where management
  focuses on instructional activities more than
  materials or student behavior (.13 SD).
• Effects of more demanding requests are greater in
  mathematics classes that use more small-group and
  mixed forms of classroom organization (.13 SD).

55
Variability Between Schools

• Models failed to explain any variability in
  effectiveness of teacher requests between
  schools.
• Neither average class instructional practices nor
  average class compositions accounted for the
  variance.
• Small number of schools (17 for reading, 18 for
  mathematics) makes it difficult to model
  variation in effects between schools.

56
Conclusions

• Quality of student responses varied mostly
  between lessons, but some classes schools
  displayed greater tendency to engage students
  through demanding oral exchanges.
• More demanding teacher requests led to more
  demanding student responses, regardless of
  subject matter a possible representation of
  high-quality teaching.
• But effectiveness of teacher requests varied
  across classes schools a possible
  representation of contextual effects.

57
Conclusions Continued

• Higher-levels at-risk populations associated with
  lower-levels demanding student responses, though
  no indication higher-levels at-risk populations
  influence effects of teacher requests.
• Evidence greater focus on demanding content
  associated with more demanding student responses
  may represent unmeasured aspects of classrooms
  schools, teacher knowledge or pedagogical
  skills and/or school-level curricular decisions
  (e.g., in reading).

58
Conclusions Continued

• Effects of teacher requests associated with other
  practices may represent unmeasured aspects of
  classrooms and schools, teacher knowledge or
  pedagogical skills.
• More fine-grained measures about school policies
  derived from qualitative data may help to reveal
  possible sources of variation in the effects of
  teacher requests between schools.

59
Principals and the Organization of Students and
Teachers in an Era of High-Stakes Accountability

• Jeremy Price, Daria Buese,
• and Caroline Eick

60
Overview

• We look at ways that 16 principals, during the
  2003-2004 school year, approach the challenge of
  organizing students and teachers in their
  schools.
• We consider principals responses to federal,
  state and district instructional policy
  initiatives.
• We pay attention to how principals use data to
  allocate human resources to meet the learning
  needs of various categories of students.

61
Factors Influencing Organizational Design

• Mediating Effects
• Organic Controls
• Distribution of Resources
• Access to Knowledge
• Assessment Practices

62
Policy Context

• Curriculum revised to meet the expectations of
  the State Core Learning goals.
• Multiple assessments identified as significant in
  monitoring student progress towards AYP (e.g.,
  MSA, CTBS, MCPS-AP).
• Collaboration promoted among teachers,
  principals, specialists and staff.

63
Using Data to Increase Instructional Capacity

• We talk about this whole instructional piece as
  a continuous cycle that goes on between planning
  instruction and assessmentThis cycle is
  becoming something that drives the program here.
  (Young)
• Im moving in a way to have teachers be the
  collectors of data and them sharing with me what
  the data is telling them about the performance of
  youngsters and how its going to drive their
  instructionbuilding their capacity to use data.
  (Edwards)

64
Data Avalanche

• Data to diagnose learning needs.
• You should have the data dripping off the
  ceiling. (Young)
• Data to identify student achievement level in
  mathematics and reading.
• "Well, okay, you have Joe in your classroom
  reading/language arts, he scored a two this time.
  What can we do more with him so we can get him
  to that three the next time? (Moore)

65
Organizing Human Resources to Address Student
Learning Needs

• Red-flagging
• Plug-in or Pull-out
• Double-Dipping

66
Teacher Organization and AYP

• Principals routinely spoke of bringing together
  specialists and classroom teachers in
  collaborative teams.
• Principals varied in how they used specialists in
  working teams.
• Principals exerted different degrees of
  flexibility in determining duration, frequency,
  and content of team meetings.

67
Data Analysis and Working Teams

• they meet to look at the student work.
  (Consuelos)
• looking at student work is important. (Bauer)
• Teachers are encouraged to discuss their
  students in terms of what the students learned
  (Hansen)
• looking at student work to drive instruction.
  (Fry)
• and they're looking at best practices with how
  children are doing and really doing a good review
  of student work. (Moore)
• Were looking at student work inteamwork.
  (Brooks)
• looking at student work and trying to figure
  out what we needed to do next. (Greene)

68
Conclusion

• Principals perspectives about the organization
  of teachers and students suggest that data and
  data analysis play a central role in
  instructional decision making. Data and data
  analysis are used for
• Determining students learning needs
• Assigning resources to particular categories of
  students
• Organizing teams of specialists and teachers
• Shaping the focus and content of working team
  meeting