Using Middle Grades Student Achievement Data to Support

1
Using Middle Grades Student Achievement Data to
Support
Theoretical Teacher Quality Measures

• Robert M. CapraroMary Margaret CapraroTamara
  Anthony Carter Adam P. HarbaughEmilie
  NaiserChristopher Romero

Texas AM University
2
Theoretical Framework

• It is important to understand the nexus of
  theorized Teacher Quality Measures (TQMs) and
  classroom enactments of learning goals. (Kelly
  Lesh, 2000)
• Applicability and interpretability of TQMs is
  essential to construct development.
• TQMs devoid of meaningful content expectations in
  classroom enactments are meaningless to
  mathematics education research.

3
Genesis of our TQMs

• Adapted from Curriculum Quality Measures of AAAS
  middle grades textbook analysis
  http//www.project2061.org/tools/textbook/matheval
  
• Probing understanding
• Encouraging curiosity questioning
• Providing practice
• Using representations

4
A Teachers Perspective

• Emily Naiserenaiser_at_bryanisd.org

5
What benefits has the project provided for me?

• Reflection on my teaching
• Time to analyze tapes and set goals
• A way to monitor growth
• A new perspective - the opportunity for you to be
  a learner in your own class

6
Research Questions

• Do our TQMs provide quantifiable differences on
  focused, high-quality assessments and minimal
  skills tests?
• To what extent are the measures useful for
  improving student achievement?
• Can student achievement data be used to
  substantiate and support the theoretical model?
• 4. To what extent does teacher in-service affect
  teaching practice?
• 5. How do teachers represent mathematical ideas?

7
Reliability Verifiability

• Quantitative rating of qualitative data was
  subjected to inter- and intra-rater reliability
  assessment.
• Formal coding of the videos began following
  achievement of 0.80 reliability.

8
Development of a Coding Schema

• Video Data coded in 20 second intervals.
• Teacher assessed during each interval for each
  indicator on a tripartite scale
• 2 teacher met the indicator
• 1 teacher partially met the indicator
• 0 teacher did not meet the indicator
• X no opportunity to meet the indicator

9
Why 20 seconds?

• We found 20 seconds was the most appropriate
  partition for assessment of TQMs.
• Longer intervals could lead to aggregation of
  occurrences within a particular interval.
• Shorter intervals provide less opportunity to
  demonstrate TQM.

10
Example of Coding Artifact

11
Teacher Background Data

• Over two years, two 6th grade teachers were
  studied Ms. H. Ms. W.
• Video data were part of a larger NSF-IERI
  project.
• Classroom learning goal of featured lessons was
  to use, interpret, and compare numbers in several
  equivalent forms particularly fractions,
  decimals, and percents.

12
Questions

• Robert M. Caprarorcapraro_at_coe.tamu.edu

13
Criterion 1 Probing Understanding

• Tamara Anthony Cartertcarter_at_tamu.edu

14
Theoretical Basis

• Asking more open-ended questionscan contribute
  to the construction of more sophisticated
  mathematical knowledge by students (Martino
  Maher, 1999, p. 53-54).
• Classroom experiences need to lead to the
  formulation of generalizations, justification of
  thinking, and the search for insights, which open
  new areas of investigation (Burns, 1985).

15
Probing Questioning -- Criterion

• Does the teacher facilitate engagements with
  assessment questions or tasks that require
  students to show, use, apply, and explain their
  understanding?

16
Engagements via questions or tasks

• are focused on the use of the knowledge or
  skills specified in the learning goals
• are presented in new and unique ways.
• do not use the exact context of the learning
  goals.
• Illuminate commonly held student notions which
  are relevant to the learning goals.

17
Coding Examples

• 2 (met) Questions requiring students to apply,
  explain, express, justify, interpret, describe,
  predict, design, discuss positions, summarize
  what they have learned, or otherwise demonstrate
  understanding. Example
• 1 (partially met) Questions requiring a word or
  phrase answer, but that answer must require
  individual creativity.
• 0 (did not meet) Questions requiring only a
  specific word or phrase (no individual
  creativity) or a numerical answer or class time
  with teacher interaction without questions.
  Example

18
Criterion 2Encouraging Questioning

• Adam P. Harbaughadam-harbaugh_at_tamu.edu

19
Theoretical Basis

• Teachers can foster students motivation by
  stimulating curiosity or suspense. (NRC, 2001)
• One of the teachers roles in the classroom is to
  encourage students development of verbal
  representations of mathematics through classroom
  communication with peers and the teacher. (NCTM,
  1991)
• A persistent dialogue that elicits questions can
  extend a students search for answers.
  (Martinello, 1998)

20
Encouraging Curiosity Questioning

• Does the teacher help to create a classroom
  environment that welcomes student curiosity,
  rewards creativity, encourages a spirit of
  healthy questioning, and avoids dogmatism?

21
Teachers can encourage curiosity and questioning
by

• providing opportunities for students to express
  their curiosity and/or creativity.
• providing occasions for students to ask
  questions and guides their search for answers.
• modeling the types of questions that he/she
  expects students to ask.
• respecting and valuing students ideas.

22
Coding Examples

• Indicator The teacher respects and values
  students ideas.
• 2 (met) Eliciting student ideas AND providing
  appropriate feedback. Example
• 1 (partially met) Elicits student ideas without
  giving appropriate responses.
• 0 (did not meet) Does not take the opportunity
  to show respect for or value students ideas
  Example

23
Criterion 3Providing Practice

• Christopher Romerocromero_at_tamu.edu

24
Theoretical Framework

• Wait time is essential for appropriate student
  conceptualization.(Tobin, 1983)
• Everyday mathematical practice reflects a higher
  level of thinking than is typically accomplished
  in school.(Lester, 1989)
• Mathematics Practice is context specific
  effective practice must be embedded in real
  contexts. (Masingila, 1995)

25
Appropriate Practice Providence

• requires students to generate individual ideas
  strategies.
• is accommodated with sufficient wait time (i.e.
  at least 15 seconds).
• gives each student the opportunity to
  conceptualize the item before a path to the
  solution or the solution itself is revealed.
• efficiently handles any non-ideal,
  irreconcilable classroom issues (e.g. not enough
  manipulatives).
• practice which requires less than 15 seconds is
  likely not meaningful

26
Coding Examples

• 2 (met) The teacher shows evidence of
  appropriate practice.
• Whole class participation
• Meaningful activity
• Example
• 1 (partially met) The teacher provides less
  than appropriate practice.
• Teacher singles out one student for practice
• 0 (did not meet) The teacher provides
  inappropriate or no practice.
• Teacher dominated reporting of previous practice
• Example

27
Criterion 4Using Representations

• Mary Margaret Caprarommcapraro_at_coe.tamu.edu

28
Theoretical Basis

• Representational forms are essential to teaching
  mathematical ideas (NCTM, 1991, 2000).
• Children do not have enough abstract thinking
  ability to learn abstract mathematical
  conceptions presented in words or symbols alone
  (Piaget, 1952).
• Goldin and Kaput (1996) postulated that the
  representational forms presented by teachers
  influence how students develop mathematical
  understandings.
• Some forms of representation have been taught
  and learned as if they were the ends in
  themselves. This approach limits the power and
  utility of representations as tools for learning
  and doing mathematics (NCTM, 2000, p.14).

29
Representing Ideas Effectively

• Does the teaching include accurate and
  comprehensible representations of the learning
  goals?

30
Teachers can represent ideas effectively by

• using two or more representations (verbal,
  symbolic, pictorial, and/or manipulatives).
• accurately representing relevant aspects of the
  learning goal and bringing out the limitations of
  the representations.
• using representations as
• re-presentations of the learning goal.

31
Coding Examples

• 2 (met) The teaching uses at least two
  representations
• (verbal, symbolic, pictorial and/or
  manipulative) AND makes connections to the
  mathematics being represented for the students.
  Example
• 1 (partially met) The teaching uses only one
  representation (verbal, symbolic, pictorial
  and/or manipulative) BUT does not make a
  connection to mathematics being represented for
  the students.
• 0 (did not meet) The teacher does not take the
  opportunity to use a representation in any of the
  ways previously mentioned. Example

32
Methodology Questions

• Probing understanding
• Encouraging curiosity questioning
• Providing practice
• Using representations

33
Probing Questioning Results

• Emerging patterns
• Both teachers
• Every lesson
• All parts of lesson
• All group sizes

34
Probing Questioning Descriptive Statistics
Statistically significant difference, p 0.01
Practically significant difference, Cohens d
2.62
35
Ms. H. Time by
Probing Questions Ms. W
36
Probing Questioning Type of Question
37
Questioning and Curiosity Results

• Both teachers used the following
• Asking for student ideas, opinions, and
  explanations
• Working at the board under the direction of a
  student
• Having students share ideas with others
• Scrutinizing student ideas, opinions, and
  explanations
• Complimenting student work and ideas
• Revoicing or amplifying student ideas verbally
  and on the board
• Differences in frequency and quality of
  occurrences
• Reform-oriented vs. Teacher centered

38
Questioning and Curiosity Results
39
Time by Encouraging Curiosity Questioning
Ms. H
Ms. W
40
Questioning and Curiosity Results
Asking for student ideas, opinions, and
explanations

• Ms. W
• You guys have been working hard, you've been
  thinking good, and I want some of you to get a
  chance now to share some of your strategies.

• Ms. H
• Raise your hand if you got 53 hundredths. Raise
  your hand if you got a different answer you would
  like to share.

41
Questioning and Curiosity Results
Complimenting student work and ideas

• Ms. H
• Jonathan, you've got usually good ideas

• Ms. W
• I like how you used equivalent fractions. Do you
  all see how he used equivalent fractions?

42
Questioning and Curiosity Results
Scrutinizing student ideas, opinions, and
explanations

• Ms. H
• How come that works? You're not really
  multiplying by 5. 5 over 5 is?...Yeah, the whole
  number one

• Ms. W
• okay. So I heard you say something aboutso how
  do you know that?

43
Providing Practice -- Reflections

• Ms. W. was more consistent in her wait time
• Ms. H. provided inappropriate practice
• No quotes (of good providence) to cite

44
Providing Practice Results
45
Time by Providing Practice
Ms. H Ms. W
46
Providing Practice -- Reflections

• Longer practice blocks maximize wait time and
  encourage deeper student conceptualization
• Longer practice blocks are consistent with
  reformed paradigms
• Practice Providence requires teacher inactivity.

47
Using Representations Results
48
Ms.H. Time by Representations
Ms. W.
49
Representations - Ms. W.

• Use of fraction strips
• Strips are used accurately by the teacher
• They are generally comprehensible to students
  since they are made by the students themselves
  during a semi-structured lesson
• Students understand the limitations of the hand
  made versions
• Students use them to explain and justify their
  answers

50
Representations - Ms. H.

• Use of pattern blocks
• Blocks are used accurately/limitations not
  mentioned, however,
• Students use them in many tasks
• Equivalency (2 green triangles 1 blue diamond)
• Mixed numbers (2 hexagons 1 trapezoid)
• Rolling dice numerator and denominator (1-6)
• Pictorial representations (on board/paper)
• Real-world word problems (6 dogs 32 bags of
  food)
• Transfer to symbols (2/6 1/3)

51
Chris
52
Internal Consistency- Number

• Cronbachs Alpha for Number

53
Pre/Post Descriptives
Mean 15.3 SD 7.19 n 152
Mean 14.2 SD 6.76 n 165
54
Pre-test Performance by Year
55
Post-test Performance by Year
56
Regression Results
Regression of Posttest on TQM Measures
R2 .382 p lt.01
57
Correlation Matrix
Note. p lt . 05 p lt .001
58
Data Supports the Theoretical Model
MANOVA of TQMs on Dep. Synthetic Theoretical
Model Var.
59
To what extent does inservice impact practice
Regression of Longitudinal Posttest on TQM
Measures
R2 .155 p lt.01
60
Contact Information

• Robert M. Capraro rcapraro_at_coe.tamu.edu
• Mary Margaret Capraro mmcapraro_at_coe.tamu.edu
• Tamara Anthony Carter tcarter_at_tamu.edu
• Adam P. Harbaugh adam-harbaugh_at_tamu.edu
• Christopher Romero cromero_at_tamu.edu
• Emily Naiser enaiser_at_bryanisd.org
• For a copy of the presentation, go to
• http//www.coe.tamu.edu/rcapraro/
• NSF-IERI Grant REC-0129398
• Improving Mathematics Teaching and Achievement
  Through Professional Development