Studying the Reporting and Processing of Practice

1
Studying the Reporting and Processing of Practice

• Edit Yerushalmi and Bat Sheva EylonWeizmann
  Institute of Science
• Safed Seminar, July 2006

2
Orientation

• Focus
• the professional development program
• PD program directed at
  experienced teachers, high school physics
• Methodology strand
  proof of concept - small scale CPD

3
Well Known Challenge and Solution

• Challenge Even when instructors believe in an
  instructional goal, wish to change and direct
  their practice towards it,
• prior beliefs might conflict with it and distort
  the attempt
• VanDriel, 1997, Schoenfeld, 98, Henderson,
  2004.

Cooperative-customization workshops
To change deep-rooted practice teachers need to
take an active role, tailor innovations to
specific context, try out, reflect on their
practice, better in a community. Loucks-Horsley
et al, 1989, Putnam Borko 2000
4
Reflect

• In cooperative-customization workshops

• teachers report to peers on their new practices,
• teachers cooperatively process practice in a
  critical manner.

5
Difficulties
a) in reporting practice Ben Peretz, 2004

• Teaching events are transient, and elusive.
• Documentation is not a common teacher practice,
• and it puts a heavy load on teachers

b) in processing practice

• " the considerations underlying teachers
  instructional decisions are often entangled and
  implicit, ? not easily accessible
• teachers are expected to know the right answer
  not to experiment and possibly fail ? refrain
  from revealing difficulties they have
  encountered, and criticizing the practice and
  ideas of others
• teachers professional opportunities seldom
  involve cooperatively designing instruction ? not
  experienced in managing such a task.Loucks-Horsl
  ey et. al. 1998, Grossman et. al. 2000

6
Two Questions
What standard of reporting and processing of
practice can we expect teachers to perform in
effective customization workshops? What kind of
mentoring facilitates teachers performing to
this standard?
Methodological approach

• Microanalysis of reporting and processing in the
  context of an effective model customization
  workshop

• Suggesting design features of the customization
  workshop underlying effective reporting and
  processing

• Comparison of reporting and processing in two
  settings with and without these design features

7
Context a Model Customization Workshop

• A Challenging goal
  
  Customization and implementation of instruction
  promoting reflectivity in problem solving in the
  physics classroom
• Participants 8 high school teachers
• Schedule 3 days summer workshop yearlong
  bi-weekly meetings ( 3 hours), on-site / on-line
• Structure Flagman framework
  designed to
  support cooperative learning
• Goal achieved
  Another
  comparative study (3 yearlong workshops)

8
Flagman Model
Microanalysis of Reporting and Processing
9
Categories - Combination of theory driven (High
order) and grounded to focus on theory based
difficulties with reporting and processing.
Microanalysis of Reporting and Processing
Data one online cycle of the Flagman Framework.

Analysis Unit - statements - Each expresses a
single idea related to a specific innovation
(e.g. self-evaluation of an exam solution).
High-order categories

• Analysis of the evolution in (1) grounded
  categories (2) Mentor role

10
Flagman - Documentation - Draft

• The class 28 students, 12th grade, good grades
• Goals acquiring tools to check answers
• Concerns Should I give multiple choice or open
  problems?
• Activity plan Simple circuitsself evaluation of
  an exam solution using instructor solution
• Student materials theyve got the exam problem
  and an evaluation worksheet
• Results The exam was easy for some. Even weak
  students performed well.

11
Workshop facilitator Preliminary feedback

• did you construct the instructor
  solution in a systematic way you wanted the
  students to follow? If yes, how?
• How did you respond when students had
  difficulties?
• How did high and low achievers cope?
• Did you discuss the activity with the students?
  What did they say?

12
Flagman - Distributed documentation
Object Interaction of flagman and students'
actions, Students' attitudes Medium Materials
distributed in class Format Details Mode
object Connecting instructional design to
goals, post mortem concerns regarding students
attitudes
Processing Reporting
The instructor solution was constructed in a
systematic manner, according to the PS steps

• Steps in solving simple circuits problems
• If there are several batteries connected in the
  circuit, you have to replace them in an
  equivalent one.

The weak students viewed it as extra work!!!
13
Peers - Feedback
Reporting

• Keren it is not always possible to interchange
  with an equivalent battery, and there you have to
  use Kirchoff laws

Processing
14
Questions (flagman) and discussion (all)

• I am concerned that the weak students dont see
  the activity as worthwhile, while it is, What do
  you think?

Processing
Ben is it possible to show them? Merry Because
the outcomes are not immediate Keren They reject
because they dont understand why its good for
them Facilitator Anna, what do you think on
Bens question? Facilitator Ben, can you
answer your question? Ben NoI can talk with
them, and show them their understanding
improved Anna Do you think it is important?
Processing
15
Choosing alternative directions
Processing

• Summary Focus on the weak student
• Other questions
• 1. The weak students were harsh when
  evaluating themselves (which harmed their grade)
  do you have an idea why?
• 2.
• Possible directions 1.
• 2. Develop two
  level problem solving
• strategy more
  and less detailed

16
Evolution
17
Questions 1
What standard of reporting and processing of
practice can we expect teachers to perform in
effective customization workshops?
Microanalysis of reporting and processing in
the context of an effective model customization
workshop
Questions 2
What kind of mentoring facilitates teachers
performing to this standard?
Suggesting design features of the customization
workshop underlying effective reporting and
processing
Comparison of reporting and processing in two
settings with and without these design features
18
Is the design essential?Comparison of online and
onsite setting
Suggesting design features of the customization
workshop underlying effective reporting and
processing
Flagman model was implemented online and
onsite, Onsite 1.5 h, Bi-Weekly F2F meeting at
the Weizmann
highly structured cooperative work
Comparing the frequency with which grounded
categories of reporting and processing practice
appeared in an online implementation and a
face-to-face implementation of a Flagman
Framework cycle, in order to relate teachers
reporting and processing to the settings
respective design principles
19
Comparison Categories
teacher actions, students performance, recall of
classroom events, sparse, hierarchical, goals
alternatives for instructor's actions
Differentiation in students performance,
attitudes, Materials distributed in class,
Citations, Details, Source of organization -
flagman
Interaction of flagman and students' actions,
Connecting instructional design to goals, post
mortem concerns
Critique, revision of class materials,
Differentiating goal
Reorganization, Identification of possible
alternative actions related to goals,
hierarchical organization
Dissonance between expectations and
reality Explanations, speculations Agreement,
Suggestions
20
Online-onsite similarities

• similar types of activities related to a
  different topic, kinematics,
• same length of transcript - 242 statements
  (representative sampling of reporting
  processing ½ F2F)
• similar flagmen age, academic background,
  experience, suburban schools, school academic
  level.

21
How teachers REPORTED practice? Distribution of
grounded categories
Object
more categories, materials distributed in class
received more attention.
Medium
format
22
In accord with suggested design features

• Splitting the cooperative work into discrete
  steps, publication of textual
  artifacts in an intimate circle contribute to
  elevating reporting standards.
• Indeed, the participants asked to transfer
  meetings to online setting, they expressed that
  it better supports them
• Writing the documentation made me make order in
  my mind
• I felt as if I entered another teacher
  classroom, and could borrow a lot from him

23
How teachers PROCESSED practice? Distribution
of grounded categories
24
How teachers PROCESSED practice? Distribution
of grounded categories
Online Onsite
25
In accord with suggested design features

• Login from home allows more thorough processing
  by teachers on their peers classroom materials.
• Teachers commitment to follow formal rules
  mandates the flagman more control and
  allows a process where the teacher on stage goes
  through a reflective process, that serves all of
  his peers for reflection on their own
  experiences.

26
Questions 2
What kind of mentoring facilitates teachers
performing to this standard?
Suggesting design features of the customization
workshop underlying effective reporting and
processing
Comparing the frequency with which grounded
categories of reporting and processing practice
appeared in
an online implementation and a face-to-face
implementation of a Flagman Framework cycle
27
Summary
Microanalysis - theory driven and grounded
categories ? Reporting processing
of practice evolved through differentiation and
integration processes facilitated by mentoring
Comparison of the frequency categories appeared
in online and onsite (with and without design
features) ? Design features imposing highly
structured cooperative work promotes better
reporting and processing of practice
28
A Challenging Goal
Instructors believe instruction should promote
reflection
Students learn by reflectively attempting to work
out problems, they appreciate instructional
components directed to promote reflection (group
work, real world problems, outline in sample
solutions) Yerushalmi, Henderson, Heller,
Heller, Kuo in preperation
Yet, they implement those sparsely.

• Other considerations get in the way

In the ideal world you would use problem
solutions and grading of them far more for
teaching than for stratifying the student
population, but I think the real situation here,
and probably its typical, is that you just dont
have the time for it.
29
Comparative Study - 3 Yearlong Workshops ,

• 3 workshops, differing in mentoring approach,
• Results of macro-analysis the model workshop is
  better
• Teachers performed and initiated main stages in a
  systematic, iterative, and cumulative
  customization process.
• Main stages
• Analysis of pedagogical challenges and possible
  solutions
• Planning specific activities suited to teachers
  classrooms
• Constructing and implementing lesson plan
  classroom materials
• Evaluation and revision of implemented
  instruction
• Teachers developed variety of instructional
  strategies and materials reflecting changes in
  participants understanding.
• Yerushalmi Eylon, 2004, Yerushalmi Eylon,
  2001

30
Guideline 2 for Supporting Cooperative Learning
Explicate intermediate expert steps

• Customization stages explicit in flagman model

Cognitive Apprenticeship
31
Guideline 3 for Supporting Cooperative Learning
Incentives for true cooperation
Social interdependence theory Johnson Johnson,
2004
Teachers depend on each other to refine
instruction documentation provides anchor,
feedback points out pros and cons
? Positive interdependence
? Individual and group accountability
Each group member held responsible for specific
duties, Outcomes of FLAGMAN cycle required for
next stage
32
Distribution of Reporting and Processing
Categories