Formative Assessment: A Method to Close the Feedback Loop

1
Formative Assessment A Method to Close the
Feedback Loop

• Eugenia Etkina,
• Graduate School of Education
• Rutgers University
• 2005 Winter AAPT
• Albuquerque, NM

2
Members of Rutgers ASA Project

• Alan Van Heuvelen,
• Sahana Murthy,
• David Brookes, Aaron Warren,
• David Rosengrant, Maria Ruibal Vilassenor,
• Suzanne Brahmia, Julia Timofeeva
• NSF ASA Program
• http//paer.rutgers.edu/scientificabilities/
• http//paer.rutgers.edu/PT3

3
Outline

• Assessment
• Formative vs summative
• Three steps of formative assessment
• Self-assessment
• Examples

4
Why do we need assessment?

• One of the purposes of assessment within
  education is that of informing and improving
  students ongoing learning
• Summative and formative

5
Formative Assessment

• Formative assessment the process used by
  teachers and students to recognize and respond to
  student learning in order to enhance that
  learning during learning.
• Gains reported due to formative assessment are
  the largest reported for an educational
  intervention (Black and Wiliam).

6
Essential components of formative assessment

• Teacher giving feedback to the students
• The teacher and students taking action to improve
  learning during learning
• Self - assessment

7
Three essential steps
8
Self-assessment

• Students must be be able to understand and use
  the criteria with which they are assessed, in
  order to bridge the gap between what they know
  and can do and the desired goal

9
What do we want to assess?

• Conceptual understanding
• Problem solving
• Scientific abilities

10
What are some scientific abilities?

• Ability to represent a process in multiple ways
• Ability to design an experimental investigation
  (an observational experiment a testing
  experiment an investigation to solve a problem)
• Ability to collect and analyze experimental data
• Ability to construct and modify explanations
• Ability to evaluate all of the above

11
Formative assessment tasksand rubrics

• Assessing students scientific abilities
• Multiple representation tasks (D. Rosengrant, A.
  Van Heuvelen, E. Etkina)
• Experimental design tasks (S. Murthy, E. Etkina)
• Anomalous data tasks (D. Brookes, E. Etkina)
• Video problems tasks (D. Brookes, E. Etkina)
• Evaluation tasks (A. Warren, A. Van Heuvelen)
• http//paer.rutgers.edu/scientificabilities
• http//paer.rutgers.edu/pt3

12
Using rubrics

Design two independent experiments to determine
the specific heat of the given unknown object.
SCORE ABILITY 0 1 2 3
To identify assumptions made in the chosen procedure No attempt is made to identify any assumptions. An attempt is made to identify assumptions, but most are missing, described vaguely, or incorrect. Most assumptions are correctly identified All assumptions are correctly identified.
Student writing samples
the block and water will reach equilibrium
after 10 minutes no heat goes in and out of
the calorimeter minimal heat is lost to the
environment during transfer of the block the
temperature inside the beaker is homogenous
SCORE 3
No heat exchange between system and
surroundings, no temperature gradient
inside. SCORE 1
13
Using rubrics

Design two independent experiments to determine
the specific heat of the given unknown object.
SCORE ABILITY 0 1 2 3
To determine specifically the way in which assumptions might affect the results No attempt is made to determine the effects of assumptions. An attempt is made to determine the effects of some assumptions, but most are missing, incorrect or described vaguely Effects of most assumptions are determined correctly, though a few contain minor errors, inconsistencies, or omissions. The effects of all assumptions are correctly determined
if the heat is lost from the block during
transfer, the specific heat obtained in the
experiment would be smaller than what it should
be. SCORE 3
the assumptions will affect the results, the
actual value may be different from the
experimental one. SCORE 1
14
Development of rubrics

• Identifying important sub-abilities
• Writing descriptors (scale 0-3)
• Finding clear wording
• Scoring student work
• Discussing the items with a disagreement
• Revising wording

Name (9 rows) Able to Able to Able to Able to Able to
Suzanne 3 1 1 3 2
Sahana 3 1 3 2 2
15
Where do we use them?

• Lectures - electronic student response system wit
  peer interactions and instructor feedback
• Recitations - interactions with peers and TA
• Labs - interactions with peers, self assessment
  with rubrics,interactions with a TA
• Homework - interactions with a TA, posted
  solutions
• Exams (summative - research purposes)

16
Ability to represent phenomena in multiple ways

• Free-body diagrams where do you need to go?
• An elevator is slowing down An elevator is
  slowing down on
• on its way up its way down

Earth, cable
17
Rubric for self assessement
Free-body diagrams How to get there?
Scientific Ability 0 1 2 3
Ability to construct free-body diagrams No FBD is drawn FBD is constructed but contains major errors such as an incorrect relative length, wrong direction, extra incorrect force vectors, a missing force vector, or a force exerted on a different object. FBD contains no errors in forces but lacks a key feature such as labels or forces are mislabeled or axes are missing or mis labeled. The diagram contains no errors and each force is labeled, axes are present and labeled.
18
Formative assessment task in lecture
Where are you now?
Which free-body diagram best represents the ball
thrown In the air?
19
Do students actually use FBDs?
20
Problem solving strategy

• Where do you need to go
• Picture and Translate
• Sketch the problem situation include all known
  information.
• Choose a system object and make a list of
  objects that interact with the system.
• Indicate the direction of acceleration, if you
  know it.
• Simplify
• Determine if you can ignore any interactions of
  the environment with the system object.
• Represent Physically
• Draw a free-body diagram for the system.
• Represent mathematically
• Apply Newtons second law in component form to
  the situation you represent in the free-body
  diagram.
• Add kinematics equations if necessary.
• Solve and evaluate

21
Ability to evaluate somebodys problem solving

• Where are you now?
• The problem A 1000-kg elevator is moving down at
  6.0 m/s. It slows to a stop in 3.0 m as it
  approaches the ground floor. Determine the force
  that the cable supporting the elevator exerts on
  the elevator as it stops. Assume that g 10
  N/kg.
• Proposed solution The acceleration of the
  elevator is
• a vo2/2d (6.0 m/s)2/2(3.0 m) 6.0
  m/s2.
• The force of the cable on the elevator while
  stopping is
• T ma (1000 kg)(6.0 m/s2) 6000 N.
• How to get there?
• Identify all missing elements in this solution
• Identify any errors in this solution.
• Provide a corrected solution if there are errors.
  

22
Ability to devise relationships and test them

• Observe the fall of two objects dropped
  simultaneously. Describe your observations in
  words, with a motion diagram and mathematically.
• What if?
• Predict what will happen to the distance between
  two objects one of which was dropped slightly
  before the other
• a) The distance will stay the same b) the
  distance will decrease c) the distance will
  increase d) not enough information to answer.
• Explain your prediction explain using words,
  motion diagrams, and mathematics. List
  assumptions.
• Observe the experiment and revise your
  explanation if necessary.

23
Example of student reasoning
The motion diagrams below represent the positions
of the balls at every frame. The distance
between them should increase.

• 1.
• 1. 2.
• 2. 3.
• 3. 4.
• 4. 5.

24
Rubric for self-assessment
Scientific Ability 0 1 2 3
Is able to make a reasonable prediction based on a relationship or explanation. No attempt to make a prediction is made. A prediction is made but it doesnt follow from the relationship or explanation being tested, or it ignores or contradicts some of the assumptions inherent in the relationship or explanation. A prediction is made that follows from the relationship or explanation and incorporates the assumptions, but it contains minor errors, inconsistencies or omissions. A correct prediction is made that follows from the relationship or explanation and incorporates the assumptions.
25
Rubric for self-assessment
Is able to revise the explanation of a prediction, based on the results of an experiment. No attempt is made to explain the outcome of the experiment, revise explanation or assumptions. An attempt is made to explain the outcome and revise the previous explanation or assumptions but is (a) mostly incomplete and/or (b) based on incorrect reasoning. The revision of the previous explanation or assumptions is partially complete and correct, yet still lacking in some relevant details. The revision of the explanation or assumptions is explained completely and correctly.
26
Ability to deal with anomalous data

Real physics with formative assessment
27
Ability to deal with anomalous data

• Predict what will happen to the left bob
• (Nothing will go up will go down will go
  up and down will swing left to right)
• Explain your prediction using free-body diagrams.
  List additional assumptions.

28
Making a prediction

Fstring on bob
Fstring on bob
Left bob
Right bob
FEarth on bob
FEarth on bob
29
Revising the model or the assumptions

• Observe the experiment and revise your
  explanation if necessary.
• Decide whether you need to revise the model that
  you used to make a prediction or the additional
  assumptions.

30
The END

• Thank you!

31
Design observation experiment

• Design an experiment to determine if there is a
  relationship between the pressure and temperature
  of the gas inside a sealed hollow sphere.
• Equipment Sealed hollow metal sphere with a
  pressure gauge, hot plate, thermometer.
• a) Describe your experimental design. Include
  the following
• How will you vary the temperature? How will you
  measure it?
• How will you vary the pressure? How will you
  measure it?
• b) What other equipment do you need?
• c) Draw a labeled diagram of your experimental
  set-up.
• d) Record your observations in a table.
• e) What pattern did you find from your
  observations? What factors influenced the
  physical quantities you measured?
• f) Discuss whether your methods of measurement
  were reliable.

32
Design testing experiment

• Design an experiment to test the following rule
  an object always moves in the direction of the
  net force exerted on it. You have a dynamics
  cart, dynamics track, a spring scale, masking
  tape, a bowling ball, a mallet, a small ball and
  a cushion to play with. You can also use any
  other common equipment available in the lab. Feel
  free to use your lab partner as an object!
• State what rule you are going to test.
• Brainstorm the task and make a list of possible
  experiments whose outcome you can predict. Decide
  what experiments are best. Briefly explain why.
• Draw a labeled sketch of the experimental set-up
  and write a brief description of your procedure.
• Draw a free body diagram of the object.
• Make a prediction about the outcome of the
  experiment. Make sure that the prediction is
  based on the rule that you are testing.
• Perform the experiment. Record the outcome. Was
  your prediction confirmed?
• Based on your prediction and the outcome of your
  experiment, can you say that the rule is
  supported or not?
• .

33
Hypothetico-deductive reasoning

Explanation/ Hypothesis Experiment design Predicted outcome Observed outcome Conclusion (hypothesis supported or not)
If and I do this then And I saw/ But I saw Therefore
34

• We assume that the students who are aware that
  they have changed their beliefs and can justify
  this change on grounds such as greater
  explanatory adequacy, should be more capable of
  defending their beliefs from criticism and thus
  their learning should be less fragile. S.
  Vosniadou
• .

35
Design an Investigation to Solve a Problem

• Design two independent experiments to determine
  the width of a strand of your hair. One method
  must involve ideas of diffraction.
• Equipment Laser pointer, meter stick, holder
  for hair, screen, Vernier calipers.
• ? Devise and write an outline of the procedure.
• ? Draw a labeled diagram of your experiment.
• ? Write the mathematical procedure you will use.
• ? Write how you will measure the physical
  quantities you need.
• ? List the assumptions are you making in your
  design. What are possible experimental
  uncertainties? How could you minimize them?
• ? Perform the experiment. Record your
  measurements.
• ? Calculate the thickness based on your procedure
  and measurements. Evaluate the effect of the
  uncertainties.
• ? When finished both experiments, compare the two
  values for the thickness. What are possible
  reasons for the difference?

36
Anomalous Data Tasks

• Use existing physics knowledge (tested and
  accepted) to make a prediction about an outcome
  of a particular experiment
• Explain why you made the prediction.
• Watch the experiment, record the outcome, compare
  to the prediction.
• Revise the explanation or the assumptions if
  necessary.

37
Video Problems

• Observe the two experiments below. Use each to
  determine the .
• Describe how you will use the video to determine
  the necessary quantities.
• List all physics explanations/relationships you
  will use to determine
• List all of the assumptions that you made.
  Describe the mathematical procedure that you will
  use to find the using the measured physical
  quantities.
• Decide whether you have a reasonable agreement
  between the results of the two experiments.
  Evaluate the assumptions and uncertainties.

38
Use of rubrics

• To help instructors write assignments
• To guide students writing lab reports
• To help students with self-assessment
• To help instructors provide feedback to the
  students
• To help instructors grade student work

39
Future talks

• S. Murthy - designing labs and assessing student
  experimental abilities
• D. Rosengrant - designing multiple representation
  tasks and assessing whether students use multiple
  representations while solving multiple choice
  problems on the exams
• A. Warren - designing evaluation tasks and
  assessing whether there is a relationship between
  student evaluation ability and problem solving
  ability.

40
Assess student work
41
Rubric for self assessment
Scientific Ability 0 1 2 3
Is able to construct a different helpful representation (other than mathematical) from a previous representation No attempt is made to construct a different helpful representation. Representations are attempted, but have major flaws. Representations are created without mistakes, but there is some information missing. Different representations are correct
42
Rubric for self assessment
Scientific Ability 0 1 2 3
Is able to extract information from the representation No visible attempt is made to extract information from the representation. Information that is extracted contains errors such as labeling quantities incorrectly. Some of the information is extracted correctly, but not all of the information. All necessary information has been extracted and presented correctly.