IITE Professional Development Course

1
Module 6 Assessment

• IITE Professional Development Course
• Lucknow University (6/4/2010)
• Professor Tim Keirn
• timkeirn_at_csulb.edu

2
A Review Standards-Based Approaches and Learning
Outcomes

• Programme learning outcomes
• Course learning outcomes
• Program curricular map w/ sequenced papers for
  two certifications
• Physical Science Teacher Ed, Chemistry,
  Mathematics and Physics
• Biology/Life Science Teacher Ed, Chemistry,
  Botany and Zoology

3
Review Continued

• Learning outcomes for each paper
• Design an example of a lesson within a paper that
  is
• Inquiry-based
• Aligned to a paper specific learning outcome
• Engages students with materials from the web

4
Review Continued

• Design an assessment that is aligned to the
  inquiry-based lesson and the specified paper
  learning outcome
• Design a rubric for the aforementioned specific
  assessment
• Publish materials to the portal on the web

5
General Introduction to Assessment

• Do students learn what faculty believe they are
  teaching? How do you know? On what evidence do
  you substantiate your claims?
• As an employer of a candidate with an upper
  second B.Sc from Lucknow in e.g. Botany -- what
  do I know they know and what do I know they
  can do?
• What more do they know and what more can they do
  than someone with a lower second and compared
  to some one with a first?

6
Introduction to Assessment

• Can I assume that someone who did the same paper
  with Vivek knows and can do the same as a
  student of Nalini?
• If so -- how can you substantiate these claims?
• Think-Pair-Share Strategy Identify and discuss
  the origins of three weaknesses in the current
  means by which students are assessed at Lucknow
  University
• This may not be an exhaustive list!

7
3 Weakness of Current Assessment
Description Impact of Weakness
Exams testing factual knowledge and asked to reproduce knowledge The exams are the same each year responding without a deeper understanding of the concepts not training/ developing skills
Evaluation of the exam is effected by the readers mood, quality of other papers Unreliable evaluation
Evaluation is not continuous and comprehensive, reliable and valid Students and employers dont have a reliable confidence in what a student could actually do

8
Traditional Assessment

• Traditional assessment is inseparable from
  traditional modes of teaching and learning
• PH.D. provides discretion as to what is taught
• Stand and deliver
• Design assessment to measure knowledge retention
• Assign marks based on the volume of knowledge
  retained
• PH.D provides discretionary authority to assess
  the volume itself

9
Problems with Traditional Assessment

• Serve to discriminate between students as opposed
  to demonstrating competencies
• Almost always measures the reproduction of
  factual knowledge
• Little if any variance in both the method of
  assessment and the modality of learning
• Assessment is never deployed as a learning tool
• The secret handshake
• Blame the learner, not the teacher

10
Problems with Traditional Assessment (Cont)

• Assessment is infrequent and heavily weighted
  (high stakes)
• Summative over formative assessment
• Limited measurement of teaching efficacy
• Did the instructor get the content across?
• Did the students read and remember the book?

11
Alternative Forms of Assessment

• Standards-, disciplinary- and inquiry-based
  approaches to teaching and learning require a
  different approach to assessment
• Seek to measure
• Thinking and skill gt factual retention
• Production and application of knowledge gt
  reproduction of knowledge
• What is learned (aligned to SLO) gt What is taught

12
Alternative Assessment (Cont)

• Standards-based assessments
• Are designed to measure task competence and
  degrees of proficiency gt ranking and
  discriminating between students
• Are done in multiple forms to measure multiple
  modalities of learning
• Are learning tools in support of instruction and
  are transparent to students
• Are on-going and used to support reflection and
  improvement in teaching practice

13
Alternative Assessment Practicum

• In disciplinary groups -- design a draft of both
  a formative and summative assessment aligned to
  specific student outcome from a paper in the
  programme
• Specify the SLO
• Discuss what dimensions of a task are
  specifically measured in your standards-based
  assessments

14
SLO
Demonstrate





15
Different Forms of Assessment and Methodologies

• Formative Assessments
• Aligned to learning outcome and to summative
  assessment
• Should provide appropriate feedback to student in
  preparation for the summative assessment
• Provide appropriate feedback to instructor about
  the efficacy of the pedagogic methodology
• Monitoring for comprehension in lecture
• Think-pair-share
• Short prompts

16
Other types of formative assessment

• Multiple-choice quizzes
• Short exercises and prompts
• Meeting the challenge of marking
• Be specific about nature of feedback and limits
  of time
• Peer evaluation
• Rubrics

17
Multiple Choice Questions

• Design questions that assess thinking and skill gt
  factual content
• Blooms taxonomy
• Develop justified multiple choice questions
  that demonstrate thinking and process
• Develop distracters that demonstrate identify
  student (mis)understandings
• Questions that task students to substantiate or
  challenge claims

18
Blooms Taxonomy

• Blooms pyramid and active verbs
• Recall (list)
• Application (show)
• Analysis (compare)
• Synthesis (predict)
• Evaluation (dispute and/or substantiate)

19
Authentic Assessment

• Performance assessments tied to authentic
  disciplinary-tasks -- students produce knowledge
  as opposed to reproducing knowledge
• Laboratory practicum
• Research projects
• Assessment constructed as a problem
• Evaluating the validity of different
  interpretations and conclusions and their
  evidentiary basis
• Counterfactual questions and prompts

20
Rubrics - A Scoring Guide that Provides Criteria
to Describe Levels of Student Performance

• The advantages of using rubrics
• Instructors marks more accurately, reliably and
  quickly
• Requires greater accuracy about the criteria of
  student performance
• Serves as a learning tool and provides better
  feedback to students and makes the standard of
  performance explicit
• Creates better reliability across sections

21
Challenges to Using Rubrics

• Initially time-consuming (but in long-run saves
  time)
• Difficulty to find exact language that
  distinguishes between levels of performance and
  establishes criteria
• May require revision in initial implementation

22
Rubric Practicum

• Identify the dimensions of competence in the task
  that can be both delineated and demonstrated in
  the student performance (aligned with SLO)
• Holistic versus analytic (and the advantages of
  the latter within limits)
• Weight and scale the dimensions within the task

23
Rubric Practicum Cont.

• Establish criteria for competent performance of
  each specified dimension of the task
• Establish a scale of criteria performance
• How many clearly identifiable scales? E.g.,
• Competent and Not Competent
• Not Proficient, Proficient, Excellent
• Not Proficient, Developing, Proficient, Beyond
  Proficient, Exemplary
• of scales needs to be justified by clearly
  delineated performances of each dimension of the
  task

24
Rubric Practicum Continued

• The ideal process
• Create draft of rubric
• Implement and refine with evaluation of samples
  of student work
• Calibrate with other faculty
• Mark!

25
Rubric Exercise

• In disciplinary groups -- create a draft rubric
  for a laboratory practicum with three scales of
  performance for each dimension
• Teacher education faculty -- to do the same but
  for a pre-service teachers design of a
  laboratory practicum

26
SLO Laboratory Practicum
Dimensions Criteria
Not Proficient Proficient/Baseline skills Exemplary
Lab preparation DESCRIPTION DESCRIPTION DESCRIPT
Execution of methodology