Mastering Assessment

1
Mastering Assessment

• Todays agenda
• What a template! Or, What? A template?  
• Identifying good and bad measures. 
• Linking assessment measures with results.  
• Analyzing results to make program improvements.
  
• A Quiz!

2
Terminology

• Measure, Learning Outcome, Goal, program goal. .
  . Whatever term you use, make sure its
  measurable!
• FYI This presentation is on our assessment
  conference website at http//www.ccaurora.edu/aac
  onference/masteringassessment.shtml

3
A Quote from ABC Community College. . .

• . . . We have administrative support for
  assessment, we have a budget, we have release
  time for assessment, have participated in HLC
  assessment training workshops, and have two
  consultant evaluators on staff at our college.
• . . . We have faculty leadership and buy in for
  assessment. We have faculty development, we have
  a well balanced assessment committee, we have a
  plan, So . . .
• WHY DID WE GET A FOCUS VISIT?

4
HLC teams use six questions and others as
appropriate to prompt conversations on student
learning

• How are the organizations stated student
  learning outcomes appropriate to its mission,
  programs, and degrees?
• ABCs problem (in my opinion)
• Their learning outcomes were not visible.
• Their outcomes were not measurable.

5
HLC teams use six questions and others as
appropriate to prompt conversations on student
learning

• 2. What evidence does the organization have that
  students achieve the stated learning outcomes?
• ABCs Problem
• If your learning outcomes are not measurable,
  this question can not be answered.

6
HLC questions 3 6

• 3. In what ways does the organization analyze and
  use evidence of student learning?
• 4. How does the organization ensure shared
  responsibility for assessment of student
  learning?
• 5. How does the organization evaluate and improve
  the effectiveness of its efforts to assess and
  improve student learning?
• 6. In what ways do you inform the public and
  other stakeholders about what and how well your
  students are learning?

7
One Size Fits All Template

• Clarifies are you assessing what you should be?
  
• Identifies a measurable measure.
• Clearly links results to the measure.
• Summarizes what you learned in the process.
• Does all this while addressing those first two
  important HLC questions.
• Examples www.ccaurora.edu/assessment

8
Weak Measures

• Final papers from a variety of sections will be
  graded and analyzed. The English assessment team
  will adjust curriculum as needed.
• Students will improve 10 on the post test over
  the pre-test.
• We are using pre- and post-testing, assessment
  tests, scenario based studies and critical
  inquiry questions and problems. We also measure
  student learning by success on a state required
  examination.

9
Good Measures

• Students will pass the post test with a 70 or
  better. . .
• 70 of students who pass Mat 106 will receive a
  score of 85 or better on the Accuplacer exam.
• Students completing a critical thinking
  assignment will achieve an average score of 2.0
  on the critical thinking rubric.
• Student pass rates will increase 10 from fall
  2005 in AAA 101.

10
Measure Results Mismatch

• Measure Argumentative essays were collected from
  sections of ENG 121 at the end of fall semester.
  From those collected, 20 were randomly selected
  for review. We then compared the data compiled
  with previous years data to assess progress.
• Results CAT team members looked at and analyzed
  the data. From this, CAT members have made
  recommendations for 2007-08 and will disseminate
  this information to faculty at the all-college
  meeting in the fall, as well as through email.

11
Measure-Results Mismatch (barely)

• Measure Using ten randomly selected videos, 80
  of students would earn a satisfactory or
  excellent rating (2 or better) on the Persuasive
  Speech Rubric
• Results The outcome of the raters scores
  revealed the following rubric data
• Presentation Aids 0.42
• Delivery 2.06
• Conclusion 1.67
• Body 2.08 and Introduction 2.23

12
Excellent Measure-Results Matching

• Measure 70 of students who pass Mat 106 will
  receive a score of 85 or better on the Accuplacer
  exam. Results 61.95 of the students scored at
  85 or higher on the Accuplacer.
• Measure Students completing a critical thinking
  assignment will achieve an average score of 2.0
  on the critical thinking rubric. Results For
  Spring, 2007, the average score for Psy 101/102
  papers is 1.83. Average scores for Ant papers is
  1.53.
• Measure 100 of students expected to demonstrate
  critical inquiry skills at meets criteria or
  above in written assignments. Results 100 of
  students were able to demonstrate critical
  inquiry skills at meets criteria or above in
  their final case briefing.

13
Program Improvement As A Result of Assessment

• See information in the last box of the template
  at
• http//www.ccaurora.edu/assessment/DepartmentalPla
  n2006-7.shtml
• For flow from 2006-2007 to 2007-2008, see these
  program plans
• AAA 101, English Composition, Speech,
  Developmental English . . .

14
Create an Assessment Plan for Today

• What are we measuring
• Measurable Learning outcome
• How? The participants will be tested via quiz at
  the end of the section.
• Results?
• What did I learn?

15
Assessment Quiz

• 1. Which is the best measure?
• Finals will be graded and a team of faculty will
  interpret results to see what changes need to be
  made in the curriculum or teaching.
• 80 of student written lab reports will achieve a
  score of 2 or better on the communication rubric.
  
• 2. Which is the best measure-results match?
• Measure 75 of students will achieve a minimum
  score of 80 on the post test. Results student
  post test scores revealed a deficiency in ECG
  interpretation.
• Measure 75 of students will achieve a minimum
  score of 80 on the post test. Results students
  averaged 65 on the post test.
• Measure 75 of students will achieve a minimum
  score of 80 on the post test. Results 75 of
  students scored 73 or better on the post test.

16
Assessment Quiz Continued

• 3. Which assessment plan handout is better
• Psychology or
• CIS

17
Thank You!

• Greg Goode, VP Student Services, CCA
• CCA Assessment Committee Co-Chair
• HLC Consultant Evaluator
• Do you need help with assessment?
• 1. Visit our website for assistance
  http//www.ccaurora.edu/assessment/
• 2. Contact me at greg.goode_at_ccaurora.edu