Standards and Rubrics for Assessing Learning Outcomes in Mathematics

1
Standards and Rubrics for Assessing Learning
Outcomes in Mathematics

• GEAR Conference
• April 27th 28th

2
PresentersMaryann Faller Adirondack CCRalph
Bertelle Columbia-Greene CCJack Narayan SUNY
Oswego
3
History

• The SUNY Board of Trustees passed a resolution
  creating three levels of assessment general
  education, assessment of the major and
  system-wide assessment.
• PACGE (Provosts Advisory Council on General
  Education) was formed to provide some guidance to
  the campuses as they submitted the courses they
  wanted to use for general education in
  mathematics.

4

• PACGE developed the Guidelines for the Approval
  of State University General Education Requirement
  Courses which listed the following learning
  outcomes for mathematics.
• Students will show competence in the following
  quantitative reasoning skills
• Arithmetic
• Algebra
• Geometry
• Data analysis and
• Quantitative reasoning

5

• GEAR (General Education Assessment Review) was
  formed to assist campuses in assessing the
  learning outcomes in general education.
• ACGE (Advisory Council on General Education) was
  formed to serve as the judicator for general
  education courses and to review/revise the
  learning outcomes.
• SUNY BoT passes a resolution requiring
  strengthened campus-based assessment in
  mathematics, basic communication and critical
  thinking.

6

• At the request of the mathematics faculty from
  our campuses and the Provost, ACGE revises the
  learning outcomes in mathematics.
• New Learning Outcomes in Mathematics
• Students will demonstrate the ability to
• interpret and draw inferences from mathematical
  models such as formulas, graphs, tables and
  schematics
• represent mathematical information symbolically,
  visually, numerically and verbally
• employ quantitative methods such as, arithmetic,
  algebra, geometry, or statistics to solve
  problems  
• estimate and check mathematical results for
  reasonableness and  
• recognize the limits of mathematical and
  statistical methods.  

7

• There are three options for assessing the
  learning outcomes in mathematics.
• Nationally-normed standardized tests
• SUNY-normed standardized tests
• Using rubrics developed by discipline specific
  panels.
• The discipline panels first met at System
  Administration on February, 2005 to discuss the
  charge and other aspects of writing those rubrics.

8
Members of the Mathematics Discipline Panel

• Maryann Faller Chair, Adirondack CC
• Mel Bienenfeld - Westchester CC
• Ralph Bertelle Columbia Greene CC
• Jack Narayan SUNY Oswego
• Michael Oppedism Onondaga CC
• Robert Rogers SUNY Fredonia
• Malcomb Sherman SUNY Albany
• William Thistleton SUNY IT

9
Procedures Used In Creating a Rubric

• Determine the standard to be assessed.
• Write learning objectives for that standard.
• Determine the style and scale that will be used.
• Describe criteria for the highest and the lowest
  levels
• Describe the criteria for the levels in between
  the highest and lowest.

10
Our rubric

• After much discussion, the panel decided that we
  will have a matrix with 2 columns and 4 rows.
  The rows will represent the levels of assessment.
  They are
• 3 Exemplary
• 2 Generally Correct
• 1 Partially Correct
• 0 Incorrect

11

• The panel decided to rate the students response
  with respect to the following criteria

• Does the student understand the problem?

• Does the student use a clearly developed logical
  plan to solve the problem and is that plan
  evident in the solution?

• Is the solution totally correct?

12
Learning Outcome 1

• Standard Students will demonstrate the ability
  to interpret and draw inferences from
  mathematical models such as formulas,
  graphs, tables and schematics .
• Learning Objectives
• Given a mathematical model, the student will be
  able to
• Interpret the information
• Draw inferences from that model

13
Level
3 Exemplary The student is able to interpret the significance of the values and/or variables given in the model. The student has an understanding of how to use the model to answer the question. The question is answered accurately and completely.
2 Generally Correct The student is able to interpret the significance of the values and/or variables given in the model. The student has an understanding of how to use the model to answer the question. The question is not answered accurately or completely.
DRAFT
14
Level
1 Partially Correct The student has some misunderstanding of how the model relates to the situation. The student attempts to use the model to answer the question but lacks a clear understanding of how to carry that out. The question is not answered accurately or completely.
0 Incorrect Solution The student does not understand how the model relates to the situation. The student does not understand how to use the model to answer the question. The question is not answered accurately or completely.
DRAFT
15
Learning Outcome 2

• Standard Students will demonstrate the ability
  to represent mathematical information
  symbolically, visually, numerically and verbally
  .
• Learning Objectives
• Given mathematical information, the student will
  be able to
• Represent that information symbolically
• Represent that information visually
• Represent that information numerically
• Represent that information verbally

16
Level
3 Exemplary The student fully understands the mathematical information and the mode of representation. The student understands all required aspects of the representation and clearly demonstrates the knowledge of how to develop it. The representation of the given information is correct and accurate. It is displayed using the correct format, mathematical terminology, and/or language. Variables are clearly defined, graphs are correctly labeled and scaled, and the representation is otherwise complete as required.
DRAFT
17
Level
2 Generally Correct The student understands the essentials of the mathematical information and the required representation(s). The student understands most of the specific aspects of the representation and demonstrates the knowledge of how to develop it from the given information. This understanding or demonstration is lacking in a minor way. A misrepresentation of the mathematical information was given due to a minor computational/ copying error or the representation was not labeled or labeled incorrectly. The representation is incomplete in some minor way. .
DRAFT
18
Level
1 Partially Correct The student does not fully understand the mathematical information or the mode of representation, but some understanding is shown. The student shows some knowledge of how to develop the appropriate representation, but this knowledge is incomplete in a major way. The representation(s) show some reasonable relation to the information, but they contain major flaws, use incorrect format, mathematical terminology or language. The representation is incomplete in a major way.
DRAFT
19
Level
0 Incorrect Solution The student does not understand the mathematical information or the required representation(s). Complete misinterpretation of the problem. The student could not represent the information in any format other than the format in which the information was given. The representation(s) are incomprehensible or unrelated to the given information. The process of developing the representation is entirely incorrect. The students response does not address the question in any meaningful way or there is no response at all.
DRAFT
20
Learning Outcome 3

• Standard Students will demonstrate the ability
  to employ quantitative methods such as,
  arithmetic, algebra, geometry, or statistics to
  solve problems .
• Learning Objectives
• Given a problem, the student will be able to
• Identify the appropriate quantitative method(s)
  necessary to solve that problem.
• Use those methods to correctly solve that problem.

21
Level
3 Exemplary The student correctly understands the specific numeric, algebraic, geometric, or statistical method or equation that is needed to solve the problem. The student completes the process or solves the equation and arrives at an accurate and complete solution of the problem.
2 Generally Correct The student shows a general understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem. The student completes the process or solves the equation in a generally correct way, but with a minor flaw.
DRAFT
22
Level
1 Partially Correct The student shows only a slight understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem. The student makes an attempt at a process or equation that will solve the problem, but in a way that has very little correlation with the correct solution Some minor portion of the students overall solution is completed correctly.
0 Incorrect Solution The student shows absolutely no understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem. Little or no work is shown that in any way relates to the correct solution of the problem
DRAFT
23
Learning Outcome 4

• Standard Students will demonstrate the ability
  to estimate and check mathematical results for
  reasonableness .
• Learning Objectives
• Given a mathematical problem, the student will be
  able to
• Estimate the result of that problem
• Determine and justify the reasonableness of that
  result given the constraints of the problem.

24
Level
3 Exemplary The student can estimate and justify a mathematical result to a problem. The students justification is developed and the estimate has been found using a clearly defined, logical plan The students response is complete and accurate.
2 Generally Correct The student can estimate and justify a mathematical result to a problem but the estimate or justification contains a minor flaw The students justification is developed and the estimate has been found was lacking in some minor way The students response addresses all aspects of the question but is lacking in some minor way.
DRAFT
25
Level
1 Partially Correct The student can estimate and justify a mathematical result to a problem but the estimate or justification contains a major flaw. The students justification is not developed and the estimate has been found was lacking in some major way The students response addresses some aspect of the question correctly but is lacking in a significant way.
0 Incorrect Solution The student cannot estimate and/or justify a mathematical result to a problem. The students justification is not supported by any logic plan. The students response does not address the question in any meaningful way or there is no response at all.
DRAFT
26
Learning Outcome 5

• Standard Students will demonstrate the ability
  to recognize the limits of mathematical and
  statistical methods .
• Learning Objectives
• Given mathematical method, the student will be
  able to identify and articulate the limits of
  that mathematical method.

27
Level
3 Exemplary   Student indicates the assumptions/simplifications made in developing a mathematical/statistical model  Student provides an accurate description how the results from the model might diverge from the real life situation it models Student indicates alternative assumptions/models which might be reasonable replacements for those that were used
2 Generally Correct   Student indicates the assumptions/simplifications made in developing a mathematical/statistical model Student provides an accurate description of how the results from the model might diverge from the real life situation it models Student does not indicate alternative assumptions/models which might be reasonable replacements for those that were used
DRAFT
28
Level
1 Partially Correct   Student indicates only some of the assumptions/simplifications made in developing a mathematical/statistical model   Student realizes that some of the results of the model diverge from real life but is unable to articulate these differences. Student does not indicate alternative assumptions/models which might be reasonable replacements for those that were used
0 Incorrect Solution  Student does not indicate any assumptions/simplifications made in developing a mathematical/statistical model  Student fails to realize that the results of the model may diverge from real life Student does not indicate alternative assumptions/models which might be reasonable replacements for those that were used he student does not understand how the model relates to the situation.
DRAFT
29
An example   Suppose that you invest
500.00 in an account and that interest is
compounded continuously according to the
formula   
1. If your annual rate of return is
4, a.       How much money will you have at the
end of 10 years? b.       How long will it take
your money to double? 2. What rate of return do
you need in order for your money to double every
5 years?
30
Level 3The student writes .Substituting
correctly for t demonstrates that the student
understands how to use the model to answer the
question.The student writes .Substituting
correctly for P and r demonstrates that the
student is able to interpret the significance of
those variables given in the model .  The
student writes The balance in the account at the
end of 10 years is 745.91.This is a complete
and accurate answer.
31
Level 2The student writes .Substituting
correctly for P and r demonstrates that the
student is able to interpret the significance of
those variables given in the model . The
student writes .Substituting correctly for t
demonstrates that the student understands how to
use the model to answer the question. The
student writes The balance in the account at the
end of 10 years is 5204.05.This is a
computational error involving order of
operations. It is not unusual for a student not
to question a result like this.
32
Level 1The student writes .Substituting
incorrectly for P or r demonstrates that the
student has some misunderstanding of how the
model relates to the situation. The student
writes .The student attempts to use the model
to answer the question.  The student writes
The balance in the account at the end of 10 years
is 1.1769E20.This is the calculator display,
which is meaningless in this situation.
33
Level 0The question is left blank or whatever
is written is meaningless.
34
Questions and Answers

• Do campuses have to assess all of their courses?
• What do they do in cases where some of the
  learning outcomes are not covered in the courses?
• Can I write my own assessment?
• Can the same rubric be used for all courses?

35
Questions and Answers

• Is there money for folks at campuses to construct
  the rubrics?
• What are the learning outcomes?
• How were the learning outcomes created?
• What is a rubric?
• What is a standard?
• Are mathematicians using rubrics?
• Is there a rubric for each learning outcome?
• Does a campus have to use the same exam for all
  courses?

36
Questions and Answers

• Can a campus use pre- and post-tests?
• What happens to the data when the system gets it?
• Can reporters access the data?
• What is the process for a campus to get its
  assessment plan approved by GEAR? What is the
  time line?
• How are testing and assessment related?