MATH ASSESSMENT TEST

1
MATH ASSESSMENT TEST

• OCMA
• May, 2008

2
HISTORY OF MAT

• Test originally developed in late 60s

3
HISTORY OF MAT

• Test originally developed in late 60s
• Original purpose was to verify foreign
  credentials of applicants

4
HISTORY OF MAT

• Test originally developed in late 60s
• Original purpose was to verify foreign
  credentials of applicants
• Name changed by 1970 to Math Inventory test and
  given to all students taking first semester math

5
HISTORY OF MAT

• Test originally developed in late 60s
• Original purpose was to verify foreign
  credentials of applicants
• Name changed by 1970 to Math Inventory test and
  given to all students taking first semester math
• Purpose at this time was to collect data on
  students to determine their math skill level

6
HISTORY OF MAT

• Test originally developed in late 60s
• Original purpose was to verify foreign
  credentials of applicants
• Name changed by 1970 to Math Inventory test and
  given to all students taking first semester math
• Purpose at this time was to collect data on
  students to determine their math skill level
• Math Learning Centre developed in 1973 and MAT
  now had a remedial component

7
HISTORY OF MAT

• Test originally developed in late 60s
• Original purpose was to verify foreign
  credentials of applicants
• Name changed by 1970 to Math Inventory test and
  given to all students taking first semester math
• Purpose at this time was to collect data on
  students to determine their math skill level
• Math Learning Centre developed in 1973 and MAT
  now had a remedial component
• Remedial modules were further developed during
  the mid 70s

8
ORIGINAL TEST

• Modular multiple choice test
• Students given overall grade on the test any
  modules that were weak
• Remedial work assigned only on those topics that
  were weak
• Work completed through the Math Learning Centre
  during the semester

9
Past Present

• Diagnostic test with a remedial component

• Push to use test as an assessment tool for
  streaming

10
Past Present

• Diagnostic test with a remedial component
• Students had higher level math at entry

• Push to use test as an assessment tool for
  streaming
• Student coming in with more varied math
  background

11
Past Present

• Diagnostic test with a remedial component
• Students had higher level math at entry
• Students scored higher on average on the MAT

• Push to use test as an assessment tool for
  streaming
• Student coming in with more varied math
  background
• Students scoring lower on the MAT

12
Original Business and Technology Versions

• Business
• Fractions
• Decimals
• Percents
• Order of Operations and Law of Signs
• Equations

13
Original Technology and Business Versions

• Business
• Fractions
• Decimals
• Percents
• Order of Operations and Law of Signs
• Equations

• Technology
• Fractions
• Decimals
• Percents
• Order of Operations
• Laws of Signs
• Exponents and Radicals
• Algebraic Fractions
• Equations and Formulas
• Mensuration
• Trigonometry and Geometry
• Logarithms

14
Current Computerized Version

• Fractions
• Decimals
• Percents
• Order of Operations
• Laws of Signs
• Exponents
• Basic Algebra
• Equations
• Radicals
• Algebraic Fractions
• Mensuration
• Trigonometry and Geometry
• Logarithms

• It is possible to select different modules by
  the program of the students.

15
Current Computerized Version

• Fractions
• Decimals
• Percents
• Order of Operations
• Laws of Signs
• Exponents
• Basic Algebra
• Equations
• Radicals
• Algebraic Fractions
• Mensuration
• Trigonometry and Geometry
• Logarithms

• It is possible to select different modules by
  the program of the students.
• FALL 2007
• Business Students did only
• Topics 1-5
• Technology students did Topics 1-10, as research
  showed these were the critical modules.

16
Current Computerized Version

• Fractions
• Decimals
• Percents
• Order of Operations
• Laws of Signs
• Exponents
• Basic Algebra
• Equations
• Radicals
• Algebraic Fractions
• Mensuration
• Trigonometry and Geometry
• Logarithms

• It is possible to select different modules by
  the program of the students.
• FALL 2007
• Business Students did only
• Topics 1-5
• Technology students did Topics 1-10, as research
  showed these were the critical modules. (54
  questions)
• FALL 2008
• Business Students will add
• Topics 7 8 (40 questions)

17
PAST AND PRESENT

• PAST
• Written on first day of class

• PRESENT
• Students write prior to semester start
  available from Feb.

18
PAST AND PRESENT

• PAST
• Written on first day of class
• Pencil and Paper

• PRESENT
• Students write prior to semester start
  available from Feb.
• Computerized test completed in a proctored site.
  (Practice tests available online with built in
  remedial.)

19
PAST AND PRESENT

• PAST
• Written on first day of class
• Pencil and Paper
• Results given in weeks 2 or 3

• PRESENT
• Students write prior to semester start
  available from Feb.
• Computerized test completed in a proctored site.
  (Practice tests available online with built in
  remedial.) Results given on exit from test
• Results given on exit from test

20
PAST AND PRESENT

• PAST
• Written on first day of class
• Pencil and Paper
• Results given weeks 2 3
• Remedial done during semester through the MLC

• PRESENT
• Students write prior to semester start
  available from Feb. Practice tests available
  online with built in remedial
• Computerized test completed in a proctored site.
  (Practice tests available online with built in
  remedial.)
• Results given on exit from test
• Remedial completed online prior to September 30.

21
Current Research

• The push to change the use of the assessment test
  for streaming meant that the test needed to be
  tested.
• We are currently doing
• Item Analysis on each question in the test (new)
• Validity of the test as a predictor of success in
  semester 1 college math (this is ongoing from
  previous years)

22
Item Analysis of Test(Creating good questions in
a multiple choice test)

• The incorrect options, distractors, must be a
  clue to the nature of the misunderstanding and
  thus prescriptive of appropriate remediation
• Each question must discriminate correctly between
  the top 27 and the bottom 27 of students.

23
Validity

• Test validity refers to the degree to which the
  test actually measures what it claims to measure,
  in this case, success in first semester college
  math courses.
• It is also the extent to which inferences,
  conclusions, and decisions made on the basis of
  test scores are appropriate and meaningful.

24
ITEM ANALYSIS DISTRACATORS
Green is correct answer
The good and the bad!
25
Discrimination of Question 7
Green is correct answer.
26
Question 7

• Each distractor for this question represented a
  common error in dealing with mixed fractions
• The question discriminates correctly between the
  top 27 of students and the bottom 27 of
  students

27
Question 8

• This question did not give much useful
  information since over 90 of students answered
  correctly (index of difficulty is low)
• The distractors were reviewed and changed to
  reflect more typical mistakes that students make
  when multiplying fractions

28
Item Analysis

• We are
• Reviewing each question in this manner to ensure
  that each question adds to our knowledge of any
  weaknesses students may have i.e. it has
  appropriate distractors

29
Item Analysis

• We are
• Reviewing each question in this manner to ensure
  that each question adds to our knowledge of any
  weaknesses students may have i.e. it has
  appropriate distractors
• Reviewing the discrimination of each question to
  ensure that the question is not misleading
  (discrimination top 27 should have a higher
  percent correct than bottom 27)

30
Item Analysis

• We are
• Reviewing each question in this manner to ensure
  that each question adds to our knowledge of any
  weaknesses students may have i.e. it has
  appropriate distractors
• Reviewing the discrimination of each question to
  ensure that the question is not misleading
  (discrimination top 27 should have a higher
  percent correct than bottom 27)
• Changing our remedial to give specific mini
  lessons based on the wrong answer the student
  selects. (Currently it is a more general remedial
  on a topic such as adding fractions.)

31
Validity of Test for Placement

• This test was designed as a diagnostic tool to
  assess areas of weakness and provide remedial
  assistance.
• Over time we have shown that there is a positive
  linear relationship between the test score on our
  assessment and success in first year college
  mathematics
• BUT is it strong enough i.e. does it predict well
  enough to use as a placement test?

32
2004 Research Questions

• Does attendance affect performance in first
  semester math?
• Are basic math skills on entry to the college a
  factor in determining student success?

33
Attendance
Passed Low Attendance Independent Learners
Passed High Attendance Passers
Failed Low Attendance Failures
Failed High Attendance Heart Breakers
Correlation between final grade and attendance
was statistically significant . (r0.467,
p0.000)
34
One-way ANOVA ASSESSMENT versus Quadrant Source
DF SS MS F
P Quadrant 3 54369 18123 69.33
0.000 Error 1148 300084 261 Total
1151 354453
35
MAT AS A PREDICTOR OF SUCCESS IN MA101 MAT
CONTINUES TO BE A SIGNIFICANT PREDICTOR OF
SUCCESS IN 2007
Correlation coefficient r0.43 and p0.000
36
Validity of Test for Placement

• What does that mean to students, if they are
  streamed into an extra math course using this
  test?

37
MAT AS A PLACEMENT TEST(extra math course)
1 This is the group we are trying to help. They
do not have the basic math skills required to
pass the regular first semester math course.
38
AS AN ASSESSMENT TEST
39
COST OF ERRORS TO STUDENTS OR COLLEGE
40
? QUESTIONS