Evaluating Writing for Quantitative Reasoning

1
Evaluating Writing for Quantitative Reasoning

• Integrative Learning Project 2005 Summer
  Institute
• Carleton College Scott Bierman, Liz Ciner,
  Jackie Lauer-Glebov, Carol Rutz, Mary Savina
• Emails crutz_at_carleton.edu
• msavina_at_carleton.edu
• sbierman_at_acs.carleton.edu
• eciner_at_acs.carleton.edu
• jlauergl_at_acs.carleton.edu

2
Carletons ILP Project

• Cross-cutting literacies and skills
• Tie in with faculty workload improving
  faculty members knowledge of what everyone else
  is doing.
• Today One piece Connecting quantitative
  literacy and writing

3
Questions for you

• How do you evaluate students writing on your
  campus?
• How do you evaluate students quantitative
  reasoning (or literacy or. . . ) on your campus?
• How might you combine the two?

4
Outline

• Background Quantitative Inquiry, Reasoning, and
  Knowledge (QUIRK) initiative
• Background Writing Portfolio initiative
• Writing about QUIRK First-year seminars
• Developing criteria and reading student writing
• Intersecting the cross-cutting literacies what
  the future may hold at Carleton

5
What is QuIRK?

• Inquiry
• Knowledge
• Reasoning
• http//www.go.carleton.edu/quirkgtwww.go.carleton.e
  du/quirk

6
Background Writing Portfolio Initiative

• May 2001 Faculty vote to institute writing
  portfolio requirement. Volunteers from class of
  2004 submit portfolios in May 2002.
• May 2003 All sophomores in the class of 2005
  submit writing portfolios.
• May 2004 All sophomores in the class of 2006
  submit writing portfolios.

7
The portfolio/QUIRK cycle
Carleton College 2004, FIPSE proposal
8
Background History of QuIRK

• 2001- present faculty meet informally to
  systematically discuss concerns about students
  quantitative literacy.
• May 2003 Background white paper presented to
  Dean and President
• September 2004 FIPSE grant approved .
• September 2004 and continuing First-year
  seminars offered, course development funds for
  faculty, speakers, QuIRK faculty workshops
• June 2005 - QUANT squad formed, portfolio reading
  and rubric revised
• August 2005 workshop featuring College of San
  Mateo learning community Tools for Thought
  (Jean Mach and Mike Burke)
• December 2005 Writing workshop Writing with
  Numbers, John Bean, facilitator
• May 2006 First writing examples from QuIRK
  first-year seminars expected in writing portfolios

9
How can we help students appreciate and
strengthen QR?

• First-year seminars
• QuIRK across the curriculum, examples
• Improve QR in Biology lab reports.
• Develop a data project on the trans-Atlantic
  slave trade for use in a History course.
• Improve a QR assignment in a Writing Course.
• Create a new Political Science course emphasizing
  comparative electoral analyses.
• Faculty development and campus events

10
First-year seminars

• IDSC 100, Measured Thinking Principles of
  Quantitative Reasoning
• POSC 100, Media and Electoral Politics
• SOAN 100, Myths of Crime
• ENTS 100, Geology and Human Health
• Other QR assignments in first-year classes, e.g.
  English 109 (Rutz, Shuffleton)

11
Evaluation of First-Year Seminars (thanks to
Jackie Lauer-Glebov, Carleton Office of
Institutional Research)

• 38 students in QR seminars and 45 control
  students in other FYS seminars did not differ at
  the pretest on seven QR-related questions on
  first-week survey (e.g. I have the skills to
  read and understand a statistical analysis of
  data.)..
• Compared to pretest scores, the posttest
  responses to the QR questions by both QR and
  control students were more positive.
• The QR seminar students had more positive
  responses to the QR questions than the control
  students.

12
Evaluating QuIRK in student writing May/June
2005

• Norming sessions for Quant Squad
• Quant Squad reads 281 portfolios (of about 480
  total), flagging 381 essays with some QuIRK
  content from 102 courses, in 25 departments and
  programs.
• QuIRK taskforce revises criteria, articulates
  goals, attends record-setting Minnesota Twins
  game.

13
What we saw in student writing

• Students are aware of the power of quantitative
  claims and quantitative reasoning.
• Many student do not put numbers in context is
  this number a big one?
• Many students are less specific than they should
  be and they overuse most, many, few,
  seldom.
• Many students had problems interpreting numbers
  and using them to advance arguments.

14
Some faculty development implications

• Articulate the importance of quantitative
  reasoning to all faculty (choice of workshop
  topics).
• Rewrite assignments to encourage students to
  report and interpret the quantitative content of
  their sources.
• Demonstrate what we mean by good interpretation
  of data (e.g. for gall fly papers).
• Highlight excellent assignments in the arts and
  humanities that encourage students to use and
  interpret numbers.

15
June 2005 goals articulation

• Goals
• Thinks quantitatively
• Implements quantitative analysis competently
• Interprets and evaluates thoughtfully
• Communicates effectively

16
Revised criteria

• States questions under consideration in
  numerical/quantitative/measurable terms
• Identifies appropriate quantitative/numerical/empi
  rical evidence to address questions and issues
• Generates, collects, or accesses appropriate
  data
• Investigates questions and issues by selecting
  and carrying out appropriate quantitative or
  numerical methods
• Uses quantitative methods correctly
• Presents and/or reports the quantitative data
  appropriately
• Focuses analysis appropriately on relevant data
• Interprets results to address questions and
  issues under consideration
• Assesses the limitations of the methods employed,
  if appropriate to the task or assignment.

17
Whats next at Carleton? Using student writing
to evaluate. . .

• Visual literacy? (Working group formed January
  2005Visual representations of evidence should
  be governed by the principles of reasoning about
  quantitative evidence. For information displays,
  design reasoning must correspond to scientific
  reasoning. Clear and precise seeing becomes as
  one with clear and precise thinking Edward
  Tufte (1997), Visual Explanations
• Information literacy? (Mellon pilot project
  2001-2004)
• Group work? (ECC review in progress)
• Integrative learning in individual courses?
  (HHMI/CISMI)
• Ethical inquiry/reflection and civic engagement?
• ????

18
Thanks to . . .

• Neil Lutsky, Sam Patterson, Jackie Lauer-Glebov
  and the others who wrote the FIPSE grant proposal
• Carol Rutz
• The QUANT Squad
• Corrine Taylor, Director of the Quantitative
  Reasoning Program at Wellesley College
• Liz Ciner, John Ramsay and the others in the Dean
  of the College office
• Lynn Steen, Randy Richardson and others outside
  Carleton whove helped us