Assessing Writing and Thinking Skills

1

• Assessing Writing and Thinking Skills
• with the Cognitive Level And Quality
• of Writing Assessment and other Rubrics

Mississippi University for Women April 7, 2008
Columbus, Mississippi
T.L. Flateby, Ph.D. Director, Academic
Assessment University of South Florida
2
Critical Thinking

• General Definitions (Bloom and colleagues, Enis,
  Halpern, Facione and Facione)
• Higher-order thinking skills
• Problem solving
• Making Judgments

3
More Specific Definitions

• Diane Halpern
• the thinking involved in solving problems, making
  inferences, determining likelihoods and making
  decisions in a variety of settings (1999)
• Delphi Report (American Philosophical
  Association, 1990)

4
Joanne Kurfiss

• an investigation whose purpose is to explore a
  situation, phenomena, question or problem to
  arrive at a hypothesis or conclusion about it
  that integrates all available information and
  that can therefore be convincingly justified
  (1988, p. 2).

5
Further

• Specific reasoning skills are emphasized in
  different disciplines
• Chemists drawing important inferences from
  results, analyzing and evaluating research and
  developing new questions for investigation
• Computer science faculty breaking down complex
  problems or situations into simpler ones
• English faculty elaborating an argument and
  developing its implications (Powers D., and
  Enright, M , 1987, p. 658-682)

6
Define Critical Thinking for your purposes

• Select or develop rubrics to measure your
  definition
• To develop choose criteria, levels of
  achievement, and define levels for each criterion
  according to levels
• Develop assignments to elicit critical thinking

7
To Foster the Conversation about writing and
thinking expectations

• Across disciplines
• Between faculty and students
• Between student and student

8
One available rubric Cognitive Level and Quality
of Writing Assessment (CLAQWA)

• Developed
• In multi-disciplinary teams for use by faculty
  whose discipline is not English
• to provide consistent assessment of students
  writing and thinking
• to attend to cognitive level expected in
  assignments
• To be flexible
• identify skills appropriate for the assignment
• provide different weighting for skills

9
Cognitive Level Quality of Writing Assessment
(CLAQWA) Scale
10
Cognitive Level
11
Branching of Blooms Taxonomy of Educational
Objectives Cognitive Domain
Analysis
Evaluation/Synthesis
Application
Comprehension
Knowledge
Based on Madaus, 1973.
12
Assignments from Professionalismand Ethics in
Engineering

• 1 Summarize the 7 fundamental Canons of Ethics
  from the American Society of Civil Engineers
  Code of Ethics.
• 2 Which of the 7 canons might be applicable to
  student honesty at USF.
• 3 Given the following ethical situation
  (engineers discovers a defect in a product,
  tells boss, boss tells employee to disregard
  the defect), what action should the engineer
  take based on the ASCE code of ethics.
• 4 Develop an Honors Code for USF engineering
  students. Consider the ASCE code of ethics,
  common morality, and the theories of
  Utilitarianism and Individual Rights.

13
Verbs for Constructing Prompts
This list suggest verbs intended to guide the
development of the prompt. Using these words in
writing prompts will help communicate the
expected cognitive level to be achieved by
students.Level 4 Analysis, Synthesis,
Evaluationdiscriminate plan weighinfer orga
nize evaluatecompare generate combinecontr
ast appraise concludecreate critique supp
ortdesign judgeLevel 3 Applicationdetermine
use an approach apply a principlechart devel
op solve a problemimplement choose an
appropriate relateprepare procedure demonstrat
eLevel 2 Comprehensiondescribe estimate il
lustrategeneralize classify give an example
ofparaphrase explain state in your own
wordssummarize predict translateLevel 1
Knowledgedefine list nameidentify match
reportlabel recall selectstate recite
14
Trait Coherence Devices

• Level 5 Transitional words, phrases, sentences
  and paragraphs (coherence devices) smoothly
  connect the papers elements, ideas and/or
  details allowing the reader to follow the
  writers points effortlessly.
• Level 4 Coherence devices are rarely missing
  and do not impact the readers understanding.
• Level 3 Coherence devices appear throughout the
  paper, but additional and appropriate connectors
  would enhance the flow.
• Level 2 Coherence devices are attempted, but
  are ineffective.
• Level 1 Coherence devices are absent or
  inappropriate.
• http//www.usf.edu/assessment

15
Analytic WritingSkills Elements Ranked

• Sentence construction varies appropriately.
• Point of view is consistent.
• A main idea is present and maintained.
• Ideas are comprehensible.
• All assignment requirements are fulfilled.
• Appropriate audience(s) are consistently
  addressed.
• Purpose is clear and specific.
• Details are sufficient in quantity to develop
  main idea.
• Word choice is accurate.
• Grammar and mechanics.
• Opening supports the main idea.
• Coherence devices are present and appropriate.
• Details are sufficient in quality to develop main
  idea.
• Paragraphs support unity.
• Reasoning supports the main idea.
• Closing supports the main idea.

16
Cognitive Level Distribution
17
Peer Review

• Based on Assessments findings USF is developing
  a University-wide solution to writing and
  thinking weaknesses