Archbold FIRST 2 Workshop

1
Archbold FIRST 2 Workshop
David McConnell Center for Collaboration
Inquiry and Department of Geology The University
of Akron September 2003
McConnell, Archbold FIRST 2 , September 2003
2
What is a personal response system?
Components of a PRS
Power supply
Display
Receiver
Software
Transmitters
available from www.educue.com
Lets give it a test drive . . .
3
What is a personal response system?
Imagine a beautiful day. You feel good. You are
driving alone in your car. A favorite song comes
on the radio. Do you sing along?

• Yes
• No
• Maybe

4
Part 1 Student Reasoning Skills in General
Education Classes Could we know any less about
this?
Part 2 The Tourist, the Gunslinger, and the
Gardener Rethinking Metaphors of Teaching and
Learning to Enhance Student Reasoning
Part 3 Blooms Taxonomy A Framework for
Teaching and Learning
5
Which one of these goals is the most important
in general education courses?
1. Higher Order Thinking develop analytical,
problem solving skills, creative thinking 2.
Basic Academic Success improve listening,
reading, writing, speaking, math skills 3.
Discipline-Specific Knowledge learn facts,
concepts, understand importance of discipline 4.
Liberal Arts, Academic Values develop an
appreciation of cultures, social issues,
ethics 5. Work and Career Preparation develop
leadership, time management skills 6. Personal
Development develop commitment to values,
improve self-confidence
6
What are our teaching/learning goals?
Teaching Goals Inventory Angelo Cross,
Classroom Assessment Techniques (1993)
Four-Year College/University n 951
Community Colleges n 1873
3.09/3.05
2.83/2.86
2.5/2.27
2.41/2.28
2.29/2.12
2.02/2.16
TGI is available on-line at http//www.uiowa.edu/
centeach/tgi/index.html
7
How can we assess thinking skills?

• Specifically, can we determine if
• Students are prepared to use higher-order
  thinking skills?
• Students can improve their higher-order thinking
  skills during a general education course?

8
How can we assess thinking skills?

• Problem
• How do we track changes to a students ability to
  reason using higher-order thinking skills?

• Solution
• Group Assessment of Logical Thinking (Roadrangka
  et al., 1982)
• 12 question instrument tests six logical
  operations

• Conservation
• Proportional Reasoning
• Controlling Variables

• Combinatorial Reasoning
• Probabilistic Reasoning
• Correlation Reasoning

9

• UA Freshmen Concrete or Formal?
• Read the handout about concrete vs. formal
  operational reasoning and answer the following
  question.
• Hundreds of students in a non-major, general
  education Earth Science course at the University
  of Akron were given a logical thinking test to
  determine their stage of intellectual
  development. Based on test scores, students were
  characterized as concrete, formal, or
  transitional.
• Predict the approximate proportions of students
  who were concrete/formal thinkers.
• 60/10 2. 45/25
• 3. 25/45 4. 10/60

10

• UA Freshmen Concrete or Formal?
• How can we assess thinking skills?
• GALT test - Group Assessment of Logical Thinking
• 12 question instrument (score 0-12)

• Scoring Categories
• Concrete (0-4) - prefer fact-based approach, rely
  on memorization (25)
• Transitional (5-7) - prefer to apply ideas in a
  practical way (30)
• Formal (8-12) - abstract thinkers, can understand
  previously unseen ideas (45)

(GALT Roadrangka, V., Yeany, R.H, and Padilla,
M.J., U. of Georgia, 1982)
11
Reasoning Skills vs. Academic Stage
GALT Score
Grade Level
12
Observation A majority of students are
unprepared to understand abstract concepts
presented in introductory science class without
appropriate scaffolding activities
13
Using active learning exercises in class
Conceptests
Liquid hazardous waste is
disposed off by pumping it
down injection wells.
Which well location would
be the most suitable to
use for an injection well?
Why?
1.
A
2.
B
3.
C
14

• UA Freshmen Concrete or Formal?
• Did the average score on the GALT test change
  over the course of a semester for either the
  traditional or inquiry classes?
• Yes for both
• 2. Yes for inquiry only
• Yes for traditional only
• No for both

15

• UA Freshmen Concrete or Formal?
• Predict which group of students exhibited the
  greatest total gains in GALT scores. (i.e., gains
  in points scored on tests).
• Concrete
• 2. Transitional
• 3. Formal

16
How can we assess thinking skills?
Pre-test vs. Post-test logical thinking test
scores
n 176
Pre-GALT 6.6 /- 2.8 Post-GALT 7.5 /-
2.6 p lt 0.0005
17
points gain in GALT score vs. Course Structure
UA Freshmen Concrete or Formal?
Inquiry-based Learning, n377
GALT score 0 - 4
GALT score 5 - 7
GALT score 8 - 12
Totals
22, p lt 0.00001
1.3
2
0.1
6, p lt 0.005
0.5
0.7
0.0
Traditional Lecture, n195
18
How can we assess thinking skills?
Concrete thinkers experienced the greatest change
in logical thinking skills. Formal thinkers
experienced no change.
Based on paired T-tests
19
UA Freshmen Concrete or Formal?
20
GALT score vs. Course grade
UA Freshmen Concrete or Formal?
Course Grade, A/B
57
19
GALT score 0 - 4
GALT score 8 - 12
19
38
Course Grade, D/F
F 01 (n83)
Why did 19 of formal learners receive a D/F and
why were 19 of concrete students able to earn an
A/B?
21
Mean class score with Group vs. Individual work
UA Freshmen Concrete or Formal?
Group Inquiry
74
84
Formal
Concrete
70
79
Individual Inquiry
F 02 (n119) p lt 0.005
22
The Tourist, the Gunslinger, and the Gardener
Rethinking Metaphors of Teaching and Learning to
Enhance Student Reasoning
23
Three Teaching Metaphors
Tourist - Teaching focuses on the delivery of
information communication from the instructor.
Gunslinger - Teaching focuses on student
replication of a specific quantity of
information communication from the instructor to
the students.
Gardener - Teaching focuses on student
understanding communication from the instructor
to the students, from students to instructor, and
from students to students.
24
ASU Physics Example (Read the Physics Instruction
and Test Scores handout.) Three instructors
taught a University Physics course during the
same semester. All professors use the same
textbook, cover the same chapters, and receive
similar evaluations. Whos class did best? 1. A
2. B 3. C 4. No difference
Halloun, I.H. and D. Hestenes, The initial
knowledge state of college physics students.
American Journal of Physics, 1985. 53(11) p.
1043-1055.
25
ASU Physics Example (Read the Physics Instruction
and Test Scores handout.) Three instructors
taught a University Physics course during the
same semester. All professors use the same
textbook, cover the same chapters, and receive
similar evaluations. What was the average gain
for the three classes between the pre- and
post-tests? 1. 36 2. 22 3. 13 4. 5
Halloun, I.H. and D. Hestenes, The initial
knowledge state of college physics students.
American Journal of Physics, 1985. 53(11) p.
1043-1055.
26
The Tourist
If they dont understand, say it again . . . only
louder.
27
The Tourist

• My Personal Teaching Evolution
• Chalk and talk fresh notes, drawings, etc. on
  board, pacing matches students
• Overhead transparencies pre-written notes,
  complex diagrams some opportunity for
  adjustments during lecture
• Powerpoint presentations - pre-written notes,
  complex diagrams, perhaps visit websites little
  opportunity for adjustments
• Consequences for student performance

28
The Tourist

• Richard Feynman
• Nobel prize-winning physicist
• Taught highly-regarded series of entertaining and
  erudite introductory physics courses at Caltech
• Did NOT improve student understanding of physics

29
The Tourist

• Teaching focuses on the delivery of information
  what the instructors doing
• Technologys principal role in information
  delivery and assessment has been as an aid to the
  teacher
• Student learning does not change significantly
• Instructor wont ask for directions

If they dont understand, say it again . . . only
louder.
30

• Biology Content Comprehension
• Pre- and post-tests of student comprehension were
  compared for large introductory biology courses
  for non-majors and majors with comparable class
  sizes. The majors course presented more content.
  Mean pre-test score for the non-majors was 29
  and the for the majors was 35.
• Predict the post-test score in the non-majors and
  majors courses.
• 35/41 2. 40/41
• 3. 56/48 4. 48/56

Sundberg, M.D., M.L. Dini, and E. Li, Journal of
Research in Science Teaching, 1994. 31(6) p.
679-693.

31
The Gunslinger
What do you mean you dont know . . . I talked
about that two weeks ago.
32
The Montillation of Traxoline (attributed to Judy
Lanier) Directions Read the passage below and
answer the questions that follow. It is very
important that you learn about traxoline.
Traxoline is a new form of zionter. It is
montilled in Ceristanna. The Ceristannians
gristerlate large amounts of fevon and then
brachter it to quasel traxoline. Traxoline may
well be one of our most lukized snezlaus in the
future because of our zionter lescelidge.

• What is traxoline?
• Where is traxoline montilled?
• How is traxoline quaselled?
• Why is it important to know about traxoline?

33
The Gunslinger

• Summative vs. Formative evaluation lack of
  student learning may not be apparent until its
  too late
• Learning vs. Memorization Students see learning
  as memorization and recall of facts
• Quantity vs. Quality Teaching focus on content
  above concepts
• Right vs. Wrong Emphasis on simple dualistic
  answer structure, few gray areas

34
The Gunslinger

• Teaching focuses on the transfer of a large
  volume of information
• Communication from the instructor to students
• Typical summative evaluation of student learning
  through multiple choice exams emphasizes
  right/wrong answers
• Student understanding does not change
  significantly

What do you mean you dont know . . . I talked
about that two weeks ago.
35
Student Participation (Read the Student
Participation handout.) Researchers analyzed the
classes of twenty professors teaching upper level
students in social sciences and humanities using
traditional lecture methods. Class sizes varied
from 15-44 students. Predict the proportion of
class time devoted to student participation. 1.
2-6 2. 14-19 3. 22-26 4. 34-39
Nunn, C.E., Discussion in the college classroom
Triangulating observational and survey results.
Journal of Higher Education, 1996. 67(3) p.
243-266.

36
The Gardener
. . . what the student does is actually more
important in determining what is learned than
what the teacher does. Thomas Shuell
37
The Gardener

• Learning Environment class setting supports the
  cognitive growth of students at different stages
  of intellectual development
• Metacognition students encouraged to become
  self-reflective of their learning as they explain
  their ideas and listen to those of others
• Formative assessment evaluation of learning is
  on-going, lessons are adapted to identify and
  overcome misconceptions
• Learning Styles instruction recognizes
  diversity of learning styles (Visual - Aural -
  Read/Write - Kinesthetic )

38
The Gardener
Different disciplines, common model. All
improved student achievement

• Pre-class preparation - reading quizzes, homework
  assignments
• In class activities formative assessment
  exercises measured student understanding of key
  concepts
• Collaborative learning - Students explained what
  they knew to each other and listened while others
  explained concepts to them

Crouch Mazur, American Journal of Physics,
2001. 69(9) p.970-977.
Lord, Innovative Higher Education (Biology),
1997. 21(3) p.197-216.
Paulson, Journal of Chemical Education, 1999.
76(8) p.1136-1140.
39
The Gardener

• All Models
• Students responded to formative assessment
  questions posed by instructor
• Students explained what they knew to each other
  and listened while others explained concepts to
  them
• Allowed students to test their understanding of
  the material while they received information

. . . it would be helpful to remember that what
the student does is actually more important in
determining what is learned than what the teacher
does. Thomas J. Shuell, 1986
40
Anatomy of an active learning class
The Gardener

• Earth Science
• Introductory exercise or questions to stimulate
  discussion
• Lecture broken into one or two 1015 minute
  segments
• Students work together in 3-4 person groups
• Formative exercises during class used to assess
  student understanding and progress

41
Guiding students in class preparation
Metacognition (Students thinking about their
learning)

• Effective reading guidelines or reading
  worksheets
• Students complete self-scoring review using a
  knowledge survey
• Surveys match scores to understanding needed for
  success in course

42
Concerns for Potential Gardeners

• Potential Problems
• Questions that address higher-order thinking
  skills require time and effort to create.
• Less time available to deliver information more
  time necessary for students to process
  information.
• Student responses to essay-type questions may be
  difficult to interpret due to poor writing
  skills.
• Time necessary to review assessment exercises for
  large classes discourages the use of such methods.

• Some Solutions
• Use a limited number of forms of assessment.
• Take 10-15 minutes each class to incorporate
  assessment.
• Student responses reduced to brief terms or keyed
  to numbered lists to ease interpretation of
  answers.
• Use of collaborative groups and answer keys can
  reduce time needed to review exercises.

43
. . . but does it work?

• Hake (2002)
• Results from 6,000 physics students in national
  standardized mechanics test
• Improvement in scores from pre-test to post-test

Active learning classes
Traditional classes
44
Authentic Assessment? Are our assessment methods
meeting our teaching and learning goals?
45
Gardening Lessons
Teaching and learning goals can be ordered using
Blooms Taxonomy

• I have heard of it but cant explain much about
  it.
• I can name some of the categories of Blooms
  taxonomy.
• I can identify and describe all six categories of
  Blooms taxonomy.
• I can make up questions representative of the six
  categories of the taxonomy.

46
What is the highest level of learning that
students routinely demonstrate in general
education classes?
memorization and recall understanding using
knowledge taking apart information reorganizing
information making judgements
1. Knowledge 2. Comprehension 3. Application 4.
Analysis 5. Synthesis 6. Evaluation
Content
Higher-order thinking
47
Blooms Taxonomy
Review examples of exercises used in an Earth
Science course at UA and place each exercise into
one of the six Blooms categories.
memorization and recall understanding using
knowledge taking apart information reorganizing
information making judgements
Knowledge Comprehension Application Analysis Synth
esis Evaluation
48
SOLO Taxonomy
Blooms Taxonomy is useful for guiding the
development of questions at a range of levels but
it can not account for student responses
SOLO Structure of the Observed Learning Outcome
(Biggs Collis, 1982) SOLO Taxonomy can help
evaluate the level of student responses
especially for open-ended questions. It can serve
as a useful scoring rubric to distribute to
students before they answer a high-order question.
49
SOLO Taxonomy
Levels Character of Answer
Prestructural Student response is not
appropriate Unistructural Response addresses only
one aspect of question Multistructural Response
addresses multiple aspects of question as
independent concepts Relational Multiple aspects
are integrated into a coherent whole clear
conceptual understanding Extended
Abstract Relational response extended and
generalized beyond given information to a new
topic/area
50
SOLO Taxonomy
Example Review the descriptions of the responses
to the question Compare and contrast the effects
of blindness and deafness on language
development and place them in the appropriate
SOLO level.
Think Pair Share Exercise
51
Acknowledgments
This work benefited from discussions with
numerous colleagues at the University of Akron in
the Departments of Geology, Biology, and Physics
and the College of Education
52
Lectures vs. Alternatives Traditional lecture is
_____________ at promoting student thought than
other teaching methods. 1. More effective 2. Less
effective 3. About the same
53
Lectures and Learning
Lecture vs. Alternatives Studies reveal that
traditional lecture is less effective in
promoting thought than a variety of alternative
methods
Teaching Method Discussion Reading independent
study Other
Lecture less effective 29 1 17
No difference in methods 1 3 18
Lecture more effective 2 1 1
Bligh, D.A., Whats the use of lectures?
Josey-Bass, 2000. 346 p.
54
Lectures and Learning

• We can efficiently use lecture time to
• Deliver information.
• Assess student knowledge with multiple-choice
  exams.
• Such trusted techniques
• Produce a reasonable distribution of grades.
• Require relatively little time for preparation.
• Provide a substantial measure of control over the
  classroom environment.

But . . . research shows that such techniques may
emphasize low-level thinking skills at the
expense of student intellectual development. How
can we adapt our teaching behaviors to promote
higher-order reasoning?