Ways of Knowing in the Sciences (Integrated Liberal Studies 153)

1
On assessing student understanding of the nature
of scientific knowledge
Nancy Ruggeri Department of Curriculum
Instruction University of Wisconsin-Madison
2
Ways of Knowing in the Sciences Integrated
Liberal Studies 153

• General science course for non-majors
• Taught in the ILS program on campus
• 40 students per semester
• Activity and discussion-oriented classroom
• Four modules Planetary motion, Plate tectonics,
  Evolutionary theory, Global climate change

3
Course goals

• To understand how scientific knowledge develops
  (the how instead of the what)
• To develop critical thinking skills via
  scientific reasoning
• To increase student interest in science
• To increase student awareness of science and its
  integral relationship with society

4
Course objectives

• After taking ILS 153, students will gain a better
  understanding of
• How scientists use different ways of knowing via
  empirical (inductive) and theoretical (deductive)
  approaches to answer questions about the natural
  world
• The methods scientists use to answer questions
  (observation, experimentation and modeling)
• The notion that scientific knowledge is subject
  to change given new data, interpretations and
  models
• The types of uncertainties that are inherent in
  scientific data and interpretations
• The ways in which scientists address uncertainty
  in scientific data and interpretations
• The role of the scientific community in placing
  checks and balances on the acceptance of
  scientific findings
• The ways in which politics, economics and
  religion can influence scientific practice

5
Selected Readings

• Bryson, B. A Short History of Nearly Everything.
  . Broadway Books. New York, NY.
• Shermer, M. 2002. Why People Believe Weird
  Things Pseudoscience, superstition, and other
  confusions of our time. Henry Holt and Company.
  New York, NY.
• Huff, D. 1954. How to Lie with Statistics. W.W.
  Norton Company. New York, NY.
• Feynman, R.P. 1998. Uncertainty of Science. In
  The Meaning of it All Thoughts of a
  Citizen-scientist. Perseus, New York, NY, USA.
• Oreskes, N. 1999. The Rejection of Continental
  Drift Theory and Method in American Earth
  Science. Oxford University Press.
• Weiner, J. 1994. The Beak of the Finch A Story
  of Evolution in Our Time. Vintage Books. New
  York, NY.
• Pollack, H. 2003. Uncertain Science Uncertain
  World. Cambridge University Press.
• Weart, Spencer R. 2003. The Discovery of Global
  Warming. Harvard University Press. Cambridge, MA.
• NYT Science Times
• Excerpts from Darwin, Galileo (Great Books series)

6
(No Transcript)
7
How would you describe the instructional approach
taken in this class?

• Open-minded and creative
• Lots of interactive lectures and class
  discussions
• Very much participation based. We used each
  others knowledge to build our understanding of
  concepts
• Professors tried to shift the paradigm of
  science by developing critical thinking skills,
  not just deliver knowledge of science which
  already prove by others

8
At issue

• How to best elicit student thinking about the
  nature and process of science?
• How are the complex elements of the process of
  science identified in the course objectives
  integrated into students understanding?

9
Assessment in ILS 153

• Formative assessment
• Administered pre-instruction or during
  instruction to inform both teaching and learning
  processes
• Summative assessment
• Measure extent to which students have achieved a
  particular learning goal

10
Assessments Implemented in ILS 153
Assessment Formative Summative Graded
Pretest/Posttest v
Reflective Journals v v v
Writing Assignments v v v
In-class Group Activities v v
Large Group Discussions v (v)
Small Group Discussions v
Minute Papers v
Quizzes Exams (v) v v
Student Assessment of Learning Gains (SALG) (v)
11
PRETEST responses How would you describe how
scientists (biologists, chemists, earth
scientists) do their work? For example, what
types of activities do they do to learn about the
natural world?
Jake Scientists do their work by looking for
problems. Through experiments, scientists come to
conclusions by the results and evidence
support. They essentially solve problems and find
answers Susan They must observe and do
experiments. They must take detailed recordings
down and apply them to their question at
hand. Elina They read scientific journals,
observe the world around them, and theorize based
on these two things.
12
POSTTEST responses How would you describe how
scientists (biologists, chemists, earth
scientists) do their work? For example, what
types of activities do they do to learn about the
natural world?
Jake There is no one way of doing science.
Generally, scientists work inductively or
deductively. They collect data from observation
or come to conclusions about principles. Then
they test these findings to see if they hold up,
and publish all aspects for peer review. Susan
They take information and produce models of how
certain natural processes work. They must be
highly objective and review others work as a
community. Elina Scientists are in the business
of observation and explanation. Essentially they
find something interesting, observe a data field
regarding it (be it natural, experimental, or
quantitative) and attempt to explain the data.
They can also reverse the order and come up with
a theory, finding data to support it.
13
Concerns

• How do we avoid having students just telling us
  what we want to hear?
• Are they simply memorizing aspects of the process
  of science that we highlight, and regurgitating
  them?
• Will this understanding be forgotten two weeks
  after the semester is over, much like other
  content knowledge?

14
Possible solutions

• Create assessments that require students to
  integrate the information in context
• Students make personal connections to their
  understanding
• Reflections and essays responses can provide a
  broader demonstration of student understanding

15
Weekly Journal Reflections

• Reiterate themes from reading and lecture
• Opportunity for students to synthesize complex
  notions
• Ideally provides an opportunity to see ones own
  changes over the course of the semester

16
Example Journal Reflection Questions

• This weeks reflection question is associated
  with this blog on the NY Times http//judson.blog
  s.nytimes.com/2009/02/10/guest-column-letting-scie
  ntists-off-the-leash/
• We would like you to write a 1-2 page response
  to the article, as if you were going to post
  online.  We would like you to write it from your
  perspective as an undergraduate at a research
  university, and explicitly include at least one
  piece of information that you have learned in
  this class.  
• Course objective The ways in which politics,
  economics and religion can influence scientific
  practice

17
Example Journal Reflection Questions

• Scientists are not free agents, historians and
  sociologists have argued, and the social context
  of their work not only delimits their options but
  may even determine the content of their
  knowledge. And if all knowledge is socially
  constructed, then objectivity is a chimera. This
  radical claim strikes at the heart of scientists
  beliefs about their enterprise.
• After reading the text surrounding this quote
  from Naomi Oreskes (1999), discuss the valid
  points she makes as a historian, and compare them
  to the valid points Pollack (2003) makes (in the
  reading for today) as a scientist. Be sure to
  support your argument with evidence.

18
Final Journal Reflection

• For Monday's reflection, please choose two of the
  overarching themes from the course overview and
  discuss how the themes interrelate with one
  another, drawing on specific examples from the
  whole semester.  
• The more examples you use, the better. If you
  can think of other themes from the course, you
  are welcome to write about these as well. We
  would be very interested to hear them. This
  assignment will help you to prepare for the final.

19
Some excerpts from students final reflections

• The nature of science is very much a mess of
  interrelating ideas. Scientists have to deal with
  society, politics, objectivity, honesty,
  uncertainty, and many other factors when they
  create and present their work.
• I would argue that the idea that science is
  influenced by cultural norms, is essentially the
  same thing as saying that science is done by
  humans that are inherently biased. Scientists do,
  indeed, strive for objectivity, but at the end of
  the day, they are still (as shown by the op/ed
  article we read) very dependent on governmental
  and private donations to finance their research.

20
(No Transcript)
21
Final Exam

• Had there been a final paper for this course
  instead of an exam, write a short summary or
  outline of the 6-8-page paper you would have
  written. Provide the main thesis and some of the
  examples from the class you would have included
  in your paper. (Obviously, you cannot include
  everything we have talked about this semester, so
  you must choose a thesis and be selective about
  your examples!)
• Do you think your notions of how science is done
  and its implications for society have changed as
  a result of taking this course? If so, how do you
  think they have changed and why? If not, explain
  why not.

22
FINAL EXAM Do you think your notions of how
science is done and its implications for society
have changed as a result of taking this course?
Is so, how do you think they have changed and
why? If not, explain.

• John Yes. I never put together the pieces of
  how science in society are and so intertwined.
  The article about the funding of science made it
  really clear to me how science cant function
  without funding with requires the scientists to
  cater to these funders and often practice fast
  science rather than good science
• Elina My assumptions about science have changed
  because I actually view it as more certain now. I
  was a major skeptic when I came into the class. I
  did not know the power behind a scientific
  theory. Things that people take as fact like
  evolution and climate changeI understand, now,
  the difference between what is scientific and
  what is societal and how much society can dilute
  the certainties of science. I really appreciate
  how much more certain scientists actually are
  when they call something a theory.

23
(No Transcript)
24
What do students think they learned as a result
of taking ILS 153?
25
SALG Please comment on how your understanding of
the subject has changed CHANGED as a result of
this class

• I learned a ton from this class, not only about
  subject material but about ways of knowing and
  scientific process
• I am much more knowledgeable on all of these
  topics and feel more confident in talking about
  them. I am also more interested in each of them
  than I previously was.
• I came in knowing a lot about science, but I
  definitely think more critically about things
  like science in the news
• I am more skeptical of what I hear from people
  but I am also more confident in true scientific
  findings, because I know how much work it really
  takes.
• greater appreciation for what scientist do
• i am more interested in science now

26
SALG What will you carry with you into other
classes of other aspects of your life?

• Using evidence to support arguments was a huge
  gain from this class. It is easier to get my
  points across because I know more effective ways
  of going about it now.
• I feel like analyzing data and being critical of
  information and ideas presented to me will really
  help in reading research in my area of
  interest/career, but I don't see using the big
  concepts at all in my daily life.
• I will be more skeptical of data and the
  arguments of scientists that are not backed by
  the scientific community as a whole.
• Definitely being aware of uncertainty, and those
  that make outrageous claims with shady evidence.
• I'm surprised how much I've talked about this
  class and used the information in conversations
  and to my benefit.

27
Assessments Strengths and Weaknesses
Assessment Strengths Weaknesses
Pretest/Posttest Has potential to measure targeted learning gains and identify prior conceptions Difficult to design adequate instrument student conceptions weak tendency to tell us what we want to hear
Reflective Journals Excellent opportunity for students to play with ideas instructors get sense of how well students are integrating understanding Very labor and time intensive to grade clumsy to collect and hand back difficult to grade
Student Assessment of Learning Gains (SALG) Very informative preliminary data analysis online anonymous Lengthy requires students to complete online response rate can be low
28
Course goals

• To understand how scientific knowledge develops
  (the how instead of the what)
• To develop critical thinking skills via
  scientific reasoning
• To increase student interest in science
• To increase student awareness of science and its
  integral relationship with society

29
Challenges

• Complexity in describing the nature and process
  of science
• The moment we isolate individual elements, they
  lose the importance of context
• Contextual nature of this makes it challenging

30
Future changes for ILS 153 assessment

• Consider assigning a final paper to synthesize
  complexity of ideas
• Build stronger connections between assessments
  that target one course theme over the semester