Lesson Nine

1
Lesson 9

• How Good Is The Evidence?
• Part II

2
Personal Observation as Evidence

• Personal observation is the basis for a great
  deal of scientific research.
• People trust what they can see.
• Personal observations are vivid and we rely upon
  them as evidence.

3

• Problems with personal observation as evidence
• People see/hear what they want.
• People are biased.
• We should be wary of relying on evidence based on
  one persons observations in situations where
  multiple observers might have different
  observations.

4

• Personal observation is most reliable as evidence
  when it is based on
• Recent observations.
• Observations made by several people.
• Who have no apparent bias related to the event
  observed.
• Who were in a good position to see what happened.
• Examples
• T.V. program
• Email to friend

5
Research Studies as Evidence

• A systematic collection of observations by people
  trained to do scientific research.
• Scientific Method
• Avoid many of the biases in our observations.
• Seeks information in the form of publicly
  verifiable data. Multiple qualified people make
  their own observation.
• Level of control. Reduces error in observation
  and in the interpretation of research findings.
• Easier to achieve in physical sciences.

6

• Scientific Method
• Tries to be precise and consistent in its use of
  language.
• Helped us achieve observations that are more
  dependable.
• Faults are not totally done away with.

7
Problems with Research Findings

• When communicators appeal to research as
  evidence, you should remember these important
  points (pgs. 105-107)
• Not all research is created equal.
• Research evidence must be understood in context.
• Research findings are not solid proof of
  conclusions.
• Not all researchers are the same (biases, etc).
• Communicators sometimes distort or over-simplify
  findings.

8
Problems with Research Findings

• When communicators appeal to research as
  evidence, you should remember these important
  points (pgs. 105-107)
• Research findings change over time (social
  sciences).
• Research can vary by how controlled or artificial
  it is.
• Research can be affected by the need for
  financial gain, status, security, etc.

9

• Always ask the following critical questions about
  research evidence(pg 108)
• What is the quality of the source of the
  research?
• Has the research been replicated?
• How selective has the communicator been in
  choosing studies?
• Is there any evidence of strong-sense critical
  thinking?

10

• Always ask the following critical questions about
  research evidence(pg 108)
• Is there any reason for someone to have distorted
  the research?
• Are conditions in the research artificial and
  therefore distorted?
• How far can we generalize, given the research
  sample? We will discuss this more in a moment.
• Are there any biases or distortions in the
  surveys, questionnaires, ratings, or other
  measurements that the researcher uses?

11

• Such questions help us avoid
• Blind acceptance.
• The impossible certainty fallacy.
• Assuming that a research conclusion should be
  rejected if it is not 100 certain. All research
  is going to have some doubt associated with it.

12
Generalizing from the Research Sample

• Communicators will often use research reports to
  support generalizations.
• The dependability of such generalizations depends
  upon
• Number of people or events studied
• The variety (or breadth) of people or events
  studied
• The randomness of the events or people studied.

13

• Sampling
• The process of selecting events or persons to be
  studied (sample groups).
• Is used because researches can never study all
  events or people about which they are studying.
• Use sample group to generalize about the wider
  population of persons or events.
• Not all sample groups are created equal.
• How do we know if a sample group is dependable?

14

• By examining the three items we just listed
  number, breadth, and randomness.
• The sample size must be large enough to justify
  the generalization or conclusion.
• Usually the more people or events that are
  studied, the more dependable the conclusion.
• The sample size must contain as much diversity as
  the type of events/people about which conclusions
  are being drawn.
• The more random the sample size, the better.
  Randomness helps insure that all events/people
  about which they are drawing conclusions have an
  equal chance of being part of the sample group.

15

• Critical questions of all research studies
• How many events or people were sampled?
• How diverse (breadth) is the sample group?
• How random was the sample group?
• If we dont answer these questions we might be
  guilty of
• Overgeneralization or hasty generalization.
• Study results are much broader than the findings
  allow.

16
Example (pg. 109-110)

• Alcohol consumption is at an all-time high at
  colleges nationwide. A recent survey conducted by
  Drinksville University found that of the 250
  people surveyed, 89 percent drink on a
  semi-regular basis.
• What problems do you see with that paragraph?

17
Biased Surveys

• How dependable are surveys and questionnaires
  used to measure peoples attitudes and beliefs?
• We have to be aware of various influences that
  may affect the dependability of such surveys.
• Are the questions answered honestly?
• Are the questions ambiguous and thus subject to
  multiple interpretations? How were the questions
  worded?
• Do the questions contain built-in bias (wording,
  etc)?

18

• Example Page 110
• A college professor found that 86 percent of
  respondents believe that President Bush has
  failed the American people with respect to his
  handling of the war in Iraq
• What was the survey question used in the poll?
• What do you think about the Presidents
  misguided efforts in the war in Iraq
• The poll question has a clear built-in bias.
• Responses obtained are not an accurate indicator
  of attitudes concerning President Bushs handling
  of the war in Iraq.

19

• Another example from Page 111
• Question Should poor people who refuse to get a
  job be allowed to receive welfare benefits?
• Conclusion 93 of people responding believe poor
  people should not receive welfare benefits.
• Survey results can also be affected by where the
  question is located in the survey, the length of
  the survey, the topic of the survey, etc.
• As critical thinkers we should examine survey
  procedures carefully before accepting survey
  results.

20
Group Activity

• Review the example on pages 112-113
• Work through Passage 3 on page 119.
• What are the problems with the research survey
  mentioned there?

21
Case Examples as Evidence

• Case examples are detailed descriptions of one or
  several individuals or events that are used to
  support a conclusion.
• Case examples can be very compelling because they
  are usually easy to visualize.
• Dramatic stories appeal to our emotions and
  therefore often distract us from seeking more
  relevant research evidence.

22

• We should ask critical questions in order to
  determine the dependability of case examples as
  evidence
• Is the example typical?
• Are there powerful counterexamples?
• Are there biases in how the examples are
  reported?

23
Analogies as Evidence

• Analogies compare two known things to allow the
  reader to better understand the relationship to
  something that is unfamiliar.
• The first part of an analogy involves a familiar
  object or concept.
• The second part of the analogy contains the
  relationship between the familiar objects or
  concepts.
• This relationship is used to create a principle
  that can be used to assist the understanding of a
  different object or concept.
• Finally, the new or less familiar object or
  concept is described in the same format as the
  known object or concept.

24

• Adults cannot learn all of the intricacies of
  new computer technology. Trying to teach adults
  new computer systems is like trying to teach an
  old dog new tricks.
• As an educator it is important to weed out
  problem students early and take care of the
  problems they present because one bad egg ruins
  the omelet.

25

• Reasoning by analogy is a common way of
  presenting evidence to support a conclusion.
• But how is it for evidence?
• Accurate analogies are powerful evidence, but
  they are often difficult for people to evaluate
  (assess).
• Overly emotional analogies cloud the real issues
  in argument and prevent meaningful discussion.

26
Evaluating Analogies

• To evaluate the quality of an analogy, we need to
  focus on two things
• The number of ways the two things being compared
  are similar and different.
• The relevance of the similarities and the
  differences.
• Dependable analogies will be ones in which the
  two things we compare possess relevant
  similarities and lack relevant differences.
• When an analogy is proposed in which there are
  important relevant differences it is known as a
  Faulty Analogy.

27
 Class Assignment

• Read chapter 10 for next week.
• Evaluating an Historical Website Handout.
• Lesson 9
• The End