Measuring Students

1
Measuring Students Critical Thinking Problems
and Possible Solutions

• Aggi Tiwari RN PhD
• Department of Nursing Studies
• The University of Hong Kong

2
Measuring CT Some problems possible solutions

• (1) Is the conception of CT defensible?

3
Conceptualization of CT

• Diverse
• problem-solving (Watson Glaser, 1980)
• deductive-inductive reasoning (Ennis, 1962, 1987)
• reflective skepticism (McPeck, 1981)
• questioning assumptions (Brookfield, 1987)
• dialectical thinking (Paul, 1993)
• purposeful, self-regulatory judgment (Facione
  Facione, 1997)
• Lack of universal agreement

4
Different interpretations of CT
Different criteria of performance
Results comparable?
5
Justifications for adopting a particular
conception of CT

• Is the definition broad enough to encompass the
  key features of CT?
• Is the definition relevant for the study?
• What is the scientific basis for the conception?
• Is direct testing possible?

6
Conception of CT adopted An example

• purposeful self-regulatory judgment manifests
  itself in giving reasoned consideration to the
  evidence, context, standards, methods, and
  conceptual structures within which a decision is
  made about what to believe or what to do
• (Facione Facione, 1997, p.1)

7
Justifications for adopting Facione Faciones
definition of CT

• (i) Broad enough to encompass
• deductive-inductive reasoning
• problem-solving
• reflective skepticism
• dialectical thinking

8
Justifications for adopting Facione Faciones
definition of CT

• (ii) Relevant for participants of study
• Practicing nurses need to -
• think logically
• suspend judgment temporarily
• reason from different perspectives
• regularly monitor own reasoning

9
Justifications for adopting Facione Faciones
definition of CT

• (iii) Derived from rigorous scientific methods,
  has interdisciplinary consensus, subjected to
  on-going empirical testing.
• (iv) Can be tested directly
• CCTDI
• CCTST
• HCTSR

10
Measuring CT Some problems possible solutions

• (2) What procedures should be used to test the
  impact of a specific intervention on students
  CT?

11
Study Design - Example A

• Time Series (Cook Campbell, 1979)
• Pretest measure - students CT assessed before
  implementation of PBL
• Post-treatment measures - measurements of
  students CT taken on completion

12
Study Design - Example A

• Criticism
• No control group - would students CT be enhanced
  because of some other reasons?
• Would the effects of the intervention be retained?

13
Study Design - Example B

• Group Pretest Exp. Post-exp.
• Stage Stage

X O2
E
O1
O3
O1
O2
O3
C
14
Study Design - Example B

• Control group provides a baseline to evaluate the
  effects of the intervention.
• Need to overcome the design deficits in
  nonequivalent control group design e.g. pretest.
• Would the effects of the intervention be retained
  after the intervening period?
• The addition of a post-experimental follow-up
  study.

15
Measuring CT Some problems possible solutions

• (3) Are the CT measuring tools valid and
  reliable?

16
Choosing CT measuring toolsSome considerations

• CT is complex, not a collection of simple skills
  - unlikely that a single tool can cover all
  dimensions of CT.
• Combination of measurements - the strengths of
  each measuring method reflected in the overall
  results, deficiencies of one method compensated
  by the others.

17
Choosing CT measuring toolsSome considerations

• Both the skills and dispositional aspects of CT
  should be measured.
• The CT measuring tool should be based on a
  defensible conception of CT.
• Constructed at a level appropriate for those
  taking the test.
• Is the tool culturally sensitive?

18
Examples of CT measuring tools

• Example (i)
• CCTDI - one-shot measurement of HK and
  Australian nursing students CT
• Example (ii)
• CCTDI interviews - repeated measures of
  experimental and control groups
• Example (iii)
• CCTDI HCTSR - pretest and posttest of HK
  nursing students CT

19
Measuring CT Some problems possible solutions

• (4) How to ensure that the interpretation is
  culturally sensitive?

20
The case of the intellectually biased students???

• Disposition toward truth-seeking
• being eager to seek the best knowledge in a
  given context, courageous about asking questions,
  and honest and objective about pursuing inquiry
  even if the findings do not support ones
  self-interests or ones preconceived opinions
  (Facione, Sanchez Facione, 1994, p.6).

21
The case of the intellectually biased students???

• Exp. Group Control Group
• Pretest 33.7 33.8
• Posttest 1 34.0 34.1
• Posttest 2 34.7 34.6
• Maximum 60 Minimum 10
• Ambivalent inclination 30 - 40
• Positive inclination 40
• Negative inclination 30-

22
In the Chinese context...

• Social harmony - foundation of Chinese culture
• Underpinning communication and interpersonal
  relationships
• Maintaining harmony is paramount
• Preserving peaceful relations through
  communication processes
• Hanxu
• Mianzi

23
In the Chinese context...

• Hanxu (implicit communication)
• leaves the unspoken to the listeners
• Mianzi (face-directed communication strategies)
• intentions or disagreements deliberately
  articulated indirectly
• Buanzuei (talk back)
• challenging or questioning others
• upsets the established hierarchical relationships
  and social harmony

24
The case of the intellectually biased students???

• Chinese students not inclined to seek the truth?
• Chinese students value harmony in interpersonal
  relationships more than the pursuit of truth?

25
Summary

• Measuring students CT
• (1) Is the conception of CT defensible?
• (2) What procedures should be used to test the
  impact of a specific intervention on students
  CT?
• (3) Are the CT measuring tools valid and
  reliable?
• (4) How to ensure that the interpretation is
  culturally sensitive?