The Mystery of b : b false

1
The Mystery of b (b false)

• Stuart Reges
• University of Washington
• http//www.cs.washington.edu/homes/reges/mystery

2
Data overview

• 1988 AP Computer Science Exam (Pascal)
• Divided into A and B (roughly CS1 and CS2)
• 35 A multiple choice
• 15 B multiple choice
• 3 A free response
• 2 B free response
• 7,374 AB tests 3,344 A-only tests

3
Analysis overview

• Correlations between multiple choice questions
• Any correlation below 0.2 ignored
• Correlations between multiple choice and free
  response questions
• Didn't find much of interest in correlating free
  response with free response

4
The mystery?

• In general, not much correlated at the level of
  0.2 or higher
• But, five multiple choice questions kept popping
  up over and over
• nickname the powerhouse questions
• One question in particular (question 23) stood
  out

5
A multiple choice correlations
6
Minus powerhouse questions
7
B multiple choice correlations
8
Minus powerhouse questions
9
Free response correlations
10
Free response vs powerhouse
11
Greatest AB vs A-only deltas
12
Mysterious question 23

• If b is a Boolean variable, then the statementb
  (b false) has what effect?
• It causes a compile-time error message.
• It causes a run-time error message.
• It causes b to have value false regardless of its
  value just before the statement was executed.
• It always changes the value of b.
• It changes the value of b if and only if b had
  value true just before the statement was
  executed.

13
And question 20

• procedure Wow(n integer)
• begin
• if n gt 1 then Wow(n div 2)
• write(n, ' ')
• end
• The procedure call Wow(16) will yield as output
  which of the following sequences of numbers?
• 10 8 6 4 2
• 16 8 4 2 1
• 1 2 4 8 16
• 32 16 8 4 2
• 2 4 8 16 32

14
What does this mean?

• Bob Floyd These questions seem to test whether a
  student has a model of computation whether they
  can play computer in their head
• Don Knuth "I conclude that roughly 2 of all
  people think algorithmically, in the sense that
  they can reason rapidly about algorithmic
  processes."

15
More Knuth

• The other missing concept that seems to separate
  mathematicians from computer scientists is
  related to the assignment operation , which
  changes values of quantities. More precisely, I
  would say that the missing concept is the dynamic
  notion of the state of a process. How did I get
  here? What is true now? What should happen next
  if Im going to get to the end? Changing states
  of affairs, or snapshots of a computation, seem
  to be intimately related to algorithms and
  algorithmic thinking.

16
Snapshot example

• public static int mystery(int n)
• int x 0
• while (n 2 0)
• // Point A
• n n / 2
• x
• // Point B
• // Point C
• return x
• Is (n 2 0) always true, never true, or
  sometimes true/sometimes false at points A, B and
  C?

17
What next?

• More exploration, more studies
• Powerhouse questions suggest certain kinds of
  questions to explore
• Is this measuring some innate aptitude? Some
  acquired ability? Or is it a fluke?
• Join the discussion
• http//www.cs.washington.edu/homes/reges/mystery