Recursive descent parsing

1
Recursive descent parsing
2
The Stack

• One easy way to do recursive descent parsing is
  to have each parse method take the tokens it
  needs, build a parse tree, and put the parse tree
  on a global stack
• Write a parse method for each nonterminal in the
  grammar
• Each parse method should get the tokens it needs,
  and only those tokens
• Those tokens (usually) go on the stack
• Each parse method may call other parse methods,
  and expect those methods to leave their results
  on the stack
• Each (successful) parse method should leave one
  result on the stack

3
From Recognizer to Parser

• Make a copy of your Recognizer (you may want it
  later) and rename it Parser
• The Recognizer code will form the skeleton of
  your Parser
• Create a StackltTreeltStringgtgt as a globally
  available instance variable
• You will use this to hold the trees as you build
  them
• The methods from the Recognizer all return a
  boolean do not change this
• Your new results will go onto the Stack
• Each time you recognize something, also build a
  Tree to represent it, and put this Tree onto the
  Stack
• Most of the time, you will assemble the new Tree
  from one or two new pieces and from Trees you
  previously put onto the Stack

4
Example while statement

• ltwhile statementgt while ltconditiongt ltblockgt
• The parse method for a ltwhile statementgt does
  this
• Calls the Tokenizer, which returns a while
  token
• Makes the while into a Tree, which it puts on
  the stack
• Calls the parser for ltconditiongt, which parses a
  condition and puts a Tree representation of that
  condition on the stack
• Stack now contains while ltconditiongt (stack
  top is on the right), where ltconditiongt stands
  for some created Tree
• Calls the parser for ltblockgt, which parses a
  block and puts a Tree representation of that
  block on the stack
• Stack now contains while ltconditiongt
  ltblockgt
• Pops the top three things from the stack,
  assembles them into a Tree representing a while
  statement, and pushes this Tree onto the stack

5
Sample Java code

• public boolean whileCommand() if
  (keyword("while")) if (condition())
  if (block())
  makeTree(3, 2, 1) return true
  error("Error in
  \"while\" statement") return false

6
makeTree
private void makeTree(int rootIndex, int...
childIndices) // Get root from stack
TreeltTokengt root getStackItem(rootIndex)
// Get other trees from stack and add them
as children of root for (int i 0 i lt
childIndices.length i)
root.addChild(getStackItem(childIndicesi))
// Pop root and all children from
stack for (int i 0 i lt
childIndices.length i)
stack.pop() // Put the root
back on the stack stack.push(root)
7
getStackItem
private TreeltTokengt getStackItem(int n)
return stack.get(stack.size() - n)
8
Fancier error messages

• public boolean whileCommand() if
  (keyword("while")) if (condition())
  if (block())
  makeTree(3, 2, 1) // or some such
  return true
  error("Error in \"while\" block")
  error("Error in \"while\" condition")
  return false

9
Alternative code

• public boolean whileCommand() if
  (keyword("while") condition()
  block()) makeTree(3, 2, 1) // or some
  such return true return
  false
• No room for an error condition in this code

10
Alternative code with one message

• public boolean whileCommand() if
  (keyword("while")) if (condition())
  (block()) makeTree(3, 2, 1) //
  or some such return true
  error("Error in \"while\" statement")
  return false

11
Tricky problem Defining lttermgt

• lttermgt ltfactorgt lttermgt ltfactorgt
• (For simplicity, Im ignoring the / and
  operators)
• This is logically correct, but it defines the
  wrong tree for expressions such as x y z --
  treats it as x (y z)
• lttermgt lttermgt ltfactorgt ltfactorgt
• This is equally correct, and correctly defines x
  y z as meaning (x y) z
• However, the left recursion cant be programmed
  To recognize a term, first recognize a term
• Solution lttermgt ltfactorgt ltfactorgt
• This turns the recursion into an iteration
• The result is easy to program correctly

12
Code for isTerm()

• public boolean isTerm() if (!isFactor())
  return false while (isMultiplyOperator())
  if (!isFactor()) error("No
  term after '' or '/'")
  makeTree(2, 3, 1) // , first factor, second
  factor return true
• Heres a snippet of code from my JUnit test
  methods
• use("x y z")assertTrue(parser.isTerm())ass
  ertTree("((x, y), z)")
• use(String) and assertTree(String) are helper
  methods that Ive written you should be able to
  figure out what they do

13
An isCommand() method

• public boolean command() if
  (isAction()) return true if
  (isRepeatStatement()) return true if
  (isWhileStatement()) return true ...

14
My helper methods

• I wrote a number of helper methods for the Parser
  and for the ParserTest classes
• One very useful method is tree, in the ParserTest
  class
• tree just takes Objects and builds a tree from
  them
• This method lets me build parse trees for use in
  assertEquals tests
• Another is assertStackTop, which is just
• private void assertStackTop(Tree bt)
  assertEquals(bt, parser.stack.peek())
• Examples
• Tree condition tree("", "2", "2")
• assertStackTop(tree("if", condition, "list"))

15
The End