Semantic Predicate Hoisting

1
Semantic Predicate Hoisting

• Terence Parr
• University of San Francisco

2
Topics

• Motivation and problem definition
• Semantics
• Predicates versus gated alternatives

3
Context-sensitive parsing

• Fortran array reference versus function call
  A(I,3) versus MAX(A,B)
• Semantic predicate solution Call symbol table to
  choose between alternatives (k1 suffices)

expr options k1 arrayRef
array-ref-action funcRef
function-call-action arrayRef
isvar(input.LT(1))? ID ( exprlist )
funcRef isfunc(input.LT(1))? ID (
exprlist )
4
DFA predicting alts of expr

• Restricted to k1 syntactic lookahead
• Checks predicates after lookahead checked

5
Keywords as variables

• Some twisted languages allow keywords as
  variables if if call call and call if
• One solution keep keywords as IDs then use
  predicate when you need it to be keyword

// LL(1) if keywords were not IDs stat keyIF
expr stat keyCALL ID '' ''
expr ID keyIF input.LT(1).equals("if")
? ID keyCALL input.LT(1).equals("call")?
ID
6
DFA predicting stat

• ID ID requiressemanticpredicateresolution
• if impliesalt 1, callimplies alt 2

7
Predicate Semantics

• Upon syntactic nondeterminism, use semantic
  predicates to resolve if n-1 predicates available
  for n alternatives (dont hoist for deterministic
  alts)
• nth predicate is simply !(p1p2..pn-1)
• a p1? A p2? A A
• Predicates encountered during DFA construction
  combined
• p1? followed by p2? gt p1p2
• p1? p2? gt p1p2
• Predicates beyond hoisting depth 0 ignoreda A
  p1? A p2?
• Predicates are both disambiguating and validating
• Predicates executed for correct syntactic context
• All paths matching ambiguous input must be
  covered with a predicate expression

8
Syntactic Context Evaluation

• dont want predicates to execute unless the
  syntactic context is right might crash

a b p2? ID b p1? ID INT
9
Incomplete Coverage

• For input ID, the second alt of b is not covered
  with a predicate cannot guarantee correctness
  report error.

a b ID b p? ID ID
cov.g15 The following alternatives are
insufficiently covered with
predicates 1
Issue caught by Paul Lucas (previously at BEA)
at last years workshop!
10
Combining predicates

• Syntactically ambiguous grammar (all paths match
  B).

a b B b p1? c p2? B c
p3? B
11
Predicates vs Gated Alternatives

• Predicates do not gate in/out an alternative,
  which causes a problem in lexer!!
• a matches A and B (likekeyword vs ID
  ambiguity)
• Want a gate that makes A or Binvisible e.g.,
  might want "aa" asB not 2 As like intint as
  ID or 2keywords

lexer grammar T A p1? 'a' B p2?
('a''b')
12
Summary

• Hoisted predicates were invented in PCCTS with ad
  hoc solution bug city!
• Missing from ANTLR v2 we miss them!
• ANTLR v3 defines them formally and implementation
  is simple addition to LL() analysis algorithm
  and code generation very consistent and clean
  this time!