Extreme Grammaring

1
Extreme Grammaring

• Development of an
• industrial strength
• ISO VDM-SL Grammar

2
Introduction

• Need of building a parser for VDM from a VDM-SL
  grammar (VooDooM project)
• Although parsing is a subset of a well studied
  area like compilers, grammars were always looked
  upon as the ugly duckling.

Solution

• Extreme Programming Engineering of Grammars
• Extreme Grammaring

3
Background

• VDM
• Vienna Development Method (VDM) is one of the
  most mature formal methods
• Primarily intended for formal specification and
  development of functional aspects of software
  systems.
• The importance of a VDM-SL Grammar
• Documentation
• Build a parser (metric generators, language
  generators, ...)

4
Starting point

• Previous work
• VDM-SL grammar in Happy Alex
• Some problems
• State of Art (Hacking v.s. Engineering)
• Grammar was encoded directly
• Difficult to maintain/change (300 rules)
• Lack of tool support
• ...

5
Principles of Grammar Engineering

• Introduced by Lämmel in Towards an engineering
  discipline for grammaware
• Start from specifications - base-line grammar
• Implement by customization - technology,
  implementation
• Separate concerns - modularization
• Enable evolution - minimize impact of changes
• Ensure quality - metrics, correctness
• Automate - traceability and scalability

6
Extreme Programming

• The Planning Game - scope, priorities, technical
  estimates
• Small Releases - very short release cycle
• Metaphor - shared story
• Simple Design - remove complexity when found
• Testing - continuous unit testing, test-first
  design
• Refactoring - restructure without functionality
  changes
• Pair Programming - two programmers one machine
• Collective Ownership - change each others code
  (anytime)
• Continuous Integration - build and test
• 40-Hour Week - work more produce less
• On-Site Customer - user in team
• Coding Standards - no irrelevant personal
  preferences

7
Extreme Grammaring

• The Planning Game - scope, priorities, technical
  estimates
• Small Releases - very short release cycle
• Metaphor - shared story
• Simple Design - remove complexity when found
• Testing - continuous unit testing, test-first
  design
• Refactoring - restructure without functionality
  changes
• Pair Programming - two programmers one machine
• Collective Ownership - change each others code
  (anytime)
• Continuous Integration - build and test
• 40-Hour Week - work more produce less
• On-Site Customer - user in team
• Coding Standards - no irrelevant personal
  preferences

8
The Planning Game

• Scope
• Follow strictly the ISO VDM-SL grammar spec
• Priorities
• Disambiguate types
• Disambiguate full grammar
• Tree construction
• Technical estimates
• Not defined...

9
Small Releases

• Programmed releases (completed)
• 0.0.1 - Grammar typed from standard
• 0.0.2 - Disambiguated grammar
• 0.0.3 - AST construction
• Future Releases
• 1.0 - Haskell front-end (finished - 29-12-2004)
• 2.0 - Java front-end

10
Testing

• White box
• Structural testing
• Full visibility into how system works
• Black box
• Functional or behavioral testing
• Only the interface with exterior is available

11
Grammar Unit Testing

• Unit test
• Test a single method, function, etc...
• Different types of unit tests
• Parsing (succeeds, fails)
• Well-formness of the tree
• Test suite
• Combination of all unit test

12
Test Coverage

• Rule coverage
• Introduced by Purdon (1971)
• Explores all rules of a grammar
• Simple measure but doesnt cover all cases
• Context-dependent rule coverage
• Introduced by Lämmel in Grammar Testing
• Generalization of the above in which the context
  is taken in account
• No known implementations

13
Test Coverage Metrics

• KP - Kernel Productions
• KPr - Kernel Priorities
• S - States
• RSa - Rule Size average
• RSm - Rule Size maximum
• RC - Rule Coverage Percentage

14
Test Coverage Metrics (2)

• Although the Generics test-suite does not
  change the coverage gets lower (Injections, total
  nr rules)
• The expressions and functiontypes were only
  added in 0.0.2 version.

15
Refactoring

• Semantic preserving transformations
• Study made by Lämmel in Grammar Adaptation
• Operators
• preserve - replace a phrase by an equivalent
• fold - replace for its definition
• unfold - extract definition
• introduce - introduction of non-terminals
• eliminate - elimination of non-terminals
• rename - renaming non-terminals

16
Continuous Integration

• The integration test suite is a set of generic
  real world examples
• Only 52 coverage
• Examples are difficult to find
• Most of the examples use language extensions
• Examples
• Found on internet
• Used a pre-processor for extracting code from
  literal programs.

17
Code Standards

• Nothing found about the subject
• The following can be applied
• Limiting the number of children in a rule
• Limiting the number of alternatives in a rule
• Prefer some sort of constructs than other
• Convention for the non-terminal names
• Convention for syntax specification
• Limit module size
• ...

18
Technological Alternatives

• Most parser technologies are too restrictive
• Lex Yacc uses LALR(1)
• ANTLR uses LL(K)
• Have other problems, like
• Lexical, context-free abstract syntax separated
• Difficult to disambiguate (left-recursive
  demon)
• Grammars are technology dependent
• Solution
• Generalized LR Parsing using SDF Grammars

19
Supporting the Methodology

• SDF - Syntax Definition Formalism
• Purely declarative
• Very expressive with natural and concise syntax
• Modular structure
• Supported by Scannerless Generalized LR Parsing
• Supports compositional grammars
• Allows parsing with ambiguities (allows earlier
  testing)
• Disambiguation is separated from the grammar
  using priority and associative rules

20
SDF - Technology

• Parsing
• sdf2table, sglr
• Testing
• test-unit, ambtracker, SdfCoverage
• Tree visualization
• tree2graph, graph2dot
• Transformation
• trm2baf, implodePT
• Haskell Generation
• Sdf2Haskell (AST, Pretty Printer)

21
Syntax Definition Formalism

• Optional ?
• Repetition ,
• Simple, e.g. Identifier 2
• With separators, e.g. Indentifier ,
• Alternative

22
SDF - Example
BNF A B C B b C c
c , C
SDF B -gt A C -gt A b -gt B c , -gt C
Both grammars recognize a single b or a list of
c separated with a ,
23
Setting up the bases

• Hard copy of the ISO VDM-SL standard (ISO/IEC
  13817-1)
• Initial test suite
• Real world examples (loc 1970)
• Exercises from Formal Methods course (loc 507)
• Software
• CVS to keep track of all changes
• parse-unit (sdf unit testing tool)
• Sdf2 software bundle (sdf2table, sglr)
• SdfCoverage
• Starndard unix tools (text editor, make, ...)

24
Development cycle

• Initial grammar
• Correction
• Correct grammar rules
• Correct test suite
• Disambiguation
• Add filters
• Change grammar shape
• Steps 2 and 3 should make heavy use of testing

25
Grammar correction

• Isolate problem
• Source location
• Grammar rules involved
• Correct grammar
• Change syntax (test suite)
• Run to verify test succeeds
• Run entire test battery
• Commit Change
• Document change in message

26
Grammar Disambiguation

• Isolate problem
• Source location
• Grammar rules involved
• Create unit test
• Captures error
• Run to guarantee this
• Correct grammar
• Add disambiguation filter (change syntax)
• Run to verify unit test succeeds
• Run entire test battery
• Commit Change
• Document change in message

27
Grammar Metrics

• Simple metrics
• Total Number of Terminals (AVG per rule)
• Total Number of Non-terminals (AVG per rule)
• Complex metrics
• Introduced by Malloy in A metrics suite for
  grammar-base software
• McCabe Cyclomatic complexity
• Halstead Effort
• ...

28
Problems found

• ISO Document has ambiguities in its specification
• Syntax
• Expressions
• Apply v.s. RecordConstructor
• Apply v.s. IsDefTypeExpr
• EqualsDefinition
• CallStatement v.s. Expression
• Lexical
• Quotes are allowed in strings and in characters

29
Future plans

• Short-term (VDM parser clients)
• VooDooM
• Formal methods projects
• MCZ Objectifier
• Camila revival?
• Long-term
• Topic open for discussion...

30
Whats next?

• Test set completion (fill the rest 44)
• Test generation
• Add examples manually
• Analyze the rules that were not covered
• Try to find pathologies
• Compute Grammar Metrics
• Test the methodology developing other grammars.

31
Conclusion

• Work was completed in only 3 weeks
• A complete grammar of the ISO VDM-SL is for the
  first time public available (parser)
• A strong methodology for grammar developing was
  defined
• Grammar testing were put to practice
• Different types of tests
• Test coverage

32
Thank you!
33
Questions / Discussion