Formal Methods in Vulnerability Testing

1
Formal Methods in Vulnerability Testing

• O. Monkewich
• R.L. Probert

2
Acknowledgements

• The authors gratefully acknowledge the support
  for this research by Industry Canada Protocol
  Analysis Laboratory and appreciate suggestions
  for improvements from their engineering group,
  including
• Deputy Director General, William McCrum
• Lewis Robart
• Colman Ho
• David Gibson
• Peter Chau
• Partial support from NSERC and CITO is also
  gratefully acknowledged

3
Introduction

• Work carried out for Industry Canada on Voice
  over IP (VoIP) for public networks
• New Methodology for Vulnerability Testing
• Automation based on Formal Languages in
  conjunction with commercial tools
• Test Suite automatically generated/executed
• Test generation guided by heuristic rules
• Session Initiation Protocol (SIP) as example

4
Why Our Approach is Different

• Formal, but accessible to all, languages and
  methods
• Use of international standards
• Excellent commercial support tools
• Formal models of protocols with validation,
  simulation
• Formalized methodology for vulnerability testing
• Heuristic vulnerability search methods
• Includes searches for semantic vulnerabilities as
  well as syntactic
• Ongoing studies successfully illustrate and
  validate our approach

5
What are Formal Methods

• Methods that Combine
• Formal Languages
• Formalized Methodology
• Sound Engineering Principles
• Computer Automation
• International Standards
• Enhance Design, Specification and Testing

6
What are Formal Languages

• Languages defined by means of mathematics
• can prove correctness mathematically
• defined in international standards
• Why different from C programming language
• C components do not map well onto mathematical
  domains

7
Which Formal Languages

• Specification and Description Language (SDL)
• Message Sequence Charts (MSC)
• Abstract Syntax Notation One (ASN.1)
• Testing and Test Control Notation (TTCN)

8
Methodology

• Formal Languages and Tools
• means for writing specifications and models that
  are understood by the computer
• specifications are executable
• validated by the computer
• simulated to show behaviour to the designer
• tests generated automatically
• no errors or hidden programming side effects
• Heuristic rules guide the computer to seek out
  vulnerabilities
• all vulnerably, syntactical and semantic

9
The Formalized Process
10
Test Configuration
11
Sample Formal Specification
12
Vulnerability Heuristic

• Protocol is a Finite State Machine (FSM)
• it performs a function while in one state
• then moves on to the next state
• A protocol is more vulnerable in one state than
  another
• A risk factor is assigned to each transition to
  the next state
• Automatic navigation through the FSM is guided by
  the risk factors assigned to each state

13
Heuristic Rule

• The risk value of NextState is Score(NextStatei)
• Risk to perform a protocol function is given by
  Score gt maxScore(NextStatei)
• A test scenario is a pass through one branch of
  FSM from root node to leaf node
• Vulnerability test scenario is a pass through a
  high risk branch
• A Vulnerability Test Suite is a set of executable
  test scenarios to test product vulnerability

14
SDL Model with HeuristicScores
15
Tracing the Test Path

• A segment of protocol behaviour is a path from
  state-to-state based on inputs and decisions
  along the way
• many possible paths
• may be an unpredictable sequence
• vulnerability risk value may accumulate
• Both, semantic and syntactic errors must be
  factored into the risk calculation
• Vulnerability risk value of a path is too
  difficult to determine without a proper
  methodology

16
SDL no HeuristicScores
17
Test Generation with Tree Walk
18
Low Vulnerability Test Cases
19
Higher Vulnerability Test Cases
20
SDL Processwith HeuristicScores
21
Test Generation with Tree Walk
22
Higher Level Vulnerability Test Cases Only
23
Two non-test cases
24
The Test Suite

• The preceding test cases in Message Sequence
  Charts are automatically translated to the
  Testing and Test Control Notation (TTCN)
• Individual test cases are integrated into a
  single test suite with common variable
  declarations, structure and platform interface
• TTCN is translated to ANSI C and compiled on a
  suitable test platform

25
Test Case Execution

• All test cases are executed in a sequence
  specified in the TTCN test suite
• Test cases are selected or deselected at run time
  (based on protocol options implemented)
• Verdicts automatically assigned without the need
  for human analysis of the test data
• Vulnerability Found
• Vulnerability Not Found
• Inconclusive
• Test log of all test events is generated
  automatically for detailed analysis if necessary

26
Value to Industry

• Perform tests privately in industrys own
  laboratories
• Correct vulnerabilities before product goes into
  the field
• Use a common core set of tests that is
  internationally standardized
• Add additional test cases specific to
  organizations interests
• Industry organizations have indicated interest

27
Conclusions

• Modelling communication protocols using formal
  languages is well understood today
• Paths through the protocol when providing a
  service
• numerous
• unpredictable
• have different vulnerability risk values
• Test generated for semantic and syntactic
  vulnerabilities
• automated, controlled by heuristic rules
• heuristic algorithms can be developed to seek out
  most vulnerable paths through the protocol
• Formal methods are machine intensive
• greatly reduce unintended errors
• reduced risk of intended malicious tampering

28
Main Source of Formal Languages and Methodology

• http//www.itu.int/ITU-T/studygroups/com17/index.a
  sp

29
THE END