Using Static and Dynamic Application Analysis Tools to Identify Areas of Risk

1
Using Static and Dynamic Application Analysis
Tools to Identify Areas of Risk
Allen Stoker Indianapolis Quality Assurance
AssociationOctober 13th, 2006
2
Static and Dynamic Analysis

• Static Analysis
• The process of evaluating a system or component
  based on its form, structure, content, or
  documentation
• Static analysis is performed by evaluating the
  source artifacts of an application
• Dynamic Analysis
• The process of evaluating a program based on its
  behavior during execution
• Dynamic analysis is performed using tools that
  are integrated with the runtime environment that
  track the activity of the program while it is
  running

3
The Value Proposition

• Static analysis is a low cost, low effort
  activity that provides insight to application
  characteristics
• Identification of high risk code segments
• Indication of execution paths can help resource
  planning
• Dynamic analysis provides a perspective on test
  effectiveness as well as exposing indications of
  non-functional issues
• Performance
• Scalability
• Reliability
• The integration of developers and testers

4
Preparing for Analysis

• Understand the availability of information and
  define achievable goals
• Select appropriate tools
• Identify tools based on the alignment of
  capabilities
• Many development environments include built-in
  tools
• Google static analysis tool, dynamic analysis
  tool
• Purchased tools may provide more advanced
  capturing, analysis and reporting of the
  information
• Establish the Analysis process
• Determine when and how the process will be run
• Establish a repository of data for later trending

5
Static Analysis Demonstration

• IntelliJ Idea 5.1.2
• Metrics Reloaded plug-in
• http//www.sixthandredriver.com
• Pre-built Library Application (Java)
• QStudio Java Static Analysis (QJ-pro)
• http//www.qa-systems.com

6
Static Analysis Demo II
7
Static Analysis Demo III
8
Static Analysis Demo IV
9
Key Static Analysis Outputs

• Cyclomatic Complexity
• A measurement of the intricacy of a program
  module based on the number of repetitive cycles
  or loops that are made in the program logic
• This metric identifies unique paths through the
  module
• Lines of Code
• Lines per module (class)
• Lines per function (method)
• Relative Lines per function (method)

10
Preparing for Dynamic Analysis

• General preparation for analysis applies
• Understand the runtime environment
• Knowledgeable technical resources will likely be
  required to establish the runtime environment
• Ongoing development resources may be required to
  support the process over time
• May require hardware and/or special configuration
  to run
• Acquire analysis tools
• Higher dependency on runtime environment
• Fewer free tools
• More Tool reuse between developers and test
  analysts

11
Dynamic Analysis Demonstration

• Apache Tomcat (Application Server)
• EMMA Code Coverage tool (open source)
• http//emma.sourceforge.net
• Pre-built Library Application (Java)

12
Dynamic Analysis Demo II
13
Dynamic Analysis Demo III
14
Dynamic Analysis Demo IV
15
Dynamic Analysis Demo V
16
Key Dynamic Analysis Outputs

• Code Coverage
• Traces the actual execution path and identifies
  the lines of code executed
• Non-executed code indicates untested branches (or
  unreachable code)
• Performance Profiling
• Measures the amount of time accumulated in
  monitored application functions
• Memory Leak Detection
• Measures memory utilization across function
  executions to trace leakage

17
Applying the Results

• Question units with higher than normal complexity
  metrics
• Review code with low coverage percentage to
  identify needs for more test cases
• Track metrics over time to monitor trending
• Dont draw final conclusions from coverage
  metrics alone
• Coverage results do not account for data value
  test coverage (ranges, boundaries, etc)
• Build reporting into an automated process

18
Value Recap

• Static analysis provides a low cost, low effort
  reporting that can identify complex components
  requiring more intensive testing
• Static analysis can provide information about the
  source code useful in resource planning
• Dynamic analysis can be more difficult, but may
  help identify untested code and various runtime
  issues including performance and reliability

19

• Questions?

20

• Thank You!

• Allen Stoker
• Allen.Stoker_at_PortalWizard.com