Refining%20of%20Failure%20Detection%20Technique%20in%20Web%20Applications

1
Refining of Failure Detection Technique in Web
Applications
Tomohiro Nakamura, Greg Friedman, Peter Bodík

• Refining Failure Detection Technique for Web
  Applications developed by Greg Friedman
  (Stanford), Peter Bodik (Berkeley) etc. using
  more features of Web application log
• Apply new feature
• Detect anomaly by Chi-Squared testing on page
  transition intervals
• Compare characters of the new feature and the
  previous (page hit frequencies)
• Page transition intervals is useful to detect
  failure earlier ?
• Page transition intervals has almost same
  detection ratio ?

2
Background

• Failure Detection Technique in Web Application
  presented at 2004 winter ROC retreat
• Detect anomalies every minute by comparing page
  hit frequencies from last 6 hours vs. last
  several minutes
• Use Chi-Squared testing on page hit frequencies
  to detect anomalies
• Show the most anomalous page and the page
  transition from/to the page to localize the
  failure
• The result of testing on 5 sets of HTTP logs
• Failure detection rate 100
• 7 days 1 hour earlier notification of the
  failure
• Failure localization rate 40

3
Basic idea

• Failure Detection Technique in Web Application
  presented in 2004 winter ROC retreat is powerful
• But only page hit frequencies are used to detect
  failure
• HTTP log has more features which will be
  meaningful
• In some case, stats of page transition will
  change before anomaly found on page hit
  frequencies and using the stats makes detection /
  localization earlier / better
• The stats of page transition intervals (average
  time between page hits of each session) is used
  for anomaly detection

4
Basic idea (cont.)
Time
History
Current
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Average interval time From Page A To Page
B From Page B To Page C
Average
TAB
tAB
TBC
tBC
Compare interval times of History and Current
(TAB - tAB ) 2
(TBC - tBC ) 2

Chi2-Squared Test
Chi2
TAB
TBC
5
Example(1) broken signup page

• Both detect 7 days earlier (Page transition
  intervals 8 hour earlier)

6
Example(2) landing looping problem

• Page transition intervals detect earlier (with
  some FP)

FP false positive
7
Example(3) account page problem

• Page hit frequencies detect failure 4 hours
  earlier

8
Conclusion Future work

• Pros and Cons of Failure Detection using Page
  transition interval
• PRO
• Detect failures earlier in many cases (some cases
  not)
• Detection ratio is likely to be almost same as
  the previous
• CON
• False positive ratio is higher in some cases
• Significance range is lower than the previous in
  general(difficult to show severity of the
  detected failure)
• Future work
• Robustness verification (ex. Multiple log files,
  Errors in log files)
• Investigate Ensemble Method (each method has Pros
  Cons)
• Investigate failure detection coverage of these
  tool
• Other features / algorithms to detect failures
• What else?