A Case Study in Database Reliability: Component Types, Usage Profiles, and Testing

1
A Case Study in Database Reliability Component
Types,Usage Profiles, and Testing

• Colin Taylor, IBM Toronto Lab, Canada
• Mechelle Gittens, IBM Toronto Lab, Canada
• Andriy Miranskyy, University of Western Ontario,
  Canada

2
Outline

• Motivation
• Database under test
• Relevant terms
• Case study
• Conclusion
• Future work

3
Motivation

• Improved DBMS reliability
• Efficient assignment of DBMS testing effort

4
Database Under Test

• Distributed enterprise relational DBMS
• gt20 MLOC
• Has evolved over gt20 years and gt10 major versions

5
Relevant Terms

• LOC
• LOCC
• Component call frequency
• Usage profiles FT and ST
• Component types Development and Architectural
• Defects

6
Case Study

• Case study presented as an analysis of our
  perceptions
• Contour plots
• Tabular summary of contour plots
• Correlation tables

7
Case Study Perceptions

• Number of defects LOC
• Number of defects Component call frequency
• For the Development component type, FT Profile
  more strongly correlated to number of defects
  than ST Profile

8
Case Study Analysis
9
Conclusion

• Weak support for Perceptions 1 and 2
• Analysis of Perception 3 inconclusive
• Component size, the size of changes, and call
  frequency are variables that impact DBMS
  reliability
• Testing effort should be assigned according to
  relational DBMS architecture

10
Future Work

• Static code complexity variable
• Component overlap

11
Colin A. Taylor

• colint_at_ca.ibm.com

12
Etcetera
13
Related Work

• Majority of studies deal with test tool
  implementation only two papers consider the
  relationships between variables that influence
  DBMS reliability
• Chays, D., Dan, S., Frankl, P. G., Vokolos, F.
  I., and Weber, E. J. 2000. A framework for
  testing database applications.
• Deng, Y., Frankl, P., and Wang, J. 2004. Testing
  web database applications.
• Suárez-Cabal, M. J. and Tuya, J. 2004. Using an
  SQL coverage measurement for testing database
  applications.
• Wu, X., Wang, Y., and Zheng, Y. 2003. Privacy
  preserving database application testing.
• Gittens, M., Lutfiyya, H., Bauer, M., Godwin, D.,
  Kim, Y. W., and Gupta, P. 2002. An empirical
  evaluation of system and regression testing.
• Sullivan, M., Chillarege, R. A comparison of
  software defects in database management systems
  and operating systems.