Testing with CTest and CDash

1
Testing with CTest and CDash

• Julien JomierKitware Inc.

2
Testing Overview
Developers
CVS/SVN
Web Browser
CTest
CDash
3
Testing Terminology
Sites
Builds
dash5.kitware
Linux-g
Linux-gcc4
Linux-gcc3
iris.elmtech
Dashboard
4
Why Test Daily?

• Large code base too large for any single
  developer to understand
• Developers distributed around the world
• Identify problems as they occur
• Insure that object API remains unchanged
• Provide feedback to developers as they experiment
  with new implementations

5
If its not tested it doesnt work

• Dasboards and Software Process
• Roots in GE Six Sigma initiative
• Cross platform testing
• Nightly regression testing Memory
  (Purify/ValGrind) testing
• Coverage testing
• Style checking
• Documentation evaluation

6
How to write a good test?

• Use empirical values (5/22.5)
• Test most of the use cases (even the boring/most
  difficult ones)
• Make your tests as efficient as possible
• Have someone else write the test for your
  algorithm

7
Regression Testing

• Compare generated image against standard
  correct image
• pixel-by-pixel comparison
• can use a threshold metric
• adjusted for effects like dithering

8
CTest

• Testing client
• Distributed with CMake
• Submit test results to the dashboard
• XML submission
• Independent of CMake

9
CTest with CMake

• CMakeLists.txt
• INCLUDE(CTest)
• ENABLE_TESTING()
• Add a simple test
• ADD_TEST(CDashTest1 CDashTest CDashTest)
• ADD_EXECUTABLE(CDashTest main.cxx)
• CTestConfig.txt
• set(CTEST_PROJECT_NAME "MyProject")
• set(CTEST_NIGHTLY_START_TIME "210000 EDT")
• set(CTEST_DROP_METHOD "http")
• set(CTEST_DROP_SITE "localhost")
• set(CTEST_DROP_LOCATION "/CDash/submit.php?project
  MyProject")
• set(CTEST_DROP_SITE_CDASH TRUE)

10
Testing Submission
Compiler/Tester
CDash
CTest
Backup XML File
HTTP PUT
Fill Database
XML Submission
XML Parsing
Update.xml Configure.xml Build.xml Test.xml Covera
ge.xml DynamicAnalysis.xml
Send Email
Update RSS Feed
11
CDash

• New generation of Dart
• Written in Php/Javascript
• MySQL
• Open Source
• Run on top of a web server
• XSL implementation
• Web 2.0

12
CDash Features

• Ajax requests
• Graphs build, test times
• Builds group
• User management
• Email notifications
• Timing defects

13
Coverage Support

• Email Low Coverage
• Bullseye support

14
Build Notes

• Share with other developers
• Keep it simple

15
My CDash

• Personal Page
• Subscribe to project
• View sites
• Email preferences

16
Project Roles

• Normal user
• Site maintainer
• Project administrator

17
Claiming Sites

• Site information
• Automatic (CTest 2.6)
• Information history
• Manual editing
• Personal view/email

18
CDash Misc.

• Automatic Geolocation from IP addresses
• Support for iPhone

19
Installing CDash

• (L/W)AMP Apache, MySQL, PHP- XAMPP, EasyPHP
• Download CDash (www.cdash.org) in your Apache web
  directory
• Edit config.php
• Point Web Browser to CDash- Setup the database-
  Create first project

20
Style Checking

• KWStyle
• C Style checker
• Configuration via XML
• HTML Generator (TBI in CDash)
• Open Source
• Written in C

21
Contribute

• Website- http//www.cdash.org-
  http//public.kitware.com/KWStyle
• Bug Tracker- http//www.cmake.org/Bug
• Mailing List- http//public.kitware.com/cgi-bin/m
  ailman/listinfo/cdash

22
Happy Testing!