Improving Software with the UW Metronome

1
Improving Software with the UW Metronome

• Becky Gietzel
• Todd L Miller

2
Three Ways to Improve Software

• Automated Builds and Tests
• Continuous Integration
• Platform and Version Testing

3
Talk Overview

• Introduction to the UW Metronome
• Use Case Condor Developers
• Getting Started with the UW Metronome

4
The Metronome at UW

• An automated build and test facility at
  UW-Madison
• Infrastructure (BaTLab) maintains services and
  fifty distinct platforms.
• Software (Metronome) reproducibly and reliably
  runs builds or tests of your software.

5
Metronome Philosophy

• Reproducibility.
• The benefits of reproducible testing need not be
  further elaborated.
• Reproducible builds
• dont have to be stored (for very long).
• dont require a specific person or machine to
  build.
• simplify life-cycle management.
• Support, rather than restrict, users.

6
Advantages of Metronome

• Not tied to a particular tool or language.
• Automatically-generated database of build and
  test results, with web interface.
• Uses Condor to distribute runs.
• The UW Metronome maintains an infrastructure on
  your behalf.

7
Metronomes Web Interface
8
Usage Concepts

• Platforms
• Prerequisites
• Input Output Specifications
• Ordered Commands

9
Ordered Commands
Execute Node
Execute Node
Submit Node
Execute Node
pre_all
remote_pre_declare
remote_declare
platform_pre
platform_pre
platform_pre
remote_pre
platform_post
platform_post
platform_post
remote_task
remote_post
post_all
10
Use Case Condor Developers

• Fully automatic builds and tests.
• Separate build and test runs.
• Easy to see if build or test failed.
• Easy to fetch a particular build.
• Easy to develop the test suite.
• Enables cross-platform testing.
• Used for Condor release binaries.

11
Condor Developers (2)

• Nightly builds and tests.
• Hourly runs on a subset of platforms.
• Windows/Linux pairing especially helpful.
• Tests patched OSes.
• Still want lower-latency testing.

12
Condor Developers (3)

• Nice features.
• Printing URL of run results on submission.
• Streaming user-level output and error.
• Big features.
• Speed improvements on submit nodes.
• Concurrent testing.
• Major web interface additions.

13
Getting Started

• Metronome documentation
• nmi.cs.wisc.edu
• Build and Test Lab info
• nmi.cs.wisc.edu/account_request
• nmi.cs.wisc.edu/support
• Live Lab status
• nmi-web.cs.wisc.edu/nmi
• Any questions?