Introduction to Software Build Technology

1
Introduction toSoftware Build Technology

• Philip Johnson
• Collaborative Software Development Laboratory
• Information and Computer Sciences
• University of Hawaii
• Honolulu HI 96822

2
Objectives

• Understand motivation for build and packaging
  technologies.
• Why was running your classmates Robocode robot
  so hard?
• Become familiar with a sample of build
  technologies.
• Get introduced to the build technologies and
  standards to be used in this class.

3
Motivation

• Modern team-based, cross-platform development
  involves
• Multiple developers and users.
• System is developed/used on multiple platforms
• Win2K, Linux, Win98, Solaris
• Developers/users have different environments
• c\jdk1.5.0_08, d\jdk1.5, c\myjava
• Differences between user and developer
  distributions
• Source distribution for developers
• Binary distribution for users
• Developers may prefer different IDEs
• Eclipse, JBuilder, Emacs, RationalRose
• Users may not have any IDE at all.
• Frequent releases and distributions.
• Dependence on third party libraries (which may
  themselves be released frequently).
• These issues are not unique to Java development!

4
Motivation (cont.)

• Build/packaging techniques should allow
• Users to build and test the system without IDE.
• Developers to build and test the system using
  their IDE of choice.
• Developers to build and test the way users build
  and test.
• Configuration to different platforms and
  different environments within a platform.
• Multiple release configurations
• Automated build to prevent errors
• Rapid release, deployment, testing
• Dependency management for 3rd party libraries

5
A quick tour of build tools
6
(No Transcript)
7
Make

• Make is the canonical build tool.
• Make features
• File-level dependency analysis
• Example compile only changed files
• Unix-based (but ported to other platforms)
• Language independent (Java, C, etc.)
• ASCII configuration file syntax
• Tab characters requiredYikes!
• Supports invocation of any shell command.
• Does not support library-level dependency
  management.
• Related imake, cmake, scons

8
(No Transcript)
9
Ant

• Java-based build tool by Apache
• http//ant.apache.org/
• Features
• Java-based
• Cross-platform
• Extensible
• XML configuration files
• Open source
• Dependency management provided by Ivy.

10
(No Transcript)
11
Maven

• Project Management technology
• Build based on Ant.
• Documentation and reporting plugins.
• File-level dependency management
• Library-level dependency management
• Library repository definition management
• Convention over configuration
• Maven builds in conventions for project layout
  and workflow to simplify use.
• Great if you like them.

12
Ant and Maven

• Both based upon XML configuration files.
• Both provide a high-level "Domain Specific
  Language" for building systems.
• DSL makes assumptions about what you want to do
  and how you want to do it.
• Extensible via "plugins"
• More difficult to write

13
Rake
14
Rake

• "Thin coating" of build functionality on top of
  Ruby language
• Predefined libraries for packaging.
• Prerequisites for dependencies.
• Not a DSL
• If you know Ruby, you're good to go.

15
Buildr
16
Buildr

• A "rake for Java"
• Underlying language is Ruby
• Understands Java files and Maven file structure

17
rake and buildr

• Do not attempt to provide a DSL for builds.
• Requires you to (eventually) learn a general
  purpose programming language (Ruby).
• No "plugins" required---just write code for your
  special situations.

18
The basic divide

• Domain-specific language

• Syntactic Sugar

• Ant
• Maven

• Make
• Rake
• Buildr

Read "Why everyone (eventually) hates (or leaves)
Maven for a good discussion of the divide DSLs
get you farther, faster, initially Ultimately,
the DSL gets in your way as your needs get more
sophisticated.
19
Our approach

• We will use Maven
• You get "farther, faster" using Maven.
• Many projects use it.
• Our projects do not have (many) idiosyncracies
  that make Maven life difficult.

20
Basic Ant concepts

• Every project requires an Ant build file
• named build.xml by default.
• Each build file is composed of targets.
• Typical targets compile, junit, etc.
• Each target is composed of tasks
• Ex copy .java files to build/src, invoke
  javac...
• Targets can have dependencies
• Ex modified sources must be recompiled before
  testing.

21
Ant targets

• Dependent targets are executed exactly once.
• Example
• test depends upon compile
• deploy depends upon compile
• all depends upon test, deploy
• Compile will be executed only once.
• Some targets are executed only when needed.
• Sources recompiled only if changed.

22
Example Ant target

• lttarget name"compile"
• dependsinit"
• description"Compiles code."gt
• ltjavac srcdir"src.dir"
• destdir"build.dir/classes/gt
• lt/targetgt

23
Best practices

• Ant provides a simple, powerful language for
  building Java-based systems.
• How do we use this language effectively?
• The next section describes my best approach so
  far on how to employ Ant to achieve the Three
  Prime Directives of Open Source Software
  Engineering.
• The robocode-pmj-dacruzer example project
  provides you with a template structure to get you
  off and running quickly.

24
robocode-pmj-dacruzer An example Ant-based
build system

• Implements a pretty lame Robocode robot.
• Not very interesting.
• Its the build system thats interesting
• Provides a template
• Easy to adapt to your own projects.
• Integrates third-party build tools
• Checkstyle, PMD, FindBugs, etc.
• Is modular and extensible
• You can grow or shrink it to your needs.
• Well suited to build automation and project
  hosting
• Continuous integration (Hudson)
• Uses Ivy to download/install/manage third-party
  libraries.
• Result is Maven-ish, but without the black box
• If you want to change something, its just Ant
  code.

25
Top-level directory contents

• build.xml
• top-level build file.
• .build.xml
• module build files, generally one per tool.
• src/
• the system source files
• lib/
• library files maintained by Ivy.
• build/
• derived files generated by build process

26
Build operations

• compile
• compile system
• jar
• create jar file
• junit
• run junit tests
• javadoc
• build javadoc docs
• clean
• delete build dir.
• dist
• create zip dist file.

• checkstyle
• analyze source code
• findbugs
• analyze byte code
• pmd
• analyze source code
• sclc
• compute size of system
• jacoco
• compute coverage of test cases.
• (Others to be added later in semester!)

27
Build System Scalability Issues

• Modularity
• putting all targets in one build.xml file creates
  a very large build.xml file that is difficult to
  understand and maintain.
• Incrementality
• you may not want to require all developers to
  have to install every possible third party tool.
• Coupling
• dependencies between target and property
  definitions should be minimal and easily visible.

28
Build system architecture
.build.xml files import the build.xml file and
provide an implementation of a single extension.
pmd.build.xml
build.xml contains most project-specific definiti
ons.
findbugs.build.xml
dist.build.xml
build.xml
checkstyle.build.xml
javadoc.build.xml
sclc.build.xml
common.build.xml
.build.xml
boilerplate code
29
build.xml

• build.xml provides most (but unfortunately not
  all) of the project-specific definitions
• version number
• dependency libraries
• project name
• etc.
• Most other .build.xml files can be used with
  little to no changes.
• Typical exceptions junit.build.xml,
  emma.build.xml, jar.build.xml

30
.build.xml responsibilities

• Implements support for a single tool/function
• Checkstyle, Jar, JavaDoc, PMD, Distribution, etc.
• Typical tasks (assume a tool called 'foo')
• Download foo using Ivy, install in lib/foo
• Download a configuration file for foo, store in
  lib/configfiles.
• Provide three standard tasks
• foo.tool runs foo tool, generates data.
• foo.report creates HTML file.
• foo runs foo.tool and foo.report.
• Make 'foo' the default task.

31
User Interface

• To do initial installation and compilation
• ant
• For other operations, use -f ltfilenamegt
• ant -f dist.build.xml
• ant -f junit.build.xml
• ant -f checkstyle.build.xml
• ant -f pmd.build.xml
• ant -f emma.build.xml
• etc.

32
Considerations

• This approach provides the following
• Minimal coupling .build.xml files depend only
  on definitions in build.xml
• Modularity individual .build.xml files are
  small.
• Reporting "report" targets provide details on
  what the "tool" targets generated.
• One problem
• How to "verify" that all analyses "passed"?

33
verify.build.xml

• Provides one stop shop for system analysis
• Imports not just build.xml, but other .build.xml
  files as well.
• Runs all "tool" analyses, failing build if they
  generate any warnings/errors.
• Compile, Junit, PMD, Checkstyle, JavaDoc,
  FindBugs
• User interface
• ant -f verify.build.xml

34
The lib/ directory

• Contains third-party libraries required by
  project
• robocode
• checkstyle
• pmd
• findbugs
• etc.
• Contains configuration files for analysis tools.
• lib/configfiles/checkstyle.modules.xml
• lib/configfiles/pmd.rulesets.xml
• lib/configfiles/findbugs.filter.xml

35
The build/ directory

• Contains files derived from build process.
• Can be deleted and reconstructed.
• Not included in version control or distribution
  .zip.
• Build subdirs Targets creating them
• build/javadoc/ javadoc
• build/classes/ compile
• build/dist/ dist
• build/sclc/ size
• build/junit/ junit

36
build/ vs. bin/

• Both Ant and Eclipse (or any other IDE) compiles
  files.
• It causes problems if Ant and Eclipse are using
  the same output directories for compilation/etc.
• Therefore, Eclipse writes to the bin/ directory,
  not the build/ directory.
• Set in Project Properties Java Build Path
  Source

37
Testing Notes

• There are two kinds of tests in this system
• Manual QA using JUnit
• Ensures your code satisfies its specifications
  (if you write good test cases.)
• Automated QA using Checkstyle, FindBugs, PMD
• Ensures your code obeys best practices for coding
  style.
• Encourages incremental testing and formatting.

38
Traditional release numbering

• Traditional approach (i.e. jakarta-tomcat-5.1.0.zi
  p)
• MM.mm.bb, where
• MM Major release number(major redesign or
  incompatible changes)
• mm Minor release number(minor enhancements,
  backward compatible)
• bb Bugfix release number(bug fix, no new
  functionality)
• All numbers maintained manually.
• Advantage
• Release number comparison indicates degree of
  change.
• Disadvantage
• Requires manual editing of release value every
  time which leads to errors (easy to forget to do
  this.)

39
Our release numbering

• Our approach (i.e. robocode-pmj-dacruzer-1.3.2011.
  09.11.12.32.zip)
• Major.minor.timestamp, where
• Major major release number
• Minor minor release number
• Timestamp time of bugfix release
  (yyyy.mm.dd.hh.mm)
• Major and minor release number maintained
  manually.
• Bugfix timestamp generated automatically.
• Advantages
• Automatic except when incrementing major/minor
  release.
• Release number provides timestamp of release.
• Disadvantages
• Number of bugfix releases not indicated by
  release number.
• Very long version number.

40
IDE integration

• Important to maintain independence from IDE.
• All build functions should be possible without
  any IDE.
• But its fine to provide IDE-specific files to
  facilitate development.
• Example .class and .project files in top-level
  directory for Eclipse.
• Provide additional files/directories for other
  IDEs if desired.

41
robocode-pmj-dacruzer as a standard

• From now on, all projects must be developed,
  packaged, and submitted using the standard build
  conventions embodied in this package.

42
robocode-pmj-dacruzer as a template

• You can use robocode-pmj-dacruzer as a template
  for your own projects
• Download a fresh version.
• Remove dacruzer-specific info
• Add your project info.
• See the template instruction page in the course
  website for step-by-step instructions.

43
Ant/Eclipse integration

• Eclipse has nice support for Ant, but
• You will also need to tell Eclipse about
  ANT_HOME
• Window Preferences Ant Runtime Ant Home
• Make sure Eclipse writes to the bin/ directory
• Project Properties Java Build Path Source

44
Limitations of example

• The robocode-pmj-dacruzer template provides a
  framework plus basic build functionality, but has
  limitations
• No SVN integration
• No support for web app development.
• No support for automated metrics collection.
• We will enhance this initial template during the
  semester.

45
Meta-level goal

• Ant provides an efficient solution to satisfying
  both
• PD2 (users can install system) and
• PD3 (developers can understand and enhance
  system)
• Other approaches exist
• InstallAnywhere for PD2 (more work)
• IDE-specific solution for PD3 (limited)
• With good installation documentation, I believe
  Ant-based build can satisfy both PD2 and PD3.

46
(No Transcript)