The GNU Autotools: autoconf, automake, (libtool)

1
The GNU Autotoolsautoconf, automake, (libtool)

• Dr. Robert P. Goddard
• Applied Physics Laboratory
• University of Washington
• Robert_Goddard_at_apl.washington.edu
• Presentation for the Northwest C Users Group
• 8 September 2004

2
Outline

• Who needs Autotools?
• Open Source Software
• Roles of software users
• Files in a source distribution
• The configure script
• Autoconf, configure.ac
• Automake, Makefile.am

3
Who needs GNU Autotools?

• Use GNU Autotools if you develop software
  satisfying several of these attributes
• Distributed as source code, built and installed
  by others
• Must be portable to multiple platforms
• Developed by a team
• Complex enough to require multiple Makefiles in
  multiple directories
• Written in multiple languages
• Long lived, requiring many update cycles
• Alternatives
• Hand-crafted Makefiles
• Multiple Makefiles, one per target platform
• Configuration header files (config.h) and/or D
  switches
• ifdef, if
• Integrated development environments
• IMake

4
(Open) Source Software

• Distributed to (some) users as source code
• Portable, at least multiple Unix and Windows
• Can be modified by (some) users
• Documented, at least for end users
• Maintained, at least bug fixes
• Easy to build from source and install
• (Licensed for redistribution www.opensource.org)
• Examples OSF (GNU), Linux, X Windows, Boost,
  PVM, LAM/MPI, ACE, Loki, NEdit, SourceForge, TeX,
  FFFTW, (SST),

5
Roles of Software Users

• End User
• Installer
• Porter
• Distributor
• Developer
• Configuration Guru

• Differ by
• Expectations
• Expertise
• Tasks to accomplish
• Services needed
• Tools available
• Operating system
• Patience
• Willingness to learn
• Tolerance of inconvenience

6
Development Use Cases
7
End User

• Expects
• Executables on path
• Dynamic libraries where executables expect them
• Documentation findable and complete
• Software works on his or her platform
• Plus, for libraries
• Header files where compilers expect them
• Libraries where linkers expect them
• Platform Requirements
• For libraries, compilers and linker
• Otherwise, only basic OS services
• Some packages may require more, e.g. a
  third-party library

8
Installer (from source)

• Expects simple, standard procedure
• Download source package
• Unpack, normally with gunzip and tar
• configure
• make
• make check
• make install
• Expects some flexibility
• Where installed
• Which compilers
• Compiler/linker options
• Optional features

• Platform requirements
• Network connection
• Gzip and tar
• Compilers Linker
• GNU make
• Bourne-compatible shell
• Maybe test tools etc.
• Skill requirements
• How to run make
• What compiler options do
• Platform software installation requirements

9
Developer

• Expectations
• Multiple configurations, single code base
• Implies Separate source and build directories
• Complete set of standard make targets (without
  writing them!)
• Support for multiple
• Source directories
• Make targets
• Languages
• Support for non-standard tools (e.g. code
  generators, scripts)
• Automatic updating of include dependencies

• Platform Requirements
• Automake, autoconf, (libtool)
• GNU m4, Perl 5
• Source code control (CVS)
• Probably many other tools
• Skills Required
• Software design, programming, testing, debugging,
  source control, teamwork, planning, etc.
• Simple modifications to Autotools input files
• Simple make
• Documentation
• Skills Not Required
• Configuration subtleties

10
Autotools Overview (no libtool)
Under source control
Starting, occasional
Source files
autoscan
configure.scan
configure.ac
configure.ac
autoconf
configure
Developer, on development system
acinclude.m4
autoheader
config.h.in
distribute
automake
Makefile.am
Makefile.in
Source files
configure
config.cache
Installer, on target system
config.h.in
config.log
config.h
Makefile.in
make
config.status
Makefile
Executables, libraries, documents, etc.
End User
11
Source Distribution Package

• Makefile.in
• Makefile with special _at_XX_at_ variables
• One per source directory
• Generated by automake
• configure
• Bourne shell script. Substantial.
• One at top of directory tree
• Tests for platform capabilities, sets macro
  values, generates files (right)
• Generated by autoconf
• README, AUTHORS, COPYING
• Specific instructions for your package. You must
  write these!
• aclocal.m4, config.guess, config.h.in, depcomp,
  install-sh, missing, mkinstalldirs, etc.
• Generated automatically. Ignore them.
• INSTALL
• Standard instructions for building standard
  software. Always the same.
• Your source files, in directory tree

12
configure

• Bourne shell script
• Generated by autoconf, distributed with source,
  run on target computer
• Performs tests for working compilers, options,
  programs, libraries, headers, structures,
  functions, etc. on the target computer.
• Sets output variables and C preprocessor macro
  definitions based on tests, options,
  configure.am.
• Transforms input files (usually Makefile.in to
  Makefile) by substituting output variables for
  _at_XX_at_ placeholders.
• Writes the C macro definitions into either
• a header file (usually config.h) as define XX
  value, or
• a _at_DEFS_at_ output variable as DXXvalue for the
  Makefiles.

13
Files Generated by configure script on target
computer

• Build Tree
• Subdirectory for each one in source tree
• If configure is run in a separate build directory
• Makefile from each Makefile.in
• By substituting macro values for _at_XX_at_ variables
• Include all GNU targets and a lot more. Huge!
• config.h
• Include file defining C preprocessing macros
• Optional macros may be defined on command lines
  instead
• Config.log, config.status, maybe others
• Ignore these unless something goes wrong during
  configuration

14
Running configure

• SSTSRC/configure --help
• Prints generic instructions, plus options added
  by developer
• mkdir sst_gcc_gpg
• cd sst_gcc_gpg
• SSTSRC/configure CXXg \
• CFLAGS-g -pg \
• --prefix/usr/local/sst
• Actually, I put the configure invocation into a
  config.local script, with instructions to copy it
  into each new build directory and edit it.

15
configure output(example from SST under Cygwin)

• checking for a BSD-compatible install...
  /usr/bin/install -c
• checking whether build environment is sane... yes
• checking for gawk... gawk
• checking whether make sets (MAKE)... yes
• checking build system type... i686-pc-cygwin
• checking host system type... i686-pc-cygwin
• checking for /usr/local/include/cppunit/Asserter.h
  ... yes
• checking for pdflatex... yes
• checking for gcc... gcc
• checking for C compiler default output... a.exe
• checking whether the C compiler works... yes
• checking whether we are cross compiling... no
• checking for suffix of executables... .exe
• checking for suffix of object files... o
• checking whether we are using the GNU C
  compiler... yes
• checking whether gcc accepts -g... yes
• checking for gcc option to accept ANSI C... none
  needed
• checking for style of include used by make... GNU
• checking dependency style of gcc... gcc3

• configure creating ./config.status
• config.status creating Makefile
• config.status creating src/Makefile
• config.status creating src/tools/Makefile
• config.status creating src/dsp/Makefile
• config.status creating src/sigproc/Makefile
• config.status creating src/math/Makefile
• config.status creating src/sonar/Makefile
• config.status creating src/ocean/Makefile
• et cetera
• config.status creating src/main/Makefile
• config.status creating support/Makefile
• config.status creating support/examples/Makefile
• config.status creating support/examples/tarmod/Ma
  kefile
• config.status creating support/JUA_Paper/Makefile
  
• config.status creating support/scripts/Makefile
• config.status creating support/scripts/certifySST
  .sh
• config.status creating support/scripts/webzip.sh
• config.status creating support/sioplot/Makefile

16
autoconf, autoreconf

• Input configure.ac, which contains m4 macro
  invocations and shell commands
• Output configure script, consisting of macro
  expansions and your shell commands
• Autoreconf runs extras too aclocal, autoheader,
  automake, etc.
• General pattern Each macro generates shell code
  to test whether some tool, library, or header is
  present and working. If so, it sets output
  variables and/or C macros to record facts needed
  by Makefiles etc.
• AC_OUTPUT generates code to substitute output
  variable XX for _at_XX_at_ in YY.in, generating YY

17
Simple configure.ac

• AC_INIT
• AC_CONFIG_SRCDIR(main/foonly.cc)
• AM_INIT_AUTOMAKE(foonly, 1.0)
• AC_PROG_CC
• AC_PROG_CXX
• AM_PROG_LEX
• AC_PROG_YACC
• AC_CONFIG_FILES(Makefile lib/Makefile \
• main/Makefile)
• AC_OUTPUT

18
Example configure.ac

• AC_INIT
• AC_CONFIG_SRCDIR(src/main/sst.cc)
• AM_INIT_AUTOMAKE(sst, 2004.4)
• dnl Put the preprocessor macros in a header, not
  on command line
• AM_CONFIG_HEADER(config.h)
• dnl Put the executables in a subdirectory of
  SSTHOME
• AC_PREFIX_DEFAULT("SSTHOME/usr/local/sst")
• AC_CANONICAL_HOST
• exec_prefix"prefix/hostHOST_SUFFIX"
• dnl data_suffix is used in share/Makefile to
  plant local symbolic links.
• data_suffixecho datadir sed
  's,prefix/,,'
• AC_SUBST(data_suffix)
• dnl Set FLIBS to link flags enabling C linker
  to find Fortran library
• AC_F77_LIBRARY_LDFLAGS

• AC_LANG_PUSH(C)
• AC_HEADER_TIME
• AC_CHECK_HEADERS(pwd.h unistd.h sys/times.h)
• dnl Checks for types
• AC_TYPE_SIZE_T
• dnl Checks for structures
• AC_STRUCT_TM
• dnl SST_STRUCT_TMS defines TMS_IN_SYS_TIMES
• dnl if struct tms is in sys/times.h
• SST_STRUCT_TMS
• AC_LANG_POP(C)
• dnl Checks for library functions.
• AC_FUNC_MEMCMP
• AC_FUNC_STRCOLL

19
Automake

• Inputs configure.ac and srcdir/Makefile.am
• Output srcdir/Makefile.in
• which is later used by configure to generate
  objdir/Makefile
• Makefile.am is a Makefile except
• Supports include, conditionals,
• Special macro names are recognized
• Makefile.in variables _at_XX_at_
• Most of the meat is missing
• automake creates a zillion Make macros and rules
  to do the actual work of building your software
• and installing, uninstalling, testing,
  distributing, cleaning, etc.

• Main program example
• bin_PROGRAMS sst
• sst_SOURCES sst.cc
• sst_DEPENDENCIES ../math/libSSTmath.a
• sst_LDADD (sst_DEPENDENCIES) (FLIBS) lm
• Library example
• noinst_LIBRARIES libSSTmath.a
• libSSTmath_a_SOURCES chebyshev.c clairy.cc
  clairy.hh

20
Makefile.am Syntax

• Special names per install directory
• Form dir_PRIMARY
• Primaries PROGRAMS, LIBRARIES, LTLIBRARIES,
  DATA, HEADERS, SCRIPTS, MANS, TEXINFOS
• Directories bin, lib, include, man1, etc.
• Pseudo-directories noinst check
• Special names per target
• Form target_PRIMARY
• Primaries SOURCES, DEPENDENCIES, LDADD, LDFLAGS,
  LIBADD
• Targets Names from PROGRAMS etc.
• Absent target means all targets
• Global special name
• AUTOMAKE_OPTIONS
• Make macros used in rules
• SUBDIRS, EXTRA_DIST, lots more

• Main program example
• bin_PROGRAMS sst
• sst_SOURCES sst.cc
• sst_DEPENDENCIES ../math/libSSTmath.a
• sst_LDADD (sst_DEPENDENCIES) (FLIBS) lm
• Library example
• noinst_LIBRARIES libSSTmath.a
• libSSTmath_a_SOURCES chebyshev.c clairy.cc
  clairy.hh
• Non-special Make macros and rules simply pass
  through.

21
GNU Makefile Standardshttp//www.fsf.org

• Targets
• all
• install, uninstall, install-strip
• clean, distclean, mostlyclean, maintainer-clean
• TAGS
• info, dvi
• dist
• check, installcheck, installdirs
• Most projects need others too

• Variables
• AR, BISON, CC, FLEX, INSTALL, LD, LDCONFIG, LEX,
  MAKE, MAKEINFO, RANLIB, TEXI2DVI, YACC
• INSTALL, INSTALL_PROGRAM, INSTALL_DATA
• prefix, exec_prefix, bindir, sbindir, libexecdir,
  datadir, sysconfdir, sharedstatedir,
  localstatedir, libdir, infodir, lispdir,
  includedir, oldincludedir, mandir, man1dir (etc.)
• manext, man1ext (etc.)
• srcdir

22
Makefile.am Top level

• AUTOMAKE_OPTIONS foreign no-installman
  no-installinfo
• SUBDIRS src support test
• EXTRA_DIST arwrap config.local mdwrap
  sstDoxy.config
• Use doxygen to document source code in these
  directories
• DOXY_SUBDIRS src
• This target builds the html doxygen files and
  leaves
• each generated tree (named html) under each
  leaf subdirectory
• doxy doxy-recursive
• doxy-recursive
• list'(DOXY_SUBDIRS)' for subdir in list do
  \
• test "subdir" . (cd subdir (MAKE)
  (AM_MAKEFLAGS) doxy) \
• done

23
Makefile.am Library

• AUTOMAKE_OPTIONS foreign \
• no-installman no-installinfo
• noinst_LIBRARIES libSSTmath.a
• libSSTmath_a_SOURCES \
• clairy.cc \
• clairy.hh \
• Function2.cc \
• Function2.hh
• et cetera
• These are included only if they are in
• MATHOBJS, where configure put them if they
• aren't in the system math library.
• EXTRA_libSSTmath_a_SOURCES \
• erf.c j1.c lgamma.c
• libSSTmath_a_LIBADD _at_MATHOBJS_at_
• libSSTmath_a_DEPENDENCIES \

• Look here to find include files
• INCLUDES -I(srcdir)/..
• EXTRA_PROGRAMS clairy_test
• clairy_test_SOURCES clairy_test.cc
• clairy_test_DEPENDENCIES clairy.o
• clairy_test_LDADD clairy.o
• Remove Sun Forte's template database
• clean-local
• -rm -fr SunWS_cache
• Generate the Doxygen documentation
• DOCNAME math
• DOCSRC (libSSTmath_a_SOURCES)
• DOCDEPS sigproc describe sim
• include (srcdir)/../tools/doxy.make

24
Makefile.am Main program

• AUTOMAKE_OPTIONS foreign \
• no-installman no-installinfo
• bin_PROGRAMS sst track2traj
• sst_SOURCES sst.cc
• sst_DEPENDENCIES \
• ../symbol/libSSTsymbol.a \
• ../language/libSSTlanguage.a \
• ../describe/libSSTdescribe.a \
• ../signal/libSSTsignal.a \
• _at_CASS_LIB_at_ \
• _at_GSM_LIB_at_ \
• ../ocean/libSSTocean.a \
• ../sonar/libSSTsonar.a \
• ../math/libSSTmath.a
• sst_LDADD (sst_DEPENDENCIES) (FLIBS)

• track2traj_SOURCES track2traj.cc
• track2traj_LDADD -lm
• Look here to find include files
• INCLUDES -I(top_srcdir)/src/
• Remove Sun Forte's template database
• clean-local
• -rm -fr SunWS_cache
• Generate the Doxygen documentation
• DOCNAME sst
• DOCSRC (sst_SOURCES)
• DOCDEPS symbol cass gsm ocean \
• sonar signal math language describe
• include (srcdir)/../tools/doxy.make

25
Libtool

• Problem Dynamic libraries differ by system
• Naming conventions
• Versioning
• Commands for creating them
• Compiler options required (e.g.
  position-independent code)
• Solution
• Wrap platform-specific tools in a script
• Generate the script as output from configure
• Modify generated Makefiles to call the script

26
configure.ac for libtool

• Required
• AC_PROG_LIBTOOL
• Optional
• AC_DISABLE_SHARED
• AC_DISABLE_STATIC
• AC_DISABLE_FAST_INSTALL

27
Makefile.am for libtool

• Building a static library without libtool
• lib_LIBRARIES libshell.a
• libshell_a_SOURCES object.c subr.c symbol.c
• Building a library with libtool
• lib_LTLIBRARIES libshell.la
• libshell_la_SOURCES object.c subr.c symbol.c
• Linking an executable with libtool
• bin_PROGRAMS shell
• shell_SOURCES shell.c token.l
• shell_LDADD libshell.la
• Static vs. dynamic controlled by LDFLAGS macro

28
Advice

• If your project is sufficiently simple, its not
  worth it! Stop here! (See Who Needs Autotools
  slide.)
• Start with the simplest possible configure.ac and
  Makefile.am. Make it work on one primary system.
• Run autoscan for hints only. Look up the macros
  that it suggests. Add them one at a time if they
  seem relevant.
• Add complications only if portability problems
  arise.
• The documentation is essential! Fortunately, its
  also pretty good.
• Iterate and persevere. For complex projects, you
  will be glad you did.