Programming Languages and Compilers

1
Programming Languages and Compilers

• AERSP 590 Sec. 2

2
Programming Languages

• Fortran
• C
• C
• Java
• many others

3
Why use Standard Programming Languages?

• Programming tedious
• requiring detailed knowledge of
• instructions,
• registers,
• CPU layout,
• memory
• Source code was numerical notation
• octal code
• machine or assembly code

4
Why use Standard Programming Languages?

• method highly inconvenient
• non-standard
• non-portable
• time consuming
• difficult to debug
• led to IBM's development of FORTRAN
• John Backus

5
Revolution of FORTRAN

• Goals
• simple to understand
• easy to learn
• Programmers relieved of burden of using assembly
  language
• Allowed any scientist or engineer to concentrate
  on problem at hand
• no longer needed to be a system expert

6
Evolution of FORTRAN

• Rapid adoption
• Multiple dialects appeared
• led to problems in portability
• required knowledge of extentions
• Gave rise to standards bodies
• 1966 First 'standard' FORTRAN
• far from perfect, not everyone interpreted or
  adopted rules the same
• F77 / F90 / F95/ F2000/ F2003

7
Standards Bodies

• X3J3 - US Fortran Standards Committee
• performs technical work of producing the standard
  (both as a US national and an international
  standard)
• ISO/IEC JTC1/SC22/WG5 ("WG5" for short)
• coordinates international comment X3J3 work
• gives general advice on the direction in which
  the development of the standard should be heading

8
Standards Questions

• Should language be innovative?
• Small and simple, or big and powerful?
• Will the standard be easy or difficult to
  implement?
• Should older features be dropped from standard or
  must all codes work forever?
• Usefuleness of language subsets
• Is it what the community needs / wants?
• What does the future hold?

9
Ad-Hoc and Vendor Standards

• Java
• Sun Microsystems
• C
• Microsoft
• High Performance Fortran
• Informal standards body

10
Role of the Compiler

• Early programming
• tedious
• highly efficient
• Early FORTAN
• tradeoff between easy to learn / use and
  efficiency
• Early compiler
• great deal of attention
• small loss in efficiency

11
Role of the Compiler

• Translate source code written in a high level
  language to object code or machine language
• write source code in a straightforward manner
• express intentions clearly
• allow the compiler to make choices about
  implementation details that lead to efficient
  execution
• Rarely results in executables that are optimal

12
Compiler Optimization

• Takes an intermediate representation of source
  code and replaces it with a better version
• high-level redundancy in the source program (such
  as an inefficient algorithm) remains unchanged
• Most compilers jack of all trades
• typically only deals with a small part of a
  entire program at a time
• at most a module at a time
• usually only a procedure at a time
• the result is compiler is unable to consider at
  least some important contextual information

13
Compiler Optimization

• Post pass optimizers
• work at assembly code level
• Hand optimization
• requires knowledge of processor / memory layout
• Early compilers not as good at optimizing,
  requiring a lot of hand tuning
• hand tuned code not as portable
• what runs great here may run poorly elsewhere

14
Optimization Types

• peephole
• few instructions at a time
• late in optimization process
• local
• basic block
• loop optimization
• set of basic blocks making up a loop
• hoist out loop invariant code

15
Optimization Types

• Intraprocedural
• acts on control flow graph
• abstract representation of procedure or program
• maintained internally by compiler
• Interprocedural
• optimize interactions between procedures
• most powerful of all

16
Optimization Factors

• Number of CPUs
• Memory subsystem layout
• e.g. number, type and size of caches
• Type and number of control units
• e.g. FPU, IPU
• Number of processor registers

17
Optimization Factors

• Avoid redundancy
• If something has already been computed, it's
  generally better to store it and reuse it later,
  instead of recomputing it.
• Less code
• There is less work for the CPU, cache, and
  memory.
• Straight line code, fewer jumps
• Less complicated code. Jumps interfere with the
  prefetching of instructions, thus slowing down
  code.

18
Optimization Factors

• Code locality
• Pieces of code executed close together in time
  should be placed close together in memory, which
  increases spatial locality
• Extract more information from code
• The more information the compiler has, the better
  it can optimize.
• Use optimal routines where available
• More on this later.....
• Modern compilers have become very good at
  optimization, little need to hand tune

19
Numerical Libraries

• Developing reliable and accurate code takes
  effort
• Standardized numeric libraries present cost
  effective way of solving problems
• No need to reinvent wheel
• Reduced burden on programmer to optimize

20
Benefit of Using Numerical Libraries

• Reduce development time
• esp. debugging time, or at least that is the plan
• reduction of porting costs
• performance benefits
• out of the box
• good investment
• time savings
• optimization

21
Supported Languages

• Fortran
• most all dialects
• maturity of language and compilers negate need
  for some older routines
• C
• C
• Java
• others

22
Typically Available Libraries

• BLAS
• Basic Linear Algebra Subprograms
• LaPACK
• Linear Algebra Package
• ScaLaPack
• Scalable LaPACK

23
BLAS

• Level 1
• vector-vector operations
• Level 2
• matrix-vector operations
• Level 3
• matrix-matrix operations

24
Who else is out there?

• IMSL
• NAG
• Intel MKL
• AMD ACML
• netlib.org
• ATLAS
• GOTO
• many many many more

25
Specialty libraries

• IBM MASS
• for AIX systems
• trade precision for speed
• IBM ESSL
• for AIX systems
• Sparse Matrix
• FFT (FFTW)
• many many many more

26
How to choose

• Features that you need
• Standard and portable?
• How well does it work?
• does it work at all
• how optimal
• Cost
• cost / benefit

27
How to use

• Check documentation!!!!!!!
• review sample codes
• System / Installation dependent
• consult with system support staff and / or user
  guides for specific instructions
• Link time option
• no need to recompile main code to switch between
  libraries

28
Example Matrix Inverse
PARAMETER (NMAX8192) INTEGER IPIV(NMAX),
INDXR(NMAX), INDXC(NMAX) REAL8 A(NP,NP)
n np DO 11 J1,N
IPIV(J)0 11 CONTINUE DO 22 I1,N
BIG0. DO 13 J1,N
IF(IPIV(J).NE.1)THEN DO 12 K1,N
IF (IPIV(K).EQ.0) THEN
IF (ABS(A(J,K)).GE.BIG)THEN
BIGABS(A(J,K)) IROWJ
ICOLK ENDIF
ELSE IF (IPIV(K).GT.1) THEN
PAUSE 'Singular matrix' ENDIF 12
CONTINUE ENDIF 13 CONTINUE
IPIV(ICOL)IPIV(ICOL)1 IF
(IROW.NE.ICOL) THEN DO 14 L1,N
DUMA(IROW,L) A(IROW,L)A(ICOL,L)
A(ICOL,L)DUM 14 CONTINUE
ENDIF INDXR(I)IROW
INDXC(I)ICOL IF (A(ICOL,ICOL).EQ.0.)
PAUSE 'Singular matrix.'
PIVINV1./A(ICOL,ICOL) A(ICOL,ICOL)1.
DO 16 L1,N A(ICOL,L)A(ICOL,L)PIV
INV 16 CONTINUE

• LAPACK Examplecall dgetrf(nmax, nmax, a, lda,
  ipiv, info)if (info .eq. 0) then call
  dgetri(nmax, a, lda, ipiv, work, iwork,
  info) write(6,) 'info ', infoelse write(6,)
  'Singlular matrix'end if

DO 21 LL1,N IF(LL.NE.ICOL)THEN
DUMA(LL,ICOL) A(LL,ICOL)0.
DO 18 L1,N
A(LL,L)A(LL,L)-A(ICOL,L)DUM 18
CONTINUE ENDIF 21 CONTINUE 22
CONTINUE DO 24 LN,1,-1
IF(INDXR(L).NE.INDXC(L))THEN DO 23
K1,N DUMA(K,INDXR(L))
A(K,INDXR(L))A(K,INDXC(L))
A(K,INDXC(L))DUM 23 CONTINUE
ENDIF 24 CONTINUE RETURN END
29
Example Matrix Inversion

• Computationally very expensive
• Want to avoid
• Use BLAS or LaPACK routines

all timings on a 2.6 GHz Opteron
30
Example Other Libraries
all timings on a 2.6 GHz Opteron
31
Example Other Libraries
all timings on a 2.6 GHz Opteron
32
More examples

• Live demo to follow

33
Beyond Numerics...

• Packages to parallelize applications
• Packages to aid parallelization
• Application packages
• G03, CPMD, VASP
• Matlab, Mathematica

34
Thank You...
35
More Information

• e-mail
• beatnic_at_aset.psu.edu
• Please visit the following web pages
• http//www.kcl.ac.uk/kis/support/cit/staff/brian/f
  orsaga.html
• http//encyclopedia.thefreedictionary.com/Optimizi
  ng20compiler