Some Missing Bits

1
Some Missing Bits!

• Alistair Rendell

2
Grid
3
Grid

• Term coined in the mid 90s to denote a proposed
  distributed computing infrastructure for advanced
  science and engineering
• Unites pools of servers, storage systems, and
  networks into a single large system so the power
  of multiple-systems resources can be delivered to
  a single user point for a specific purpose
• The anatomy of the Grid Enabling Scalable
  Virtual Organizations, Ian Foster, Carl Kesselman
  and Steven Tuecke, I.J. Supercomputer
  Applications 2001.

4
Virtual Organizations

• A company needs to reach a decision on the
  placement of a new factory invokes a
  sophisticated financial forecasting model from an
  (application service provider (ASP), providing it
  with access to appropriate proprietary historical
  data from a corporate database on storage systems
  operated by an storage service provider (SSP).
  During the decision making meeting, what if
  scenarios are run collaboratively and
  interactively, even though the division heads
  participating in the decision are located in
  different cities. The ASP itself contracts with a
  cycle provider for additional oomph during
  particularly demanding scenarios, requiring of
  course that cycles meet desired security and
  performance requirements.
• Thousands of physicists at hundreds of
  laboratories and universities worldwide come
  together to design, create, operate, and analyze
  the products of a major detector at CERN. During
  the analysis phase, they pool their computing,
  storage, and networking resources to create a
  data grid capable of analyzing petabytes of
  data

5
Grid Architecture
Application
Application
Collective
Grid Protocol Architecture
Resource
Transport
Connectivity
Internet
Fabric
Link
6
Fabric

• Resources to which shared access is mediated by
  Grid
• computational resources
• storage systems
• catalogs
• network resources
• sensors
• Resource enquiry and management
• how to start jobs on computer systems
• how to write data to storage devices
• catalog query and update

7
Connectivity

• Communication protocols
• from TCP/IP stack Internet (IP), transport
  (TCP), application (DNS)
• Authentication protocols
• single sign on log on just once
• delegation permit a program to run on the users
  behalf
• integration into local security solutions cannot
  rely on wholesale replacement of local security
  solutions
• user based trust relationships user has rights
  to use sites A and B they can use A and B
  together

8
Resource

• Secure negotiation, initiation, monitoring,
  control accounting and payment of sharing
  operations on individual resources
• Information protocols used to obtain information
  about the structure and state of a resource, e.g.
  current load
• Management protocols negotiate access to shared
  resources, e.g. resource requirements

9
Collective

• Global protocols that capture interactions across
  collections of resources
• directory services discover the existence and/or
  properties of virtual organisation resources
• co-allocation, scheduling and brokering serices
• monitoring and diagnostics services
• data replication services
• workload management and collaboration frameworks
• collaborative services Access Grid

10
Applications

• User applications that operate within the virtual
  organisation environment

11
More Info

• Grid toolkits
• www.globus.org (US)
• www.unicore.org (European)
• eScience project in the UK
• www.rcuk.ac.uk/escience
• NSF have cyberinfrastructure
• http//www.communitytechnology.org/nsf_ci_report/
• Australia
• eresearch report just prepared by DEST

12
Programming Languages
13
Fortran (1957)
Programming Language Evolution
Algol 58
Cobol (1959)
PL/1 (1964)
CPL (1963)
Fortran 66
Algol-W (1966)
BCPL (1967)
Algol 68
B (1970)
Pascal (1971)
C (1972)
Fortran 77
defined (1978)
Modula-2 (1979)
C (1983)
Ada (1983)
Standard 1988
Fortran 90
Java (1993)
14
Usage in Computational Science

• Fortran, C, C, Java, Perl and Python
• usage reflecting age of language
• Fortran
• lots of dusty deck Fortran code
• C
• used in virtually all programs for O/S related
  activities
• C
• increasingly popular in last 5 years
• Java
• for visualization data processing activities
• Perl
• strong foothold in bioinformatics and for
  scripting
• Python
• replacing Perl and rapidly increasing use as a
  rapid prototyping language

Computational Science today is mixed language
15
Fortran Pioneers (NYT June 13, 2001)
They were young, in their 20's and early 30's,
brimming with energy and optimism, a tight knit
team. Their goal was to simplify programming and
open up computing to more people. The industry
consensus was that they were trying the
impossible. They suffered setbacks and
disappointments. A six- month timetable for
completion became nearly three years. But they
finished eventually, and in 1957 the result of
their labors was shown to the world. It was
called Formula Translator -- Fortran, for short.
John Backus, who led the team at I.B.M., came up
with the name to no great enthusiasm from his
colleagues. They used to joke that it sounded
like something spelled backward. But nobody could
come up with a better idea, so the name stuck, as
did Fortran, a programming language that was a
historic breakthrough in computing. Today,
Fortran is often mentioned wistfully by computer
scientists and veteran programmers as the first
programming language they learned but then
abandoned as newer languages were developed for
new kinds of computing. But to point out how
quickly programming has moved to generations of
new tools does not minimize the extraordinary
advance that Fortran gave to the world of
software. Other programming languages rose from
the foundation that Fortran built.
16
Fortran Pioneers (cont)
The team was heavy with math training because so
much of computing at the time was numerical
analysis and mathematics, but it was an eclectic
group -- a crystallographer, a cryptographer, a
chess wizard, an employee lent from an aircraft
manufacturer, a researcher borrowed from M.I.T.
and a young woman who joined the project straight
out of Vassar College. --snip The success of
the team hinged on two things. First, the group
devised a programming language that resembled a
combination of pidgin English and algebra. The
result was a computing nomenclature that was
similar to algebraic formulas that scientists and
engineers used in their work. So Fortran opened
up programming to the people whose problems were
being put on computers in those days. With some
training, they were no longer dependent on the
computing priesthood to translate their problems
into the language of the machine.
17
Fortran I

• At time of first fortran compiler machines had
  very small memories (15KB)
• Fortran authors did not invent idea of using a
  High Level Language (HLL)
• they produced the first successful HLL!
• Fortran I compiler held the record for optimizing
  code for 20 years

18
Fortran II, III, IV and Fortran66

• Fortran II (58) added linking
• Fortran III (58) use of assembly inline with
  Fortran (not released)
• can actually do this on the Sun!
• Fortran IV (61) added common blocks and
  equivalence
• Fortran 66 first ASA standard

19
Fortran 77

• Do loops with decreasing control variable
• Structured if blocks
• if/then/else/endif
• Pretest of do loops
• before a do loop always ran once
• still provided as option by some Fortran
  compilers
• Character data type

Then development stopped for many years
- allowing other languages to catch up!
20
Fortran 90 (July 91)

• Array operations
• Dynamic memory allocation
• Pointers
• User defined data types
• Modules
• user defined collections of useful global objects
  or procedures
• provide support for encapsulating derived data
  types
• Free format source
• Modern control structures (select case)
• Internal procedures and recursive procedures
• Parametrization of intrinsic types to give better
  control of numerical precision

A very significant upgrade from Fortran 77 and a
good language for numerical computing but is it
too late?
21
Fortran 95

• A minor enhancement from F90
• FORALL statement and construct
• PURE and ELEMENTAL procedures
• Structure and pointer default initialization
• Promoted in part by developments in high
  performance Fortran (HPF)
• HPF is a version of Fortran for parallel machines

22
Beyond Fortran 95

• Currently Fortran 2003 is in draft
• Derived type enhancements parameterized derived
  types, improved control of accessibility,
  improved structure constructors, and finalizers.
• Object oriented programming support type
  extension and inheritance, polymorphism, dynamic
  type allocation, and type-bound procedures.
• Data manipulation enhancements allocatable
  components, deferred type parameters, VOLATILE
  attribute, explicit type specification in array
  constructors, pointer enhancements, extended
  initialization expressions, and enhanced
  intrinsic procedures.
• Input/output enhancements asynchronous transfer,
  stream access, user specified transfer operations
  for derived types, user specified control of
  rounding during format conversions, named
  constants for preconnected units, the flush
  statement, regularization of keywords, and access
  to error messages.
• Procedure pointers.
• Support for IEC 60559 (IEEE 754) exceptions.
• Interoperability with the C programming language.
  
• Support for international usage access to ISO
  10646 4-byte characters and choice of decimal or
  comma in numeric formatted input/output.
• Enhanced integration with the host operating
  system access to command line arguments,
  environment variables, and processor error
  messages.

23
F95 and Computational Science

• Extensive support for numerical precision
• Array operations
• array intrinsic operations
• argument passing
• arbitrary indexing
• Built in complex arithmetic
• Rich set of intrinsic functions
• built in exponentiation operator

24
Numerical Precision in Fortran 95
program fp integer, parameter
r4selected_real_kind(5,10) real(kindr4)
s print '(" radix ",i15) ,radix(s)
print '(" digits ",i15) ,digits(s)
print '(" maxexponent ",i15) ,maxexponent(s)
print '(" minexponent ",i15) ,minexponent(s)
print '(" precision ",i15)
,precision(s) print '(" range ",i15)
,range(s) print '(" epsilon
",e15.8)',epsilon(s) print '(" tiny
",e15.8)',tiny(s) print '(" huge
",e15.8)',huge(s) end program fp
25
Output
radix 2 digits
24 maxexponent 128
minexponent -125 precision
6 range 37
epsilon 0.11920929E-06 tiny
0.11754944E-37 huge 0.34028235E39

• All the good stuff you learnt about in first few
  lectures!

26
Array Operations
real8 a(4,4),b(4,4) do i1,4 do j1,4
a(j,i)a(j,i)b(i,j) enddo enddo

• can be replaced by

real8 a(4,4),b(4,4) a a b

• or select elements via

real8 a(4,4),b(4,4) a(142, 241) a(142,
241)b(121, 131)
Very nice for matrix operations
27
Arrays as Arguments

• Array dimension can be passed as argument
• greatly eases indexing within function

real8 a(4,32),b(4,32) call example(a, b, 4,
32) -- snip -- subroutine
example(x,y,m,n) real8 x(m,n), y(m,n) do
i1,n do j1,m x(j,i)x(j,i)y(i,j)
enddo enddo return end
28
The Speed Thing

• Fortran 77 lacked pointers
• permitted enhanced optimisation
• Fortran 90 allows pointers
• but target must be explicitly labelled
• Fortran permitted static storage allocation
• saving time in procedure evaluation
• but no recursive functions
• Fortran 90 permits recursive functions
• but must be explicitly labeled
• Fortran passes all variable by reference
• fastest option
• Fortran disallows aliasing of arguments in
  procedure calls
• allows better compiler optimisation

29
High Performance Fortran (HPF)

• Includes Fortran 95 at its core
• Adds directives for data distribution
• SIMD based parallelism

real8 bob(100,100,100),rich(100,100,100) !H
PF distribute bob(block,cyclic,) !HPF
distribute rich(cyclic(10))

• Lots of complicated issues associated with
  passing arrays to functions
• Very limited support
• although earth simulator uses version of HPF

30
Why we used C!

• All computational science codes use bits of C for
  interacting with the O/S
• many codes are written entirely in C
• Master C and Fortran will be easy
• build on your existing knowledge of C from other
  courses
• In practice larger programs likely to be written
  in C rather than C
• see article Comparison of C and F90 for OO
  Scientific Programming

31
Java and Numerical Computing

• Benefits
• portability
• safe to host computer
• simple OO model
• widespread use
• But not yet widespread in computational science
• used for some GUI, post processing and
  coordination of large numerical applications
• Java Grande Forum focuses on Java for HPC
• www.javagrade.org
• See paper Java and Numerical Computing, by
  Boisvert et al

32
The Future

• Increasing realization that no current language
  is able to handle the multicore revolution that
  is about to be unleashed
• The DARPA High Productivity Computing Systems
  (HPCS) is focused on providing a new generation
  of economically viable high productivity
  computing systems for national security and for
  the industrial user community part of this is
  new HPC languages
• HPCS includes development of the Chapel and X10
  partitioned global address space (PGAS)
  languages (discussion from X10 web links)