MATLAB A Language for Scientific Computing

1
MATLAB A Language for Scientific Computing

• Krishna Monian
• Hari Kannan

2
Why Domain Specific Languages?

HPC Productivity An Overarching View, Dr.
Jeremy Kepner, MIT Lincoln Laboratory
3
Scientific Computing

• What and How?
• Constructing mathematical models, using computers
  to analyze scientific and engineering problems
• Programmers typically use high level languages
• Mathematically intensive MATLAB, Fortran etc.
• Computationally intensive C, Fortran etc.
• Requires a uniform computing environment
• Challenges for Scientific Computing
• Single processors insufficient to run these
  applications
• High level languages are not inherently parallel
  instrumentation/library routines required for
  this purpose
• Cutting edge computations limited by computing
  power and communication overheads

4
Patterns in Scientific Computing

• Computation
• Data parallel
• Intensive computation - FFT, DNA sequencing
• Array and matrix manipulation
• Memory Access
• Regular Matrix multiplication, FFT
• Irregular Cryptanalysis
• Communication
• Point to point communication (e.g. SMG2000
  financial simulation)
• Collective communication (e.g. GridCCM grid
  based computing)
• Physical system simulations need to communicate
  small amounts of data frequently and globally

5
MATLAB Matrix Laboratory

• Developed by Cleve Moler in 1970s
• Designed for scientific computations, to be quick
  when performing matrix manipulations
• Was later rewritten, with more functionality for
  plotting routines, graphical interfaces etc.
• Highly data parallel
• Not developed with coarse-grain parallelism in
  mind

6
Advantages of MATLAB

• Provides automatic vectorization
• SIMD parallelism - great for operations like
  Image Processing
• Notation
• Simple and well-understood
• Low development time
• Visualization, graphical interface
• Toolboxes
• Numerous routines available (e.g. FFT etc)
• Due to interpretive nature, can perform real-time
  checks (e.g. array bounds)

7
Drawbacks

• Limited Flexibility
• Programmer limited to high level constructs and
  components
• No support for references limits usefulness for
  trees, lists etc.
• Interpreted Language
• No compiler optimizations loops are a problem
• Thread parallelization is not automated
• Does not handle multi-dimensional arrays well
• Does not interface well with other programs

8
Hardware/Software for MATLAB

• Hardware
• Vector architectures (most scientific apps)
• High spatial locality (matrix/FFTs)
• High memory bandwidth
• Software
• Translate MATLAB to C, for compiler optimizations
• Libraries that support parallelization

9
Parallelizing MATLAB

• MATLAB does not have inherent support for thread
  parallelism
• Two strategies for extending MATLAB to make it
  parallel
• Explicitly parallel constructs (e.g. running MPI
  on top of MATLAB)
• Have global arrays distributed across memory
  (e.g. GAMMA)

10
Parallel MATLAB models

• Message Passing Approach
• User explicitly sends messages with the code
• Minimum functionality to implement a parallel
  program
• Serial-gtParallel conversion 25-50 increase in
  code size
• Implementations MATLAB MPI
• Incorporated into MathWorks Distributed
  Computing Toolbox

11
Parallel MATLAB models

• Global Arrays
• Pure
• Global view of distributed array
• User does not access local part of the array
• Operations performed on a global structure
• Ease of programming
• Need to implement parallel version of serial
  operation
• Performance overhead from libraries
• Achieving scalable performance is a challenge
• Fragmented
• User has access to local part of the array
• Limited library overhead Generally better
  performance

12
Other Parallel MATLAB models

• File System Based Communication (Paralize)
• Beowulf clusters - NFS
• Read/Write files to receive/send data
• Synchronization via file-locks
• Bandwidth comparable to C-based MPI but inferior
  latency
• Pure m-file implementation
• No compilation required
• Portable to all platforms
• Polymorphism (MATLABP)
• Function parallelization using overloading
• Programming complexity still remains
• Lazy Evaluation (DLAB)
• Non-blocking server calls

13
Out-of-Core Maps and Hierarchical Arrays(pMatlab
eXtreme Virtual Memory)

• Global Maps Describes how to partition the array
  between processors
• Out-of-core Map Describes how to partition data
  on each processor into core blocks
• Core block Blocks that resides in memory at a
  given point of time while the remain reside in
  disk

• Hierarchical Arrays
• Provides a single, global view of data that are
  distributed throughout a parallel computers
  address space
• Algorithm reimplementation not required when the
  system architecture changes
• Allows implicit data-distribution
• Incremental development possible

14
Wishlist

• MATLAB
• Support for primitives for auto-parallelization
• Optimized looping constructs
• Notion of data types
• Notion of references
• Hardware
• No one current architecture supports all
  scientific application models
• Specialized designs
• Supercomputers with heterogeneous cores

15
Inplace a method for using references

• Matlab package for dealing with vectors and
  matrices, passed by reference. It modifies values
  in place, eliminating copy overhead
• Developed at Stanford (http//www.stanford.edu/dg
  leich/programs/inplace/)
• Makes programming easier, removing onus from the
  programmer to copy values manually
• It provides two classes, ipdouble and ipint32
  (inplace double and inplace int32) that wrap
  Matlab's double and int32 matrices and vectors

16
References

• Parallel MATLAB Doing it Right, R. Choy. et
  al, in Proc. of the IEEE05.
• HPC Productivity An Overarching View, Dr.
  Jeremy Kepner, MIT Lincoln Laboratory.
• pMATLAB Parallel Library, Nadya Travinin,
  Jeremy Kepner, MIT Lincoln Laboratory.
• www.matlab.com
• MATLAB MPI, J. Kepner, S. Ahalt, Journal of
  Parallel and Distributed Computing, 2004
• Parallel MATLAB Survey, R. Choy, 2003
• Parallel Out-of-Core Programming in MATLAB -
  Using the PGAS Model, Hahn Kim, Jeremy Kepner,
  2006