Parallel Java Programming using Clusters

1
Parallel Java Programming using Clusters
The American University in CairoComputer Science
Department

• CSCI-585-03
• Prepared By Ahmed Abbas

2
Outline

• Why using Java for parallel programming
• Message Passing Models using Java
• Parallelism in Java
• Java Threads
• Threads Architecture
• Threads Life Cycle
• RMI
• Mechanism
• Client/Server Model
• KaRMI replacement
• JavaParty

3
Outline

• Cluster Architecture Configuration
• PVM
• XPVM
• JPVM
• Overview
• Introduction
• Interface
• Configuration
• Running Example
• References

4
Why using Java for parallel computation?

• Java compiler functions Just-in-Time (JIT) that
  improved its performance by faster interpretation
  of bytecode.
• Java is platform independent for developing
  parallel computation programs in heterogeneous
  environments.
• Java has built-in primitives for writing
  multithreaded programs using Java Threads.
• Java Application Programming Interface (API)
  includes multi-level support for network
  communication.
• Establishment of low level sockets between hosts
• Support for distributed object schemes as (RMI).
• Java allows programs to do more than one thing at
  a time (concurrency) using (multithreading)
  technique.

5
Message Passing Models

• MPI Flavors
• MPIJ project is a completely java-based MPI
  implementation.
• It is developed using Java language.
• PVM Flavors
• JPVM project (formerly JavaPVM) is a native
  method interface to PVM for the Java platform.
• The JPVM project is a PVM-like library of object
  classes implemented in and for use with Java.
• JavaParty Flavors
• JavaParty accomplishes its extensions with
  standard java compilers and JVMs.
• JavaParty is viewed as a party of JVMs
  cooperating on common task.
• JavaParty was built over Java RMI.

6
Parallelism in Java

• Java implements parallelism in two approaches
• 1. Java Threading.
• API ltjava.lang.Threadgt
• 2. Remote Method Invocation (RMI).
• API ltjava.rmi.gt

7
Java Threads Architecture

• Thread Definition
• It is a single sequential flow of control within
  a program.
• It has a beginning, a sequence, and an end and at
  any given time during the runtime of the thread,
  there is a single point of execution.
• However, a thread itself is not a program it
  cannot run on its own. Rather, it runs within a
  program.
• Thread Lock
• Most applications require threads to communicate
  and synchronize their behavior to one another.
• This is done using locks for shared resources.
• To prevent multiple accesses, threads can acquire
  and release a lock before using resources.
• Locks around shared variables allow Java threads
  to quickly and easily communicate and synchronize

8
Java Thread Life Cycle
9
Java Thread Life Cycle

• Creating a Thread
• - myThread new Thread()
• Starting a Thread
• - myThread.start()
• - myThread is in Runnable State
• - myThread could scheduled using Thread Priority
• Making a Thread Not Runnable
• Thread is Not Runnable if any of these events
  occurs
• Invocation of sleep method myThread.sleep(1000)/
  / time in msecs
• If a thread is waiting for a condition, then
  another object must notify the waiting thread of
  a change in condition by calling notify or
  notifyAll
• Thread is blocked on I/O, then the I/O must
  complete.(deadlock)
• Stopping a Thread
• There is not a stop() method to be invoked.
• A Thread arranges for its own death by having its
  run method to terminate naturally when run()
  methods exits.
• isAlive Thread State
• returns true thread has been started and not
  stopped.
• returns false thread is either new or dead.

10
Java Thread States
11
Remote Method Invocation (RMI)

• A distributed running object in one JVM invokes
  method(s) on another running object in a
  different JVM.

12
RMI Mechanism

• Client (A) invokes a method on a remote interface
  ,proxy object, called the stub, that receives the
  method invocation.
• The stub then forwards the call across the
  network, where a skeleton, proxy object,
  receives it.
• The skeleton will then invoke the method on the
  Server (B).
• When the invocation is done any return arguments
  are forwarded back across the network to (A)
  through the same mechanism.
• In RMI, remote objects may be passed as arguments
  or returned as results from any method
  invocation, whether local or remote.
• A remote object, as an argument, is
  passed-by-reference.
• A remote object may be typecast into any of the
  remote interface that it supports. Local objects
  are passed by value.

13
RMI Client-Server Model

• Server
• create some remote objects
• make references to them available
• wait for clients to invoke methods
• Client
• obtain remote reference to an object on server
• invoke methods on the object

14
Naming and Registry Service

• Registry
• a table
• manages remote references
• associates names with remote references
• provides an API to bind and lookup
• Naming
• service that provides access to registry
• allows server to bind ltname, remote object gt
• allows client to lookup a name

15
Serializing Objects in RMI

• Another Java facility that supports RMI is object
  serialization. The ltjava.io.Serializablegt package
  includes classes that can convert an object into
  a stream of bytes and reassemble the bytes back
  into an identical copy of the original object.
• Using these classes, an object in one process can
  be serialized and transmitted over a network
  connection to another process on a remote host.
• The object can then be reassembled on the remote
  host. An object that you want to serialize has to
  implement the ltjava.io.Serializablegt interface.

16
KaRMI a drop-in replacement for RMI

• RMI is designed for wide-area, high-latency
  networks and uses a slow object serialization.
• KaRMI was introduced, a much faster drop-in RMI
  and an efficient drop-in serialization designed
  and implemented completely in Java.
• Moreover, this redesigned RMI supports non-TCP/IP
  communication networks, for instance
  high-performance networks such as Myrinet.
• KaRMI keeps the RMI API, so converting an
  application to KaRMI is as easy as substituting
  all RMI packages in import declarations with
  KaRMI.

17
JavaParty Overview

• JavaParty allows easy port of multi-threaded Java
  programs to distributed environments such as
  clusters.
• Regular Java already supports parallel
  applications with threads and synchronization
  mechanisms.
• The normal way of porting a parallel application
  to a distributed environment is the use of a
  communication library (KaRMI).
• JavaParty classes can be declared as remote,
  while regular Java classes are limited to one
  Java virtual machine, remote classes and their
  instances are visible and accessible anywhere in
  the distributed JavaParty environment.
• As far as remote classes are concerned, the
  JavaParty environment can be viewed as a JVM that
  is distributed over several computers.

18
Cluster Architecture Configuration
19
PVM -- Parallel Virtual Machine

• The PVM system is composed of two parts
• 1. Daemon called pvmd3/pvmd
• It resides on all the computers making up the
  virtual machine.
• When a user wants to run a PVM application, he
  first creates a virtual machine by starting up
  PVM.
• 2. Library of PVM interface routines
  (libpvm3.a)
• This library contains user callable routines for
  message passing spawning processes.
• These processes coordinate tasks and modify the
  virtual machine application programs via user
  implemented code.
• The PVM console, called pvm
• It is a stand alone PVM task which allows the
  user to interactively start query and modify the
  virtual machine.

20
XPVM -- X-Parallel Virtual Machine

• XPVM provides a graphical interface to the PVM
  console commands and information, along with
  several animated views to monitor the execution
  of PVM programs.
• These views provide information about the
  interactions among tasks in a parallel PVM
  program, to assist in debugging and performance
  tuning.
• To analyze a program using XPVM, a user need only
  compile their program using the PVM library,
  version 3.3 or later, which has been instrumented
  to capture tracing information at run-time.
• Then, any task spawned from XPVM will return
  trace event information, for analysis in real
  time, or for a playback from saved trace files

21
XPVM -- X-Parallel Virtual Machine

• Console Commands
• XPVM Monitor Views
• Network View
• Space-Time View
• Utilization View
• XPVM Debugging Features

22
XPVM Sample Output Screen
23
JPVM -- Java Parallel Virtual Machine

• JPVM is a PVM-like library of object classes
  implemented in and for use with the Java
  Programming language
• JPVM is the combination of both ease of
  programming inherited from Java and high
  performance through parallelism inherited from
  PVM.
• The current implemented JPVM version is (v 0.2.1)
  and requires JVM which running version is
  J2SE_SDK 1.4.2_04

24
JPVM is not operable with standard PVM

• Unfortunately, JPVM is not operable with PVM.
• JPVM is a completely separate, non-interoperable
  implementation.

25
JPVM -- Introduction

• PVM and MPI are packages that provide an explicit
  message-passing, distributed memory MIMD
  programming model.
• These software systems support simple, portable
  library interfaces for typical high-performance
  computing languages such as C and FORTRAN.
• Java provides a portable, uniform interface to
  threads.
• Using threads instead of traditional heavy weight
  processes has been found to allow finer-grained
  computations to achieve good performance in
  distributed memory parallel processing
  environments.
• From the system implementation perspective, Java
  supports a high degree of code portability and a
  uniform API for operating system services such as
  network communications.
• The JPVM library is a software system for
  explicit message passing based distributed memory
  MIMD parallel programming in Java.

26
JPVM -- Interface

• The central interface through which most JPVM
  interaction takes place is exported by
  jpvmEnvironment Java class.
• Objects of jpvmEnvironment represent
  communications end-points within the system, and
  are identified by system-wide unique identifiers
  of the type jpvmTaskId (similar to a PVM task
  identifier).
• However, PVM-3 restricts each task to having a
  single communications end-point, and
  correspondingly a single task identifier.
• JPVM allows tasks to maintain a logically
  unlimited number of communication connections
  simply by allocating multiple instances of
  jpvmEnvironment.
• After an instance of jpvmEnvironment is
  allocated, its containing task invokes basic JPVM
  services such as task creation and message
  passing.
• For example, a task can determine its identity
  for JPVM communication by invoking the
  pvm_mytid() method.
• For example, a task can detach a jpvmEnvironment
  from the JPVM system by invoking the pvm_exit()
  method.

27
Basic jpvmEnvironment Interface

• class jpvmEnvironment
• //Constructor registers task with the JPVM
  system
• public jpvmEnvironment()
• public void pvm_exit()
• //Identity
• public jpvmTaskId pvm_mytid()
• public jpvmTaskId pvm_parent()
• //Task creation
• public int pvm_spawn(String task_name, int num,
  jpvmTaskId tids)
• //Send messages
• public void pvm_send(jpvmBuffer buf, jpvmTaskId
  tid, int tag)
• public void pvm_mcast(jpvmBuffer buf, jpvmTaskId
  tids, int ntids, int tag)
• //Receive messages, blocking (non-blocking
  versions not depicted)

28
JPVM Task Creation pvm_spawn()

• The first action performed by a typical JPVM
  program is creation of tasks to achieve parallel
  execution.
• Task creation in JPVM is supported by the
  pvm_spawn() method.
• Each task created via pvm_spawn() executes in its
  own JVM instance.

29
JPVM Message Passing

• Message passing in JPVM is performed using the
  pvm_send () and pvm_recv () methods of the
  jpvmEnvironment class.
• However, before data can be sent, it must be
  collected into a jpvm-Buffer object, similar to
  PVM buffers, jpvmBuffer objects are the message
  content containers of JPVM.
• The jpvmBuffer interface has two basic groups of
  methods
• Methods that pack data into a buffer.
• Methods that extract data from a buffer.
• class jpvmBuffer
• public jpvmBuffer()
• public void pack(int v, int n, int stride)
• public void pack(int s)
• public void pack(float v, int n, int stride)
• public void pack(float s)
• . . .
• public void unpack(int v, int n, int stride)
• public int upkint()
• public void unpack(float v, int n, int
  stride)
• public float upkfloat()
• . . .

30
JPVM System Configuration

• JPVM library routines require run-time during
  execution in the form of a set of JPVM daemon
  processes running on the available collection of
  processors.
• These daemon processes are required to support
  task creation.
• JPVM system configuration is a two-phase process
• First, daemon processes must be started manually
  on all hosts of interest within the cluster.
• The daemon program is provided in the form of a
  java class jpvmDaemon, so daemon startup simply
  involves running JVM processes to execute the
  daemon class instances.
• Second, after daemons are started, they must be
  notified of one anothers existence.
• This is accomplished through an interactive JPVM
  console program.
• Again, the console program is a java class
  jpvmConsole and should be started in a JVM
  process.
• Besides adding hosts, the JPVM console can be
  used to list the hosts available to the system
  and JPVM tasks running in the system.

31
JPVM Running an Example hello.java

• import jpvm.
• class hello
• static int num_workers 4
• public static void main(String args)
• try
• // Enroll in the parallel virtual machine...
• jpvmEnvironment jpvm new jpvmEnvironment()
• // Get my task id...
• jpvmTaskId mytid jpvm.pvm_mytid()
• System.out.println("Task Id "mytid.toString())
  
• // Spawn some workers...
• jpvmTaskId tids new jpvmTaskIdnum_workers
  
• jpvm.pvm_spawn("hello_other",num_workers,tids)
• System.out.println("Worker tasks ")
• for(int i0iltnum_workersi)
• System.out.println("\t"tidsi.toString())

32
JPVM Running an Example hello_other.java

• import jpvm.
• class hello_other
• static int num_workers 1
• public static void main(String args)
• try
• // Enroll in the parallel virtual machine...
• jpvmEnvironment jpvm new jpvmEnvironment()
• // Get my parent's task id...
• jpvmTaskId parent jpvm.pvm_parent()
• // Send a message to my parent...
• jpvmBuffer buf new jpvmBuffer()
• buf.pack("Hello from jpvm task, id "
  jpvm.pvm_mytid().toString())
• jpvm.pvm_send(buf,parent,12345)

33
JPVM Starting Daemon on Cluster Nodes

• First Console
• aabbas_at_master01 aabbas rlogin slave01
• Last login Tue Apr 6 135646 from master01
• aabbas_at_slave01 aabbas java jpvm.jpvmDaemon
• jpvm daemon slave01.cs.aucegypt.edu, port 32874
• Second Console
• aabbas_at_master01 aabbas rlogin slave02
• Last login Wed May 5 135842 from master01
• aabbas_at_slave02 aabbas java jpvm.jpvmDaemon
• jpvm daemon slave02.cs.aucegypt.edu, port 32851
• Third Console
• aabbas_at_master01 aabbas java jpvm.jpvmDaemon
• jpvm daemon master01.cs.aucegypt.edu, port 32952

34
JPVM Starting Console and Adding Hosts

• Fourth Console
• aabbas_at_master01 aabbas java jpvm.jpvmConsole
• jpvmgt help
• Commands are
• add - Add a host to the virtual machine
• halt - Stop jpvm daemons
• help - Print helpful information about
  commands
• ps - List tasks
• quit - Exit console
• jpvmgt add
• Host name slave01.cs.aucegypt.edu
• Port number 32874
• jpvmgt add
• Host name slave02.cs.aucegypt.edu
• Port number 32851
• jpvmgt conf
• 3 hosts
• slave02.cs.aucegypt.edu

35
JPVM Example Output

• Fifth Console
• aabbas_at_master01 examples javac hello.java
• aabbas_at_master01 examples javac
  hello_other.java
• aabbas_at_master01 examples java hello
• Task Id master01.cs.aucegypt.edu, port 32966
• Worker tasks
• master01.cs.aucegypt.edu, port 32968
• slave01.cs.aucegypt.edu, port 32880
• slave02.cs.aucegypt.edu, port 32860
• slave02.cs.aucegypt.edu, port 32861
• Got message tag 12345 from master01.cs.aucegypt.ed
  u, port 32968
• Received Hello from jpvm task, id
  master01.cs.aucegypt.edu, port 32968
• Got message tag 12345 from slave01.cs.aucegypt.edu
  , port 32880
• Received Hello from jpvm task, id
  slave01.cs.aucegypt.edu, port 32880

36
References

• Adam Beguelin, Jack Dongarra, "PVM 3 Users Guide
  and Reference Manual", September, 1994.
• Adam J. Ferrari, "JPVM Network Parallel
  Computing in Java, Technical Report
  CS-97-29,December, 1997
• Barry Wilkinson, Michael Allen, Parallel
  Programming. Techniques and Applications Using
  Networked Workstations and Parallel Computers.
  August, 1998.
• Bernhard Haumacher, Transparent Distributed
  Threads for Java, 2003), Nice, France, April,
  22-26, 2003
• http//www.sun.com/software/solaris/jit/index.html
  
• http//www-106.ibm.com/developerworks/java/library
  /j-thread.html
• http//java.sun.com/docs/books/tutorial/essential/
  threads/lifecycle.html
• http//skaiste.elekta.lt/Books/O'Reilly/Bookshelfs
  /books/javaenterprise/dist/ch03_06.htm
• http//www.ipd.uka.de/JavaParty/index.html
• http//www.ipd.uka.de/JavaParty/transparentthreads
  .html
• http//www.netlib.org/pvm3/index.html
• http//www.netlib.org/utk/icl/xpvm/xpvm.html
• http//www.cs.virginia.edu/ajf2j/jpvm.html

37
Thank You