H2O Programming Tutorial

1
H2O Programming Tutorial
2
The model

• Entities
• Kernel service container
• Pluglet service implementation
• Actors
• Provider owns and operates the kernel
• Starts up / terminates
• Defines access control policy
• Developer implements pluglets
• Often, places digitally signed code in a
  repository
• Deployer deploys pluglets in the kernel
• Sometimes, does some aggregation to provide added
  value
• Client uses the pluglets
• Calls remote methods on pluglets
• Aggregates pluglets if needed

3
Example scenarios

• 1. Provider deployer
• e.g. resource legacy application

• 3. Client deployer
• e.g. client runs custom distributed application
  on shared resources

• 2. Reseller Developer deployer
• e.g. computational service offered within a grid
  system

4
Approach

• In this tutorial, we will follow scenario 3
• Client developer deployer
• Provider offers raw resource (e.g. CPU)
• Client utilizes the resoruce by deploying
  application-specific pluglets and delegating
  workload to them
• General outline
• Login to the kernel
• Deploy pluglet(s)
• Do something using pluglet(s)
• Destroy pluglet(s)
• Logout

5
Step 1.What is the pluglet?

• Java class
• Implements Pluglet interface
• To communicate with the kernel
• Implements functional interfaces
• To communicate with clients
• Functional interface RMI remote interface
• extends java.rmi.Remote
• methods throw RemoteException

tutorial/step1/srv/Hello.java
public interface Hello extends Remote String
hello() throws RemoteException
6
Hello pluglet implementation
tutorial/step1/srv/impl/HelloImpl.java
public class HelloImpl implements Pluglet, Hello
public void init(RuntimeContext rcxt,
Properties initParams) System.out.println(
"Hello initialized") public void
start() System.out.println("Hello
started") public void stop()
System.out.println("Hello stopped")
public void destroy() System.out.println("
Hello destroyed") public String hello()
return "Date from server " new
Date(System.currentTimeMillis())
7
H2O Client-Side API

• Core abstractions
• KernelContext represents the kernel
• PlugletContext represents the pluglet
• KernelContext
• Deploy or find pluglets
• Find who is logged in (list sessions)
• Get notified about login or deploy events
• Shutdown the kernel
• PlugletContext
• Access pluglet metadata (name, time deployed,
  etc.)
• Change pluglet state (destroy, suspend, )
• Get notified about state changes
• Bind to pluglet using specified
  messaging/transport

8
Hello Client code
tutorial/step1/HelloMain.java
URI uriKernel new URI(args0) KernelContext
kernelContext H2O.login(uriKernel, null,

H2O.TRUST_ALWAYS) URI classPath new URI
URI.create("tutorial/step1-srv.jar") try
DeploymentDescriptor desc new
DeploymentDescriptor() desc.setPlugletName(He
llo) desc.setPlugletClassName("edu.emory.math
cs.h2o.example.
tutorial.step1.srv.impl.HelloImpl)
PlugletContext plugletContext
kernelContext.deployAndWait(desc) try
Hello hello (Hello) plugletContext.bind()
String response hello.hello()
System.out.println("\nServer said " response
"\n") finally plugletContext.dest
roy() finally kernelContext.logout()

9
Compile Hello
10
Run Hello

• Start the kernel
• Use the GUI, or call h2o/bin/h2o-kernel
• Security policy must allow guest login
• Run the example

java jar tutorial/lib/step1.jar ltkernel_urigt
11
Step 2.Customizing transport

• Communication in H2O
• RMIX layer
• Connection to the pluglet represented by RMIX
  stub
• Configurable / accessible via RMIX API
• Binding to pluglets
• Customizable via ExportProperties
• communication protocol (e.g. SOAP, JRMPX, RPC, )
  
• transport (TCP, SSL/TCP, )
• Can be queried using RmixStubs.getParameter()

12
Customizing transport
tutorial/step2/HelloMain.java
Hello hello (Hello) plugletContext.bind(Expor
tProperties.CURRENT_UNENCRYPTED) // inspect the
networking properties of the remote
endpoint System.out.println("Remote host "
RmixStubs.getParameter(hello, Parameters.PARAM_END
POINT_HOST)) System.out.println("Remote port "
RmixStubs.getParameter(hello,
Parameters.PARAM_ENDPOINT_PORT)) System.out.print
ln("Remote csf " RmixStubs.getParameter(hell
o, Parameters.PARAM_ENDPOINT_CSF_DESCR))
13
Step 3.Pluglet metadata

• Passed to pluglet during initialization
• Deployer can pass custom initialization arguments
• Additionally, pluglet can access its own pluglet
  context, and the local kernel context
• Can e.g. deploy / bind to other pluglets

14
Pluglet metadata
tutorial/step3/HelloMain.java
DeploymentDescriptor desc new
DeploymentDescriptor()...Properties
initialArgs new Properties()initialArgs.setPro
perty("someProperty", "someValue")desc.setInitia
lArguments(initialArgs) PlugletContext
plugletContext kernelContext.deployAndWait(desc)

tutorial/step3/srv/impl/HelloImpl.java
public class HelloImpl implements Pluglet, Hello
public void init(RuntimeContext rcxt,
Properties initParams) PlugletContext
plugletContext rcxt.getPlugletContext()
System.out.println("name " plugletContext.getNa
me()) Date depl new Date(plugletContext.g
etTimeDeployed())
System.out.println("deployed " depl)
initParams.list(System.out) ...
15
Step 4.Event notification

• Kernel-generated events
• New pluglet deployed
• New session opened
• Pluglet / session state changed
• Listeners can be registered to receive event
  notifications
• Every session has its own event-dispatch thread

16
Event notification
tutorial/step4/HelloMain.java
KernelContext kernelContext H2O.login(uriKernel,
null, H2O.TRUST_ALWAYS) // registration of the
listener that will receive notification each//
time a pluglet is deployed into this
kernel. kernelContext.addDeployListener(new
MyDeployListener())
private static class MyDeployListener implements
DeployListener public void
plugletDeployed(DeployEvent evt)
System.out.println("Pluglet " evt.getName() "
has been " "deployed
on " new Date(evt.getTimeDeployed()))
17
Event notification
tutorial/step4/HelloMain.java
DeploymentDescriptor desc new
DeploymentDescriptor() ... PlugletContext
plugletContext kernelContext.deploy(desc,
false)plugletContext.addStateListener(new
MyStateListener())plugletContext.activate() plu
gletContext.waitUntilActive()
private static class MyStateListener implements
PlugletStateListener public void
plugletStateChanged(PlugletStateEvent evt)
System.out.println("Pluglet state change"
" from " evt.getOldState()
" to "
evt.getNewState())
18
Step 5. Accessing transport information at the
server side

• Pluglets can find out who calls them
• Use RMIX API to track transport info
• Client host / port, X.509 identity, etc.
• May be a basis for pluglet-specific authorization

19
Server-side transport
tutorial/step5/srv/impl/HelloImpl.java
Rmix.ServerContext scxt Rmix.getServerContext()
if (scxt null) return // local call String
prot scxt.getProtocol() " ("
scxt.getProviderName() ")"TransportInfo tinfo
scxt.getTransportInfo()if (tinfo instanceof
SocketInfo) SocketInfo stinfo (SocketInfo)
tinfo SocketAddress remoteAddr
stinfo.getRemoteSocketAddress() String
descriptor stinfo.getDescriptor() String
auth "(non-SSL)" if (tinfo instanceof
SSLSocketInfo) SSLSocketInfo sstinfo
(SSLSocketInfo) tinfo SSLSession sses
sstinfo.getSession() String dn
try dn ( (X509Certificate)
sses.getPeerCertificates()0).
getSubjectDN().toString()
catch (SSLPeerUnverifiedException e)
dn "ltnon-authenticatedgt" auth
"SSL " sses.getCipherSuite() ", " dn
System.out.println("Hello remote call
" prot " auth "
auth " descriptor " descriptor) else
System.out.println("Hello remote call " prot
" details unknown
non-socket transport")
20
Server-side transport
tutorial/step5/HelloMain.java
// SOAP 1.1 over current transport with disabled
compressionHello hello1 (Hello)
plugletContext.bind(new ExportProperties( new
ExportProperties.Messaging("SOAP 1.1"),
ExportProperties.Transport.CURRENT_UNENCRYPTED))
// default protocol (JRMPX) over plain TCPHello
hello2 (Hello)plugletContext.bind(ExportProperti
es.PLAIN_SOCKET) // default protocol (JRMPX)
over plain TCP with compression layer atopHello
hello3 (Hello) plugletContext.bind(ExportProp
erties.PLAIN_SOCKET_COMPRESSED) String response1
hello1.hello(World) System.out.println("\nSe
rver said " response1 "\n") String
response2 hello2.hello(World)System.out.prin
tln("\nServer said " response2 "\n") String
response3 hello3.hello(World)System.out.prin
tln("\nServer said " response3 "\n")
21
Step 6.Asynchronous calls

• Pluglet methods may be called asynchronously
• Results returned via futures and callbacks
• Transparent with respect to pluglet
  implementation
• Asynchronous interfaces
• Extend pluglet interfaces
• Use naming convention to introduce
  special-purpose asynchronous methods
• RMIX stubs automatically implement async
  interfaces

tutorial/step6/srv/AsyncHello.java
public interface AsyncHello extends Hello
Future asyncHello(String msg) throws
java.rmi.RemoteException Future
cbasyncHello(String msg, Callback callback)
throws RemoteException
22
Asynchronous calls Futures

• Substitute results from calls
• Can be used to query for completion
• Wait by neccessity
• Synchronous wait for a result as late as possible

tutorial/step6/HelloMain.java
AsyncHello hello (AsyncHello)pc.bind() Future
f hello.asyncHello("World!") // .. now the
real application could do some useful work
here // .. and finally, when the result is
needed// (wait by neccessity) String s
(String)f1.get()System.out.println("Server
response " s)
23
Asynchronous calls Callbacks

• Methods invoked upon completion
• Performed in a background processing thread,
  asynchronously to the invoker thread
• Example usage scenario
• Initiate multiple concurrent calls
• Wait until all of them complete
• Common application of CountdownLatch

package edu.emory.mathcs.util.concurrent public
interface Callback void completed(Object
result) void failed(Throwable cause)
24
Asynchronous calls Callbacks
tutorial/step6/HelloMain.java
AsyncHello hello1 (AsyncHello)
pc1.bind()AsyncHello hello2 (AsyncHello)
pc2.bind() final CountDownLatch latch new
CountDownLatch(2) Callback latchCountdown new
Callback() public void completed(Object obj)
System.out.println("Response " obj)
latch.countDown() public void
failed(Throwable cause)
System.out.println("Remote call failed!")
latch.countDown() hello1.cbasyncHello("Wo
rld from hello1!", latchCountdown)hello2.cbasync
Hello("World from hello2!", latchCountdown) latch
.await(5000, TimeUnit.MILLISECONDS) if
(latch.getCount() gt 0) System.out.println("Afte
r 5 s, still not all calls have returned!)
25
Step 7Remote objects

• Non-trivial applications use many remote objects
• Complex dependency graphs
• Object references may be passed between calls
• RMIX layer in H2O
• Pluglets may create and export remote objects
• Their lifetime is limited by the pluglet lifetime
• Automatic export
• Objects implementing Remote, when passed to or
  returned from remote call, are replaced by stubs

26
Remote objects
tutorial/step7/srv/Hello.java
public interface Hello extends Remote Echo
getEcho() throws RemoteException
tutorial/step7/srv/impl/HelloImpl.java
public class HelloImpl implements Hello ...
public Echo getEcho() return new
EchoImpl()
tutorial/step7/srv/Echo.java
public interface Echo extends Remote String
echo(String echo) throws RemoteException
tutorial/step7/srv/impl/EchoImpl.java
public class EchoImpl implements Echo public
String echo(String echo) throws RemoteException
return echo " " echo
27
Remote objects
tutorial/step7/HelloMain.java
Hello hello (Hello)plugletContext.bind() //
call the remote method that returns a remote
reference to the echo Echo echo
hello.getEcho() // demonstrate that "echo" is a
remote object System.out.println("Echo socket
factory " RmixStubs.getParameter(hello,
Parameters.PARAM_ENDPOINT_
CSF_DESCR)) // perform remote call on
"echo" String response echo.echo("echo")System
.out.println("\nEcho said " response "\n")
28
Step 8Connecting pluglets

• Metacomputing applications
• Usually require interconnections between
  components
• Non-trivial connection topologies
• Tree, mesh, tori, full graphs,
• In H2O must be able to interconnect pluglets
• Approach pluglets act as clients to other
  kernels or to their local kernel
• Connectivity provided by RMIX
• May use specified protocols and transports

29
Connecting pluglets
tutorial/step8/srv/impl/Hello1Impl.java
public class Hello1Impl implements Pluglet,
Hello1 private KernelContext kernelContext
private Hello2 peer public void
init(RuntimeContext rcxt, Properties initParams)
PlugletContext plugletContext
rcxt.getPlugletContext()
this.kernelContext plugletContext.getKernelConte
xt() ... public synchronized void
connect(String peerGUID) throws RemoteException
PlugletContext pc kernelContext.getPlugl
et(peerGUID) this.peer (Hello2)
pc.bind(ExportProperties.INPROC_PASSBYREF)
public String hello(String msg) throws
RemoteException Hello2 peer
synchronized (this) peer this.peer
return peer.hello(msg)