Desktop Grids

1
Desktop Grids
Entropia Desktop Grid
Commercial Desktop Grid System, designed for
enterprise computing
Windows-only system
Out of business
BOINC
Berkeley Open Infrastructure for Network Computing
Example of a desktop grid and volunteer computing
Open source many clients (Linux, Windows,
MacOS). Servers mostly for nix systems, some
Windows servers mostly experimental.
SZTAKI Grid
Outgrowth of the BOINC project
Adds support for connecting local grids into a
hierarchy of grids
2
Desktop Grids
Digipede
C based grid system. Windows only. Closed
source
3
Desktop Grids
Why desktop grids?
Power
550 AMD Triple Core, 4GB memory, 320 GB hard
drive 800 Intel Quad core processor, 5GB
memory, 640 GB hard drive 1000 Core2 Quad
Processor, 6GB memory, 750 GB hard drive
Computers are mostly idle
Return on investment
Companies have already spent the money on
desktops. No need for expensive new hardware if
you can harness the power of the desktops
Numbers of desktops worldwide expected to exceed
1 billion by 2010
4
Desktop Grids
Requirements for desktop grid systems
Efficient Harvest most of the unused cycles on
a computer.
Robust Jobs must complete with predictable
performance, masking underlying resouce failures.
Must tolerate job, machine, and network failures.
Secure Protect both the integrity of of the
distributed application as well as the integrity
of the physical node (no tampering with the
user's registry, modifying user's data, etc)
Scalable Must scale to 1000s, 10,000s, 100,000s
of machines. Not much additional administration
should be required. Must scale up and down,
depending on who's on and who's off the network.
Manageable Can't depends on dozens of admins to
manage the grid.
Unobtrusive The desktop user can't be aware
that their computer is also being used as a node
in a desktop grid. The primary use of the
computer is for spreadsheet, word docs, email,
internet, etc
5
Desktop Grids
Easy to integrate applications Must support
applications developed with varied programming
languages, models,and tools, all with minimal
development effort. This includes binary-only
applications where no source code is available
can not require recompilation of code
Must see the same improvements in throughput as
you would see on a cluster or supercomputer.
6
Desktop Grids
Entropia grid architecture
Physical Node Management
Gathers statistics on the machine CPU speed,
memory, disk size
Controls applications that run in the Entropia
Virtual Machine
Resource Scheduling
Matches units of computation with the appropriate
resource
Must handle failure rates that are greater than
in traditional clustered systems
7
Desktop Grids
Job Management
The end user submits a single logical job. The
job manager decomposes the logical job into
individual sub-jobs. It provides access to the
status of subjobs and aggregates the results of
the sub-jobs
8
Desktop Grids
The end-user submits an application binary, along
with any dll's required, registers any data sets
that will be used and Visual Basic scripts
that will be used to control sub-job execution.
9
Desktop Grids
Sub-job scheduler
Maintains a list of clients on which it is
allowed to run jobs
Client information includes attributes of the
physical nodes, whether they're connected, and
status
Maintains several priority queues for sub-jobs
Each scheduler manages jobs for 1000s of clients
Reports failed sub-jobs to the Job Manager
Helps to cache executable and data files
Node Manager
Centralized interface to manage all clients on
the grid. Allows administrators to monitor, add,
remove, stop, and restart clients. Also monitors
the status of each client issues, how much work
has been performed.
10
Desktop Grids
Desktop Client
Runs sub-jobs on the machine at low priority
Pauses jobs if desktop use is high.
Makes sure that jobs do not get out of control.
Kills any out-of-control jobs
Mediates access to system resources file
system, registry, and gui. This forms the basis
for the sandboxed execution environment
11
Desktop Grids
Entropia Virtual Machine
Desktop Controller is assigned a sub-job and
monitors it for memory, CPU, I/O usage, number of
processes and threads
12
Desktop Grids
Harvest unused cycles. All process run at the
lowest priority.
Resource limits are configurable
Single Desktop Controller on each machine
VM Portal Thread
Thread created when the process is started. It
is hidden from the process
Receives a heart beat from the Desktop
Controller. If this is lost, the thread
terminates itself.
This communication path also controls the pausing
and resuming of jobs
13
Desktop Grids
Sandbox Layer
14
Desktop Grids
Sandbox layer (continued)
Works by rewriting the import table of the binary
application so that vm.dll is the first dll
loaded at runtime
When dll_main from vm.dll runs, it modifies the
binary and any other dll files to intercept
system calls
Some of the Windows APIs (file system, registry,
and network) are mediated. Others (mouse,
graphical user interface, shutting down or
logging off the current user) are disabled.
15
Desktop Grids
Device Driver Mediation
Installed as part of the EVM by the
administrator. Software interrupt handler -
replaces entries in the jump table with pointers
to the EVM functions for certain system calls (eg
File I/O)
Desktop Conroller shares with the device driver
all of the processes running under the EVM - so
the driver knows which process to intercept.
Small amount of overhead for those system calls
that are intercepted. But not all calls are
intercepted - File I/O, process/thread creation,
registry access. For example File I/O, but not
read/write
Also protects the EVM - non-sandboxed
applications can't look inside the Entropia
directory
16
Desktop Grids
Self-modifying code is not permitted to execute
in the EVM
The binarys virtual address space is locked down.
The code portions of the address space are set as
executable and non-writable. Then the rest of
the virtual address space has their permissions
set as non-executable.
One exception the Java Virtual Machine, which
uses a just-in-time compiler.
17
Desktop Grids
Sandbox layer (continued)
File System virtualization
File I/O routines are intercepted and virtualized
to redirect and restrict a subjobs access to the
filesystem For example, a subjob believes that
it is accessing a file in the directory
C\Program Files\ when in fact it is accessing a
sandbox directory (e.g., C\Entropia\root\C\Prog
ram Files\). The subjob only sees C\Program
Files\. Certain existing directories (e.g.
C\WINNT\System\) on the desktop can be accessed
by subjobs as read-only
Registry Virualization
Certain parts of the registry are marked as read
only, and the rest marked as not accessible All
writes to the registry are redirected. Updates
to a restricted registry entry will result in a
copy on write.
18
Desktop Grids
Sandboxing (cont.)
Validate binaries for execution
A cryptographic checksum of each submitted binary
file is created. These values are used to verify
that the files to execute have not changed since
it was patched.
A configuration file containing the list of
sandboxed files and their checksums is also sent
to the client to provide file validation. The
configuration file is encrypted during subjob
submission. It is stored on disk in encrypted
form, and the key is securely communicated to the
EVM.
Before any application binary or dll file can be
invoked the checksum is first validated.
Intercept the CreateProcess call. A sandboxed
application is allowed to launch another
application only if (a) it is registered with the
EVM in the configuration file and (b) its
checksum matches.
19
Desktop Grids
Prevent subjob processes from opening with write
permissions on any binary listed in the
configuration file
20
Desktop Grids
Application Security protecting the jobs and
data from the desktop user
To start, desktop users must not have
administrative privileges, and the EVM and the
subjob processes are run under a special Entropia
user account. This prevents other users from
being able to look at the data of the EVM and
running subjob processes.
The Entropia sandbox keeps all data files
encrypted on disk, so that their contents are not
accessible even if the disk is compromised.
In addition, the sandbox automatically monitors
and checks the integrity of a grid applications
binary, input and result files with an encrypted
checksum file.
If any tampering is found, the subjob is
scheduled on another client.
21
Desktop Grids
BOINC Desktop Grid Architecture
BOINC Server
BOINC Client
22
Desktop Grids
The server (http//boinc.berkeley.edu/trac/wiki/Se
rverIntro)
Apache Web Server MySQL database PHP Python
The scheduler CGI program handles requests from
clients, receiving completed results and sending
new work to compute
Daemons Feeder Loads tasks from the database
for the scheduler Transitioner Handles state
transitions of workunits and results. Generates
results from workunits when they are first
created. Validator Compares results from work
units (User-supplied daemon. May use fuzzy
comparison in cases of floating-point results
where there might be differences in precision.)
If the results are valid, creates the canonical
results. Assimilator Processes the canonical
result File deleter Deletes output files
23
Desktop Grids
BOINC Server Features
Homogeneous redundancy (sending workunits only to
computers of the same platform -- e.g. Win XP
SP2 only.)
Workunit trickling (sending information to the
server before the workunit completes)
Locality scheduling (sending workunits to
computers that already have the necessary files
and creating work on demand)
Work distribution based on host parameters
(workunits requiring 512 MB of RAM, for example,
will only be sent to hosts having at least that
much RAM)
Servers can be clustered for high demand
applications. One server can handle 1000 clients
Sends redundant workunits to various clients to
aid in validation of results.
24
Desktop Grids
BOINC Projects
The BOINC runtime library is implemented in C
and is easiest to use from C/C programs. For
this approach, the software must be recompiled
with the BOINC API for each supported platform
(Linux, Windows, MacOS)
Or, for legacy applications where source code is
not available or for applications written in
other languages, BOINC provides a wrapper which
acts as a main program, managing communication
with the BOINC client, and running the
application as a subprocess.
Digitally sign the application.
(Optional) Register with an Account Manager, such
as Grid Republic or BAM (BOINCStats Account
Manager). This makes it easier for users to find
a project, sign up for it, and manage their
account using the BOINC desktop client.
25
Desktop Grids
BOINC Desktop Application
26
Desktop Grids
BOINC Desktop Application
Runs with the same privileges as the user who
started the applications. Don't run as an
administrator! The only other security provided
is from the digital signatures of the downloaded
applications. Data may also be encrypted, but
users may still be able to view unencrypted data
in a debugger
Allows users to sign up easily for new projects.
Lets you configure your machine for the amount of
workload and the scheduling of the workload, CPU,
memory, and disk use.
Lets you view various statistics about the
projects running on your machine.
27
Desktop Grids
Desktop Grid Applications
The applications exhibit large degrees of
parallelism (thousands to even hundreds of
millions) with little or no coupling
May lead to the reevaluation of many existing
algorithms to find novel uncoupled approaches
GIMPS (Great Internet Mersenne Prime Search)
First application ported to the Entropia Desktop
Grid
Virtual Screening The testing of hundreds of
thousands (to millions) of candidate drug
molecules to see if they alter the activity of a
target protein by evaluating the binding
affinity of the test molecule to a specific place
on a protein in a process called docking. The
amount of data required for each molecular
evaluation is small - basically the atomic
coordinates of the molecules - and the essential
results are even smaller, a binding score.
28
Desktop Grids
Desktop Grid Applications (cont)
Sequence Analysis DNA or protein sequence
analysis applications, including the BLAST,
HMMER, and versions of SmithWaterman programs.
One sequence or set of sequences is compared to
another sequence or set of sequences and
evaluated for similarity. The sequence sizes
vary, but each comparison is independent so that
sets of millions of sequences (gigabytes) can be
partitioned into thousands of slices. Each
compute client receives a set of sequences to
compare and the size of the database, enabling it
to calculate expectation values properly for the
final composite result.
Molecular Properties and Structure These
programs are deployed in a data parallel mode
similar to docking, where the data-parallelism
arises from independent molecule evaluations.
29
Desktop Grids
Desktop Grid Applications (cont)
Financial Risk Management Involves the use of
Monte Carlo methods to evaluate a wide range of
possible outcomes The number needed to achieve
seed independence is about 10,000. Further
increases in the number of samples as well as
increased model complexity can increase the
accuracy of results. Each sample simulation is
independent, and can be executed on a distinct
processors
30
Desktop Grids
Desktop Grid Performance
Entropia
31
Desktop Grids
Desktop Grid Performance
SZTAKI Grid Running averages from the previous
48 hours 3488 Hosts 983 Gflops avg. 3.4 Tflops
peak (http//szdg.lpds.sztaki.hu/szdg/)
32
Desktop Grids
Desktop Grid Performance
BONIC (The Computational and Storage Potential of
Volunteer Computing David P.
Anderson1 and Gilles Fedak2) The BOINC client
executing the Whetstone and Dhrystone benchmarks..