Hungarian GRID Projects and Cluster Grid Initiative

1
Hungarian GRID Projects and ClusterGrid
Initiative

• P. Kacsuk
• MTA SZTAKI
• kacsuk_at_sztaki.hu
• www.lpds.sztaki.hu

2
Hungarian Grid projects
3
Ojectives of the VISSZKI project

• Testing various tools and methods for creating a
  Virtual supercomputer (metacomputer)
• Testing and evaluating Globus and Condor
• Elaborating a national Grid infrastructure
  service based on Globus and Condor by connecting
  various clusters

4
Results of the VISSZKI project
Low-level parallel development
PVM
MW
MPI
Grid level job management
Condor-G
Grid middleware
Globus
Grid fabric
Local job management
Condor
Condor
Condor
Condor
Clusters
Clusters
Clusters
Clusters
5
Structure of the DemoGrid project
GRID Generic architecture

• GRID subsystems
• study and development
• Data Storage subsystem
• Relational database, OO database, geometric
  database, distributed file system
• Monitoring subsystem
• Security subsystem
• Demo applications
• astro-physics
• human brain simulation
• particle physics
• car engine design

Data Storage Subsystem
Security subsystem
Monitoring subsystem
RDB
OODB
DFS
GDB
Applications
Decomp
Data
Tightly
Loosely
Hardver
Storage
CPU
Network
6
Structure of the Hungarian Supercomputing Grid
2.5 Gb/s Internet
7
The Hungarian Supercomputing GRID project
GRID application
GRID application
Web based GRID access
GRID portal
High-level parallel development layer
P-GRADE
Low-level parallel development
PVM
MW
MPI
Grid level job management
Condor-G
Grid middleware
Globus
Grid fabric
Condor, PBS, LSF
Condor
Condor, LSF, Sun Grid Engine
Condor, PBS, LSF
Klaszterek
Compaq Alpha Server
Compaq Alpha Server
SUN HPC
8
Distributed supercomputing P-GRADE

• P-GRADE (Parallel GRid Application Development
  Environment)
• The first highly integrated parallel Grid
  application development system in the world
• Provides
• Parallel, supercomputing programming for the Grid
  
• Fast and efficient development of Grid programs
• Observation and visualization of Grid programs
• Fault and performance analysis of Grid programs

9
Condor/P-GRADE on the whole range of parallel
and distributed systems
GFlops
Super-computers
10
Berlin CCGrid Grid Demo workshop Flocking of
P-GRADE programs by Condor
P-GRADE
Budapest
m0
m1
n0
n1
Budapest
Madison
p0
p1
Westminster
11
Next step Check-pointing and migration of
P-GRADE programs
Wisconsin
P-GRADE GUI
Budapest
London
m0
m1
n0
n1
12
Further develoment TotalGrid

• TotalGrid is a total Grid solution that
  integrates the different software layers of a
  Grid (see next slide) and provides for companies
  and universities
• exploitation of free cycles of desktop machines
  in a Grid environment after the working/labor
  hours
• achieving supercomputer capacity using the actual
  desktops of the institution without further
  investments
• Development and test of Grid programs

13
Layers of TotalGrid
P-GRADE
PERL-GRID
Condor v. SGE
PVM v. MPI
Internet
Ethernet
14
Hungarian Cluster Grid Initiative

• Goal To connect the new clusters of the
  Hungarian higher education institutions into a
  Grid
• By autumn 42 new clusters will be established at
  various universities of Hungary.
• Each cluster contains 20 PCs and a network server
  PC.
• Day-time the components of the clusters are used
  for education
• At night all the clusters are connected to the
  Hungarian Grid by the Hungarian Academic network
  (2.5 Gbit/sec)
• Total Grid capacity in 2002 882 PCs
• In 2003 further 57 similar clusters will join the
  Hungarian Grid
• Total Grid capacity in 2003 2079 PCs
• Open Grid other clusters can join at any time

15
Structure of the Hungarian Cluster Grid
2002 4221 PC Linux clusters, total 882 PCs
TotalGrid
2003 9921 PC Linux clusters, total 2079 PCs
TotalGrid
2.5 Gb/s Internet
TotalGrid
16
Live demonstration of TotalGrid

• MEANDER Nowcast Program Package
• Goal Ultra-short forecasting (30 mins) of
  dangerous weather situations (storms, fog, etc.)
• Method Analysis of all the available meteorology
  information for producing parameters on a regular
  mesh (10km-gt1km)
• Collaborative partners
• OMSZ (Hungarian Meteorology Service)
• MTA SZTAKI

17
Structure of MEANDER
First guess data ALADIN
SYNOP data
Satelite
Radar
Lightning
CANARI
Delta analysis
decode
Basic fields pressure, temperature, humidity,
wind.
Radar to grid
Rainfall state
Derived fields Type of clouds, visibility, etc.
Satelite to grid
Visibility
Overcast
GRID
Type of clouds
Current time
Visualization
For meteorologistsHAWK
For users GIF
18
P-GRADE version of MEANDER
19
Implementation of the Delta method in P-GRADE
20
Live demo of MEANDER based on TotalGrid
Parallel execution
21
(No Transcript)
22
On-line Performance Visualization in TotalGrid
Task of SZTAKI in the DataGrid project
Parallel execution and GRM
23
PROVE visualization of the delta method
24
Conclusions

• Already several important results that can be
  used both for
• the academic world (Cluster Grid, SuperGrid)
• commercial companies (TotalGrid)
• Further efforts and projects are needed
• to make these Grids
• more robust
• more user-friendly
• having more new functionalities

25

Thanks for your attention
?
Further information www.lpds.sztaki.hu