The Distributed ASCI Supercomputer

1
The Distributed ASCI Supercomputer

• The third generation

Dick Epema (TUD) (with many slides from Henri Bal)
Parallel and Distributed Group
2
Distributed ASCI Supercomputer

• Joint infrastructure of the ASCI research school
• Clusters integrated in a single distributed
  testbed
• Long history and continuity

DAS-1 (1997)
DAS-2 (2002)
DAS-3 (2006)
3
DAS is a Computer Science grid

• Motivation CS needs its own infrastructure for
• Systems research and experimentation
• Application experiments
• DAS is simpler and more homogeneous than most
  production grids
• Single operating system
• A simple grid that works

4
DAS-3 overall structure
UvA/MultimediaN (46)
VU (85 nodes)
UvA/VL-e (40)
SURFnet6
Operational oct. 2006
10 Gb/s lambdas
TU Delft (68)
Leiden (32)
5
Cluster configuration
LU TUD UvA-VLe UvA-MN VU TOTALS
Head
storage 10TB 5TB 2TB 2TB 10TB 29TB
CPU 2x2.4GHz DC 2x2.4GHz DC 2x2.2GHz DC 2x2.2GHz DC 2x2.4GHz DC
memory 16GB 16GB 8GB 16GB 8GB 64GB
Myri 10G 1 1 1 1
10GE 1 1 1 1 1

Compute 32 68 40 (1) 46 85 271
storage 400GB 250GB 250GB 2x250GB 250GB 84 TB
CPU 2x2.6GHz 2x2.4GHz 2x2.2GHz DC 2x2.4GHz 2x2.4GHz DC 1.9 THz
memory 4GB 4GB 4GB 4GB 4GB 1048 GB
Myri 10G 1 1 1 1

Myrinet
10G ports 33 (7) 41 47 86 (2)
10GE ports 8 8 8 8 320 Gb/s

Nortel
1GE ports 32 (16) 136 (8) 40 (8) 46 (2) 85 (11) 339 Gb/s
10GE ports 1 (1) 9 (3) 2 2 1 (1)
6
Projects using DAS-3

• Virtual Lab for e-Science
• Grid computing, scheduling, workflow, PSE,
  visualization
• MultimediaN
• Searching, classifying multimedia data
• NWO projects, e.g., StarPlane and GUARD-G
• NCF projects (off-peak hours)
• And many more (P2P, )

7
Projects using DAS StarPlane

• Key idea
• Applications can dynamically allocate light paths
• Applications can change the topology of the
  wide-area network, possibly even atthe
  sub-second timescale
• VU (Bal, Bos, Maassen)
• UvA (de Laat, Grosso, Xu, Velders)

8
Projects using DAS GUARD-G

• How to turn grids into a predictable utility for
  computing (much like the telephone system)
• Problems
• Predictability of workloads
• Predictability of system availability (grids are
  faulty!)
• Allocation of light paths very useful here
• TU Delft (Epema) Leiden (Wolters)

9
Projects using DAS KOALA, a co-allocating grid
scheduler

• Main goals
• processor co-allocation (non-)fixed/flexible
  jobs
• data co-allocation move large input files to the
  locations where the job components will run prior
  to execution
• load sharing in the absence of co-allocation
• run alongside local schedulers
• KOALA
• is written in Java
• uses Globus components (e.g., GRAM, RSL and
  GridFTP)
• has been deployed on the DAS2 in september 2005

10
Status DAS3 clusters

• Delft cluster accepted, up and running
• VU, UvA-MM acceptance this week
• UvA, Leiden acceptance this year