Towards World-Wide Open T-System

1

• Towards World-Wide Open T-System
• PSI RAS, 2006

2
Long-term Goal

• Present a simple in use, automatic dynamic
  parallelization technology for a scientists, who
  prefer Windows-based solutions
• Achieve the same performance of the
  T-applications running on Windows clusters as in
  a case of the Linux-based clusters
• Advance the Windows version of the Open T-System
  by adding visualization, fault-tolerance and
  ability to run T-applications on heterogeneous
  (WindowsLinux) metaclusters.

3
Cumputists Wishes

• Many scientists prefer to use Windows, not Linux
  for high-performance computing (by obvious
  reason)
• Many scientists are using C/C for
  high-performance application development
• Most scientists are concentrating on the
  domain-specific problems and do not like like
  study any kind of MPI-programming technology at
  all !

4
Current T-system community

• PSI RAS (Research Education)
• MSU (Computational Math., Mechanic, Chemistry
  faculties)
• NIICS (satellite image processing)
• ChelGU (conformational analysis)
• MSIU (education)

5
Fulfilling Computists Needs

• Q What is a main disadvantage of the Open
  T-System ?
• A It does not run under Windows (most popular
  answer at Intel winter school 2005 taking place
  in Nizhniy Novgorod)

6
Development priorities 2006

• Run T-applications on Windows hosts Windows
  clusters (32bit 64bit modes)
• Run T-applications using variants of
  communications using multiply pluggable DMPI
  drivers for SAN, LAN, WAN
• Make T-system model more clean for T-application
  developers by computational process visualization
• Run T-applications in fault-tolerant mode
• Run T-applications on Itanium PowerPC

7
Help for application tuning

• Develop a tool for T-program computational
  process visualization (in particular, to present
  parallelization dynamic in real time)
• Built-in lightweight performance analyzer to
  automate process of T-program optimization

8
Achieving power of metaclusters

• Fault-tolerant computation mode (using advanced
  re-computation model)
• Run Windows T-application on both Windows and
  Linux clusters (using the wine emulator)
• Dynamic enter/leave of computational
  nodes/segments

9
MacroScheduler 2006 advance

• Reducing resource exchange message count at
  large clusters
• Highly efficient algorithm adopted for
  non-trivial cluster topologies

10
Current DMPI features

• Implements most popular subset of MPI
• mpirun/mpiexec type auto-detection
• Dynamic loading for appropriate MPI driver
• Ability of static MPI binding
• Asynchronous operations support
• Large number of supported MPI PVM
  implementations

11
Additional DMPI 2006 features

• Direct TCP/IP support (able to operate without
  MPI library)
• Multiply MPI drivers working in parallel to mix
  different transport systems SCALI, Myrinet,
  Infiniband, TCP/IP, TCP/IP over HTTP.
• MPI Enter/Leave extensions (in addition to
  standard mpirun/mpiexec running mode)
• MPI_Wait T-Fun waiting coexistence
• Automatic fault detection recovering
• Configuration (topology) information

12
Cross-platform OpenTS

• Unified OpenTS source code tree for ALL supported
  platforms
• Unified DMPI source code tree for ALL supported
  platforms
• Extracting all platform-depended features into
  separate Platform Abstraction Layer (PAL)

13
OpenTS for Windows installer

• Unified installer for Windows XP/2003/MSCCS
  editions
• Installed OpenTS does not require any commercial
  package (it is enough to download Visual C 2005
  Express edition for T-program development freely
  available from www.microsoft.com site)
• Integrated with Visual Studio (T-projects)
• Self-testing during installation procedure!
• OpenTS SDK is available for T-microkernel
  developers

14
Open T-Grid 2006

• Cross-platform run T-applications on Windows and
  Linux cluster hosts at the same time (using
  WinAPI simulation on Linux)
• Fault-tolerance using new DMPI facilities for
  immediate recovering and unique functional nature
  of T-system
• User-friendly run T-visualization as an
  screensaver or as widget/applet in the taskbar
• Safety digital signature check support for
  T-executables

15
Open T-Grid 2006

• Cooperative usage of the Windows and Linux
  clusters and single hosts
• World-Wide T-system-based computational platform
• Good (and awaited) platform for scientists from
  Universities

16
Official T-System projects (2006)(donation-ordere
d)

• MSCCS (Microsoft RUS 60.000)
• T-GRID (RAS 50.000)
• T T (RAS 30.000)
• TRIADA (Russia-Belorussia 40.000)

17
Cost Analysis

• Most new OpenTS features are developed for POSIX
  environment at the current time.
• To adopt presented features for Windows we need
  reasonable financial support at the next year

18
OpenTS 2006 restrictionsscheduled to be removed

• Restricted C OOP support no tfun class
  methods, no tfun templates
• Intermediate levels of OpenTS API are open, but
  not documented yet (Active messages, Mobile
  objects, Supermemory)
• Further performance enhancements (intermediate
  levels template specialization)