WP5

1
WP5
Aplikacije
Dragan Okiljevic Belgrade University Computing
Center (RCUB) dragan.okiljevic_at_rcub.bg.ac.yu
2
SEE-GRID-1 project

• Objectives
• Establish at least 1 fully operational grid site
  in all participating SEE countries
• Each contractor to set up at least one site
  third-parties to set up additional sites
• Deploy at least 2 Grid applications developed by
  EGEE (one from the HEP, one from Biomed) in the
  regional infrastructure and demonstrate their
  usage over the regional infrastructure
• Deploy at least two Grid applications developed
  by SEE-GRID partners in the regional
  infrastructure.
• Create a Human Network in eInfrastructures and
  raise awareness in the wide RE community per
  country

3
SEE-GRID-1 Applications

• Regionals applications
• VIVE Volumetric Image Visualisation Environment
• SE4SEE Search Engine For SEE

4
VIVE - Introduction

• Objective
• Complete interactive analysis of 3D datasets from
  medical imaging devices (CT, MR, PET, SPECT)
• Easy remote access over the Internet
• Generalized 3D tools for diagnosis, surgical
  planning, and therapy evaluation
• Virtual distant examination based on 3D datasets
• Basic idea
• Split the system into a client and server part
• Dedicate the server part to access and processing
  of volumetric data
• Produce a lightweight client for user interaction
  and 3D rendering
• Exploit grid resources and paradigms to overcome
  implementation issues

5
(No Transcript)
6
Screnshots - From Atoms to Galaxies
7
What Can We Gain From the Grid?

• Distribution of processing and data over several
  nodes
• Grid submission used to allocate server jobs
• Allocating several machines for texture and
  surface mesh creation
• Field stripping is not necessary for the regular
  medical images
• Grid reduces overall demand for CPUs even with
  one server job per session
• Could build distributed virtual data centers,
  with data abstracted within electronic health
  record
• Data transfer and replication
• Replication of data to locations close to
  computation elements
• Remote file access
• Security high sensitivity of medical data
• Data encryption in transfer (GridFTP, dCache,
  RFIO and gsidcap)
• Anonymisation - removal of personal details to
  prevent intrusion at machine level, solved by
  gLite Hydra
• Application-specific monitoring
• Could select a site from performance indicators
  (responsiveness, bandwidth, round trip time,
  processing speed) provided by the grid
• Currently we just collect and display monitoring
  data using in our implementation guidelines and
  ideas from various grid/network monitoring
  projects
• Fault tolerance

8
SE4SEE - Introduction

• Grid-enabled crawling of all Web pages in SEE
  countries
• Grid-enabled categorization of these Web pages
• A navigational search facility over the
  classified content
• To sum up, we offer
• A grid-enabled, directory-based search engine
  that will enhance the cultural integration in the
  region

9
SE4SEE - Grid-Enabled Web Crawling

• The size of the Web is enormous Web crawling
  requires
• high download rates over the network
• excessive storage (both memory and disk)
• vast amount of processing power
• It is hard to crawl the Web by a centralized
  system
• Solution Grid-enabled Web crawling since
• geographically distributed grid nodes allow fast
  download rates and dispersion of the network load
• excessive disk and memory capacities together
  with the huge CPU power of GRID provides the
  necessary medium for storing and processing the
  vast Web content

10
(Backup) SE4SEE Web Portal
11
(Backup) SE4SEE Web Portal
12
(Backup) SE4SEE Web Portal
13
(Backup) SE4SEE Web Portal
Grid-Enabled Web Crawling
14
SEE-GRID-2 Application selection

• Large number of potential applications
• For the reason of scalability, it was decided
  that only a subset of the applications will be
  supported
• Candidate application developers filled online
  questionnaire submitting data on their
  applications
• Application ranking criteria developed jointly
  trough e-mail discussion within the consortium
  WP4 partners from all countries.
• 32 applications in total were submitted
  initially. 23 were assessed with the
  questionnaire.

15
SEE-GRID 2 Application statistics (1/2)
16
SEE-GRID 2 Application statistics (2/2)
17
Application selection for 1st round

• One application selected by country flagship
  applications
• Chosen applications covers a broad range of
  applications fields and scientific communities

18
Selected applications for 1st round (1/2)
19
Selected applications for 1st round (2/2)
20
SEE-GRID-2 Application support

• Application support group (ASG) experienced
  developers admins, mostly participants of SG1
  project
• National level application support
• SG - global level application support
• Work in close collaboration with WP5 (training)
  and WP3 (software requirements, maintenance of
  performance)

21
Application gridification guide

• Relevant topics for application developers
  identified trough online questionnaire system
• Some investiogation areas identified as well
  candidates for future GG topics
• Gridification guide will provide information on
  these topics
• GG collaboration medium Wiki

22
Application Lifecycle
23
Developer Resources

• Grid environment is constantly evolving, but
• Useful features persist
• New are constantly being added
• Bugs are being fixed
• Gained knowledge remains relevant, must be
  updated
• Applications can be easily migrated to
  new/updated APIs
• gLite User Guide
• https//edms.cern.ch/file/722398//gLite-3-UserGuid
  e.pdf
• SEEGRID Wiki
• http//wiki.egee-see.org/index.php/SEE-GRID_Wiki

24
Software/Job Management

• Software installation management using ESM tools
• Experiment Software Installation in LCG-2
• http//goc.grid.sinica.edu.tw/gocwiki/Experiments_
  Software_Installation
• SEEGRID Wiki SEE-GRID ESM Software Installation
  Guide
• http//wiki.egee-see.org/index.php/SEE-GRID_ESM_So
  ftware_Installation_Guide
• Site configuration
• Using ESM tools
• Example JDL and shell scripts for installation,
  validation, and removal
• Job management
• EDG WMS API (LCG-2) C Java
• http//www.to.infn.it/grid/workload_management/api
  Doc/edg-wms-api-index.html
• Exec/spawn of shell scripts that create and
  submit JDL simple, but no direct feedback on
  jobs
• gLite
• http//egee-jra1-wm.mi.infn.it/egee-jra1-wm/glite-
  wms-api-index.shtml

25
Data Management

• LCG-2 (LCG-2 User Guide, man pages)
• LCG-UTILS API C/C
• LFC API C/C, Python
• GFAL API C/C, Python
• http//grid-deployment.web.cern.ch/grid-deployment
  /gis/GFAL/GFALindex.html
• SEEGRID Wiki SG Using file replicas and RFIO UI
  configuration, rfiod, usage in apps, limitations
  and workarounds
• http//wiki.egee-see.org/index.php/SG_Using_file_r
  eplicas_and_RFIO_UI_configuration2C_rfiod2C_usa
  ge_in_apps2C_limitations_and_workarounds
• Configuring UI, SE, RB
• Site testing of RFIO/GFAL
• Typical problems and solutions
• Java access to LFC and LCG-UTILS
• SEE-GRID File Management Java API
• http//wiki.egee-see.org/index.php/SEE-GRID_File_M
  anagement_Java_API
• Customizable LFC web front end (upload, list,
  replicate, delete)
• http//grid02.rcub.bg.ac.yu/repmngr/