Iniciativas GRID en la Red Acadmica Espaola

1
Iniciativas GRIDen la Red Académica Española

• GT RedIRIS 2002
• Jesús Marco, CSIC

2
Iniciativas GRID

• Física de Altas Energías
• Desafío del próximo acelerador LHC
• EU-DataGrid (IFAE, testbed)
• CCLHC-ES
• LCG (CERN, participación española)
• LCG-ES
• DataTag
• CrossGrid
• Aplicaciones Interactivas
• Testbed
• Empresas
• EoI 6 PM

3
The Challenge of LHC Computing
CMS
ATLAS
Storage Raw recording rate 0.1 1
GBytes/sec Accumulating at 5-8
PetaBytes/year 10 PetaBytes of
disk Processing 200,000 of todays fastest
PCs
LHCb
4
The Challenge of LHC Computing
Researchers spread over all the world !
Europe 267 institutes, 4603 usersElsewhere
208 institutes, 1632 users
5
The DataGRID project

• Project supported by the EU Fifth Framework
  programme
• Principal goal collaborate with and complement
  other European and US projects
• Project objectives
• Middleware for fabric Grid management
• Large scale testbed
• Production quality demonstrations
• Three year phased developments demos
• Open source and communication
• Global GRID Forum
• Industry and Research Forum
• Main partners CERN, INFN(I), CNRS(F), PPARC(UK),
  NIKHEF(NL),ESA-Earth Observation
• Other sciences KNMI(NL), Biology, Medicine
• Industrial participation CS SI/F, DataMat/I,
  IBM/UK
• Associated partners Czech Republic, Finland,
  Germany, Hungary, Spain, Sweden (mostly computer
  scientists)
• Industry and Research Project Forum with
  representatives from
• Denmark, Greece, Israel, Japan, Norway, Poland,
  Portugal, Russia, Switzerland
• Collaboration with US similar GRID initiatives

6
Replica Catalogue
Information Service
Resource Broker
Storage Element
Logging Book-keeping
Job Submission Service
Compute Element
7
Spanish Participation in DataGRID
WP6 (TESTBED) 2001 IFAE reports in behalf of the
other HEP institutions working in the testbed
workpackage of the DataGrid project in Spain
(IFCA,CIEMAT,UAM,IFIC)         Certification
Authority         Installation kits        
Information servers (GIIS)         Condor batch
system and AFS         DataGrid project web
sites and mailing-lists for Spain
8
The CERN LHC Computing Grid project

• After the CERN Hoffmann review (2000)
• Resource implications presented to LHC experiment
  RRBs in March 2001.
• CERN Management summary presented to SPC and
  Committee of Council in March as white paper.
• Discussions between CERN Management and LHC
  experiment spokespersons.
• LHC turn-on schedule agreed machine-experiments
• CERN/2379 green paper for Council and FC in
  June
• Development and Deployment of LHC Computing Grid
  Infrastructure should be setup and managed as a
  unified project, similar in some ways to a
  detector collaboration. CERN is viewed as the
  institution that should co-ordinate it.
• There should be a Prototyping Phase in 2002-2004.
  The scale and complexity of the development is
  large and must be approached using a Project
  Structure. Work is needed in the Member State
  Institutes and at CERN.
• Human and Material Resources for CERNs part of
  Phase I are not sufficient and should be funded
  by additional contributions from interested
  Member States.
• AGREED!
• Spanish contribution includes fellowships at CERN

9
Spain, 2001 Acción Especial for Local
Infraestructure

• Objective
• Initial seed for LHC Computing at each site
• Trained Personnel
• Startup Hardware
• Trigger participation in
• CERN LHC GRID Computing project
  (ITcollaborations)
• Collaboration software
• GRID projects

10
Deliverables to fulfill the objectives
LCG-ES 3-year project Coordinated by
Manuel Delfino (PIC)
EAD, MCF, GSW, CDC
IFCA
EAD, GSW, CDC
EAD, ETD, PIC, EDS, GSW
USC
IFAE
EAD Analysis Farm EDS SW Dev Platform RSG
SW repository GSW SW Gridification MCF MC
Fabric GVM Virtual MC farm ETD Data
Transform PIC Gridified Data Store SEG
Security Architect CTS Tech MC Support CDC
Data Chal. Coord.
EAD, MCF, EDS, RSG, SEG, CTS
UB
CIEMAT
EAD,EDS, CTS
UAM
IFIC
EAD, GVM
EAD, MCF, CTS, CDC

• Stay away from glitz.
• Concentrate on deployment, MC analysis
• Use local Univ. for TT to other disciplines
• 600 KCHF materials contribution to LCG-CERN

11
The CROSSGRID project

• European Project (Cross Action CPA9,6th IST
  call,V PM) 5 M
• Objectives
• Extending GRID across Europe Testbed (WP4)
• Interactive Applications (WP1) in
• Health Care (vascular surgery)
• Environment (air pollution, meteorology,
  flooding...)
• HEP (Interactive Data Analysis)
• Partners
• Poland (CO, M.Turala), Germany (FZK), Holland,
  Portugal, Greece...(13 countries, 21
  institutions)
• Industry Datamat (I), Algosystems (Gr)
• Spain
• CSIC (IFCA, IFIC, RedIRIS), UAB, USC/CESGA
• Participating in applications
  (environment,HEP), performance and monitoring,
  resource management, testbed (CSIC WP leader)
• Started 1st March 2002
• Q1 Deliverables released ! (including all SRS,
  testbed planning)

12
CrossGrid WP1 Task 1.3 Distributed Data
Analysis in HEP
Coordinated by C.Martinez (CSIC)

• Subtask 1.3.2 Data-mining techniques on GRID
• ANN(Artificial Neural Networks) main tool for
  Data-mining in HEP
• Example of Physics Analysis using ANN

13
HEP Interactive Application
Interactive Session Resource Broker
2
Replica Manager
User
CAS service
3a
3b
1
Portal Authorization DATASET XML Input XML
Output
4
5
DISTRIBUTED PROCESSING
6
DB Installation
Interactive Session Manager
9
Interactive Session Database server
7
8
14
Storage Element as WebService?
David Rodriguez, CSIC

• Current SE in EDG
• GridFTP server
• WebService approach
• Passive SE GridFTP, or /grid , etc...
• Active SE
• SQL QUERY (ResultSet in XML) SELECT FROM
• (Three tier servlet running, like Spitfire)
  ready! (IBM IDS)
• ROOT query (does this make sense? Paw query
  does make sense, implemented...)
• PROCESSING QUERY ( Agent) Stored Procedure or
  XML description (SOAP like?)
• SQL QUERY ok for NN in HEP
• PROCESSING QUERY (Agent-like approach) needed
  likely for SOM

15
HEP Interactive Portal
V.O.Authentication DATASET Resources Monitoring
Graphic Output/(Input?)
DATASET Dictionary (Classes) Basic
Object Derived Procedures
Alphanumeric Output
Analysis Scripts
Work Persistency
16
Distributed (via MPI) NN training scaling
644577 events, 16 variables 16-10-10-1
architecture 1000 epochs for training

• First checks with nodes at Santander RedIRIS
• (Oscar Ponce Antonio Fuentes)
• remote configuration modelling including
  latency
• lt100 ms needed !

17
SOM Application for DataMining
Adaptive Competitive Learning
Downscaling Weather Forecasts
Sub-grid details scape from numerical models !!!!!
18
Atmospheric Pattern Recognition
Prototypes for a trained SOM. Close units in the
lattice are associated with similar atmospheric
patterns.
T 1000mb
T 500mb
Z, U, V 500mb
19
CrossGrid Architecture (OGSA in mind)
1.4 Meteo Pollution
1.3 Data Mining on Grid (NN)
1.3 Interactive Distributed Data Access
1.2 Flooding
1.1 BioMed
Applications
3.1 Portal Migrating Desktop
2.4 Performance Analysis
2.2 MPI Verification
2.3 Metrics and Benchmarks
Supporting Tools
Applications Development Support
1.1 Grid Visualisation Kernel
MPICH-G
1.1, 1.2 HLA and others
3.1 Roaming Access
3.2 Scheduling Agents
1.2 Distributed Data Collection
1.1, 1.3 User Interaction Services
Grid Common Services
Globus Replica Manager
3.3 Grid Monitoring
DataGrid Replica Manager
DataGrid Job Manager
3.4 Optimization of Grid Data Access
Replica Catalog
GRAM
GSI
GIS / MDS
GridFTP
Globus-IO
1.1 Resource Manager
1.1 Resource Manager
Local Resources
Resource Manager (CE)
3.4 Resource Manager
Resource Manager (SE)
1.1, 1.2 Resource Manager
VR systems (Caves, immerse desks)
Visualization tools
CPU
Secondary Storage
Scientific Instruments (Medical Scaners,
Satelites, Radars)
3.4 Optimization of Local Data Access
Tertiary Storage
20
CrossGrid WP4 - International Testbed Organisation

• Objectives
• Testing and validation for
• Applications
• Programming environment
• New services tools
• Emphasis on collaboration with DATAGRID
• extension to DATATAG
• Extension of GRID across Europe

21
CROSSGRID testbed
TCD Dublin
PSNC Poznan
UvA Amsterdam
ICM IPJ Warsaw
FZK Karlsruhe
CYFRONET Cracow
II SAS Bratislava
USC Santiago
CSIC IFCA Santander
LIP Lisbon
Auth Thessaloniki
UAB Barcelona
CSIC RedIris Madrid
CSIC IFIC Valencia
UCY Nikosia
DEMO Athens
22
CrossGrid WP4 - International Testbed Organisation

• Tasks in WP4
• 4.0 Coordination and management
• (task leader J.Marco, CSIC, Santander)
• Coordination with WP1,2,3
• Collaborative tools (webvideoconfrepository)
• Integration Team
• 4.1 Testbed setup incremental evolution
• (task leaderR.Marco, CSIC, Santander)
• Define installation
• Deploy testbed releases
• Certificates
• Security Working Group
• A.Fuentes RedIRIS

• Testbed site responsibles
• CYFRONET (Krakow) A.Ozieblo
• ICM(Warsaw) W.Wislicki
• IPJ (Warsaw) K.Nawrocki
• UvA (Amsterdam) D.van Albada
• FZK (Karlsruhe) M.Hardt
• IISAS (Bratislava) J.Astalos
• PSNC(Poznan) P.Wolniewicz
• UCY (Cyprus) G.Tsouloupas
• TCD (Dublin) B.Coghlan
• CSIC (Santander/Valencia) J.Sanchez
• UAB (Barcelona) E.Heymann
• USC/CESGA (Santiago) C.Fernandez
• Demo (Athenas) Y.Cotronis
• AuTh (Thessaloniki) C.Kanellopoulos
• LIP (Lisbon) J.Martins

23
CrossGrid WP4 - International Testbed Organisation

• Tasks in WP4
• 4.2 Integration with DATAGRID (task leader
  M.Kunze, FZK)
• Coordination of testbed setup
• Exchange knowledge
• Participate in WP meetings
• 4.3 Infrastructure Support (task leader J.Salt,
  CSIC, Valencia)
• Fabric management
• HelpDesk
• Provide Installation Kit
• Network support
• QoS (working group, I.Lopez CESGA)
• 4.4 Verification quality control (task leader
  J.Gomes, LIP)
• Feedback
• Improve stability of the testbed

JOINING DataGrid testbed 1.2 in July 2002
24
Hands on GRID _at_ IFCA (http//grid.ifca.unican.es/)
25
IFCA

• Research Institute
• University of Cantabria
• Consejo Superior de Investigaciones Científicas
• Three main research lines
• Astrophysics (XMM, Planck...)
• Statistical Physics (Lasers, fractals chaos...)
• High Energy Physics
• DELPHI, LEP (Physics Analysis)
• CDF, Fermilab (TOF detector Physics Analysis)
• CMS, LHC (Alignement Geant4 Sim, OSCAR)
• Common Interest
• Computing needs
• Data Management
• Advanced Analysis Techniques
• Optimize resources for infraestructure manpower

26
HEP Computing at IFCA

• Previous experience
• DELPHI Fast simulation
• RPC software for DELPHI on-line
• Analysis software for DELPHI (NN, IES...)
• Initiatives
• Databases (use of O/R DBMS in HEP)
• FEDER project with DB software company (Semicrol)
• GRID Initiatives
• DataGRID testbed site CA for Spain
• CROSSGRID WP1 (HEP appl, meteo), WP2, WP4
  (testbeds)
• Technology transfer with companies (Mundivia,
  CIC)
• Participation in testbed of DataTag (CDF)
• Computing for LHC (CMS)

27
GRID team in Santander

• Research line at IFCA ( Univ.Cantabria CSIC )
• staff contracts fellowships
• Expertise
• Databases use
• Testbed issues (cluster installation, security,
  CA, etc)
• Applications
• Astrophysics
• Complex systems
• HEP
• Meteo
• Collaboration and support (via projects) on
• NN, methods Dpto Matematicas
• Clusters MPI Grupo de Arquitectura de
  Computadores
• Network Centro de Calculo U.C.
• Companies
• Mundivia
• CIC-SL
• Semicrol

28
Resources

• New IFCA building
• with support for e-Science activities (2002/2003)
• New Infrastructure
• Cluster 100 IBM servers (100 available for
  GRID)
• (dual 1.26 GHz, 640Mb-4GB RAM, 80 GB/server)
• 4-way processor gatekeeper
• Gigabit local backbone
• Improved network connection
• 155 (?) Mbps Santander-RedIRIS (Geant node)

29
72 Computing Elements. Worker Nodes.8 Storage
Elements

• IBM xSeries 220
• 1 CPU 1.26 GHz
• 128Mb512Mb SDRAM
• Hard Disk
• SCSI 30Gb
• IDE 60Gb
• Network 100 Mbps
• CDROM, floppy
• Two machines with 4Gb SDRAM for tests

NEXT UPDATES 8 Network Cards 1000Mbps (for
Storage Elements,...) Join 1.26GHz CPUs in dual
setup Buy new gt1.4GHz CPUs
30
Remote Automatic Installation

• Nodes configured for PXE Boot
• Installation Server DHCP,NFS, TFTP
• 1 server for LCFG
• 1 server for PXE-Linux Kickstart
• Help sources
• PXE-Linux (from SYSLINUX, http//www.syslinux.org)
  http//syslinux.zytor.com/pxe.php
• HOWTO Install RedHat Linux via PXE and Kickstart
  http//www.stanford.edu/alfw/PXE-Kickstart/

31
A new IST Grid project space (Kyriakos
Baxevanidis)
32
EoI 6 PM (7 Junio 2002)
Proyecto Integrado EGEE (coordinado CERN)

• CSIC
• RedIRIS
• IFCA (Santander)
• IFIC (Valencia)
• IMEDEA (Palma)
• CAB (Madrid)
• CNB (Madrid)
• CBM (?) (Madrid)
• IAA (Granada)
• Centros
• CIEMAT (Madrid)
• IFAE (Barcelona)
• PIC (Barcelona)
• CESGA (Santiago)
• IAC (Tenerife)

• Universidades
• U. Cantabria
• U. Valencia
• U. Murcia
• U.A.Barcelona
• U.A.Madrid
• U.Complutense Madrid
• PYMES
• CIC-S.L. (Cantabria)
• GridSystems (Palma)

33
EoI 6 PM (7 Junio 2002)

• Red de Excelencia RTGRID
• (Real Time GRIDs)
• España
• CSIC
• Univ.Cantabria
• CESGA
• CIC-SL
• Polonia
• Cyfronet
• Grecia
• Univ. Athenas
• Univ. Thessaloniki
• Slovakia
• IISAS Bratislava
• Cyprus
• Univ. Cyprus

• Otras propuestas
• CEPBA
• UPV ?
• ...

34
In perspective

• GRIDs will help with
• Organizational and large scale issues
• Metacomputing
• Web Services are commercial
• OGSA could be the way if performance is ok
• Interactive Grid will be hard without QoS on
  networks
• Several GRID projects with Spanish participation
  progressing well
• Need for organization in Spain
• Thematic Network Teams to organize work
• e-Science centers to get local support,
  administrative organization, dissemination and
  exploitation (we need companies involved)