EU-GRID Work Program

1
EU-GRIDWork Program

• Massimo Sgaravatto INFN Padova
• Cristina Vistoli INFN Cnaf
• as INFN members of the EU-GRID technical team

2
RD Program

• Work Packages (preliminary list)
• Computing Fabric Management (T. Smith)
• Mass Storage Management (O. Barring, J.F. Baud)
• Wide Area Data Management (J. Jaen, B. Panzer, W.
  Hoschek)
• Wide Area Workload Management (M. Sgaravatto, C.
  Vistoli)
• Wide Area Application Monitoring (E. Futo)
• Application Development
• Testbeds (???)
• Presented and discussed during the Grid Workshop
  (7 Mar.)

3
Computing Fabric Management

• The management of the hardware and basic
  software required for the local computing fabric
  the processors, disks, tape drives and robots,
  the equipment providing the local area and
  storage networks, operating system installation
  and maintenance, file systems, SANs, gateways to
  external networking facilities, monitoring
  consoles, power distribution, etc
• No preliminary draft document presented

4
Mass Storage Management

• The software required to manage the mass storage
  system secondary and tertiary storage space
  management, migration of data between tertiary
  and secondary storage, provision of classes of
  service to schedule access to the same data from
  processes with different priorities
• Preliminary draft document presented covering HEP
  needs and state of the art (no details on work
  needed, work plan, )

5
Wide Area Data Management

• Universal name space, efficient data transfer
  between sites, synchronisation of remote copies,
  wide-area data access/caching, interfacing to
  mass storage management systems
• Preliminary draft document (prepared by 8
  persons) presented
• 3 Phases
• 1 year/phase
• 4-5 persons/year (preliminary assumption)

6
Wide Area Data Management - Phase 1

• Design and prototype the basic infrastructure
  (lower
• level services)
• Security
• Data Access
• Data Location
• Data Migration
• Data Replication
• Metadata Publishing and Management

7
Wide Area Data Management Phase 2

• Leverage the lower level services delivered by
  Phase 1,
• and build higher level services on top of them
  (not
• sufficiently well defined)
• Maintain sets of logically related files
• Query optimisation and estimation
• Access Pattern Management
• Transactions
• Transparent Access

8
Wide Area Data Management - Phase 3

• Not yet worked out in detail
• Prototype data grid enabled HEP analysis
  environment
• Provide tools for location and DB independence
  and seamless data integration

9
Wide Area Workload Management

• Scheduling of work submitted by users
• Scheduling support taking account of relative
  priority levels, data location, automated
  parallelisation of work
• APIs and GUIs which provide users access to the
  distributed computing facility
• Preliminary draft document presented

10
Wide Area WorkLoad Management

• Job description and resource specification
• Job decomposition
• Scheduling
• Definition of scheduling policies in order to
  find the best match between job requirements and
  available resources

11
Wide Area Workload Management

• Co-Allocation Reservation
• Co-allocation necessary because many applications
  need simultaneously multiple resources
• Advance reservation necessary to guarantee
  end-to-end QoS
• Resource Management
• Implementing scheduling policies
• Services
• Bookkeping
• Accounting
• Logging

12
Wide Area Application Monitoring

• The instrumentation of applications, and tools
  to analyze dynamically the performance of an
  application in a GRID environment automatic
  recognition of application-level performance and
  resource problems automatic re-configuration of
  the applications resources and rescheduling of
  the application components
• Presentation on Application Monitoring problem
  and status (no details on work needed, work plan,
  )

13
Wide Area Application Monitoring

• Integrate performance measurement with realtime
  performance visualization and adaptive control of
  application behavior and resource policies
• Migration from static resource optimization based
  on a posteriori measurements to closed-loop
  adaptive control using realtime data streams

14
Application Development

• The seamless integration of existing end-user
  applications with GRID middleware and the
  development of new, grid-aware applications is an
  important component of any testbed and pivotal to
  the success of the project
• Not charged to the technical group addressed to
  physicists