SPHINX: A Scheduling Middleware

1
SPHINX A Scheduling Middleware

• University of Florida
• Jang-uk In

2
SPHINX Features (Cont.)

• Grid information utilization
• Execution time estimation
• Various performance on heterogeneous resources
• Resource description
• Characterize resources with specific property
  information
• Usage policies
• Resource usage constraints in job execution

3
SPHINX Features (Cont.)

• Grid information utilization
• Grid weather
• Monitoring dynamically changed load and
  availability
• Replica management
• Discovering available replica for job planning
• Global job descriptions
• Balancing between local optimization and overall
  efficiency

4
SPHINX Features (Cont.)

• System information utilization
• Distributed, fault-tolerant scheduling
• Avoid a single point of failure
• Customizability
• Flexible system for particular scheduling
  requirements of VOs
• Extensibility
• Hierarchical architecture of a grid for
  scheduling

5
SPHINX Features (Cont.)

• System information utilization
• Interoperability with other systems
• Interaction with other modules in a scheduling
  package
• Interaction with other planning systems (Editors)
• Quality of service
• Allowing privileged usage of resources
• Classified usage of Grid resources

6
SPHINX Scheduling Client

• Interact with request submitter
• Abstract DAG produced by a workflow planner
• Chimera virtual data system
• Message passing using Clarens
• To replace the current XML-RPC protocol

7
SPHINX Scheduling Client (Cont.)

• Job submission
• Submitting jobs to grid resources using Condor-G
  etc.
• Job tracking
• Monitoring job execution status
• Flexibility to relocate/replace modules
• Job tracking can be located in the server
• Independent modulated development

8
SPHINX Scheduling Server

• Control process
• Management of scheduling components for
  processing stateful entities (jobs and DAGs)
• Message interface
• Abstraction layer
• Maintenance of currently connected clients

9
SPHINX Scheduling Server (Cont.)

• DAG reducer
• DAG pruning by checking the existence of
  intermediate and/or final output data
• Prediction engine
• Estimated resource usage requirement
• Estimated execution time

10
SPHINX Scheduling Server (Cont.)

• Execution planner
• Resource allocation to jobs
• Execution location and time assignment
• Scheduling strategies
• Policy based scheduling
• Data availability, etc.
• Data replication
• Hot spot management

11
SPHINX Scheduling Server (Cont.)

• Grid monitoring interface
• Interface to external monitoring services (MDS,
  GEMS, MonALISA)
• Scheduling databases
• Communication interface of scheduling modules
• External
• Transformation catalog
• Replica catalog
• Internal sphinx database