Distributed Policy Management and Comprehension with Classified Advertisements

1
Distributed Policy Management and Comprehension
with Classified Advertisements
2
A Distributed Policy Scenario

• A user submits a job to Condor
• The user has designed a policy defining requested
  services
• Machines in condor pool have policies restricting
  the use of services
• The users job wont run - Why?
• Is users policy to restrictive?
• Was job rejected by machine policies?

3
Policy Management

• Resource allocation challenges
• Resource heterogeneity
• Policy heterogeneity
• How to allocate resources?
• Conventional centralized allocation not
  sufficient
• Solution Matchmaking with Classified
  Advertisements (ClassAds)

4
Matchmaking
?
Job
5
Matchmaking
?
Job
6
Matchmaking
?
Job
7
Matchmaking
?
Job
8
Classified Advertisements

• Represent entities (e.g. jobs, machines) and
  their policies
• A ClassAd is a set of named expressions called
  attributes
• Types of attributes
• Characteristics of an entity (Arch, OpSys,
  Memory)
• Constraints for requested resource (Requirements)
• Preferences for requested resource (Rank)

9
Typical Classads
Type Job Owner
ncoleman Cmd run_sim Memory
31m Rank KFlops/1E3
other.Memory/32 Requirements (other.Type
Machine) (other.Arch INTEL)
(other.Opsys LINUX) (other.Memory gt
128)
Type Machine KeybrdIdle
002312 Memory 256M LoadAvg
0.042969 Kflops 21893 Arch
INTEL OpSys LINUX Name
foo.cs.wisc.edu Rank
(DayTime() gt 900) ((DayTime() lt
1700) ? 1/other.ImageSize 0)
Requirements (other.Type Job)
(other.Owner ! riffraff) (LoadAvg lt 0.3)
(KeybrdIdle gt 0015)
10
Policy Comprehension

• Why wont my job run?
• My policy is too restrictive
• My job is rejected by machines in the pool
• Looking for answers
• Use condor tools (condor_q, condor_status)
• Stare at job ClassAd to find out whats wrong

11
Condor Tools

• condor_q analyze
• Of 105 resource offers,
• 105 do not satisfy the request's constraints
• 64 resource offer constraints are not satisfied
  by this request
• User wants more details
• Which parts of job requirements expression are
  problematic?
• Is job ClassAd missing any attributes?

12
Two Cases to Examine
1. No machines meet the jobs requirements 2. The
job does not meet any machines requirements One
or both of these issues may be preventing the job
from running, but they are not interdependent.
We can analyze each one separately.
13
JOB
Example 1
Requirements (ArchSPARK) (OpSysSOL
ARIS2.7)
14
JOB
Example 1
Requirements (ArchSPARK) (OpSysSOL
ARIS2.7)
Result (Arch SPARK") did not match -
suggestion REMOVE (Opsys "SOLARIS2.7")
matched 2 - suggestion KEEP
15
JOB
Example 2
Requirements (ArchALPHA)
(OpSysWINNT) (Memorygt64)
16
JOB
Example 2
Requirements (ArchALPHA)
(OpSysWINNT) (Memorygt64)
Result (Arch "ALPHA") matched 1 -
suggestion REMOVE (OpSys "WINNT") matched 2
- suggestion KEEP (Memory gt 64) matched 4 -
suggestion KEEP
17
JOB
Example 3
Owner jsmith ImageSize 120000
Requirements ...
MACHINES
Requirements (ImageSize lt 50176)
(MemoryReq lt 49)
1
Requirements (ImageSize lt 50176)
(MemoryReq lt 49)
4
Requirements (ImageSize lt 50176)
(MemoryReq lt 49)
2
Requirements (ImageSize lt 115712)
(MemoryReq lt 98)
5
Requirements (ImageSize lt 115712)
(MemoryReq lt 98)
3
18

115712
3,5
ImageSize
50176
1,2,3,4,5
3,5
0

49
98
MemoryReq
19
JOB
Example 3
Owner jsmith ImageSize 120000
Requirements ...
Results of Test --------------- The following
attributes are missing from the job classad
MemoryReq The following attributes should be
added or modified ImageSize - suggestion
use a value less than or equal to 50176
MemoryReq - suggestion use a value less than 49
20
Current Work

• ClassAd analysis prototype implemented in Java
• Job requirements analysis
• Machine requirements analysis
• Current version supports a simple menu driven
  interface
• Working on integrating with Condor tools
• condor_q analyze
• condor_status

21
Future Work

• Applications to other uses of ClassAds in Condor
• Analysis of a successful match
• Graphical interface
• Analysis of gang matching
• ClassAds as an authorization language

22
Conclusions

• Automated fine grained policy expression analysis
  is useful and feasible
• Different issues arise with job requirements
  analysis and machine requirements analysis
• The ClassAd language is ideal for these purposes.