gLite WMS Test

1
gLite WMS Test

• New results after the WMS patching in Milan
• Hurng-Chun Lee and Dmitry Ryabchikov
• ATLAS/LCG/EGEE Taskforce
• 25 Nov. 2005

2
Background

• WMS performance is measured in terms of job
  submission and dispatching time calculated by
  using job logging info
• Previous observations of gLite 1.4 WMS test
• Bulk submission is fast ( 3 jobs/sec)
• All jobs in bulk (node jobs) are dispatched to
  the same CE
• The slow match-making ( 0.08 jobs/sec) seems to
  be the first bottleneck of dispatching
• RB does not utilize full CPU power for
  match-making
• Some bug in job list-match
• The patch of WMS is mainly for match-making
  performance enhancement
• Enabling multi-threading for match-making
• Recompiling ClassAds library with O1 optimization

3
Before the WMS patching

• 300 jobs (3 bulks submitted by 3 threads)
• Submission rate 3 jobs/sec
• Match-making rate 0.08 jobs/sec

4
After the WMS patching

• 300 jobs (3 bulks submitted by 3 threads)
• Submission rate 3 jobs/sec
• Match-making rate ( 0.37 jobs/sec) is 4 times
  faster than previous result
• RB utilizes more CPU power in the match-making
  phase
• Condor related processes are also more active
  than before

5
Comparison with LCG
LCG
gLite

• gLite 2-way Intel Xeon 2.8 CPU (with
  hyper-threading), 3 GByte memory
• LCG 2-way Intel Xeon 2.4 CPU (without
  hyper-threading), 2 GByte memory
• Both are using the same BDII

6
Comparison with LCG (with CE requirement)
LCG
gLite
7
Comparison with LCG

• gLite
• Observable effect comes from the number of
  matched CEs
• The inputsandbox effect on submission still not
  be fully understood with the data in the table
• LCG
• Match-making takes place right after submission
• No observable effect from the number of matched
  CEs
• Submission of job with inputsandbox is about 2
  times slower than simple hello world job

8
Number of matched CEs vs. match-making time
0.78 0.036 x
9
Conclusion

• After patching
• The match-making is speeded up by a factor of 4
• All node jobs still go to the same CE
• Comparison with LCG
• For simple hello world job, the performance of
  gLite is similar to LCG
• gLite can be faster than LCG if CE is confined by
  the JDL requirement
• The number of matched CEs effect in gLite is
  not observed in LCG
• With the inputsandbox sharing feature, we expect
  the performance of gLite is better than LCG for
  realistic job
• More tests for
• Understanding the job processing behavior after
  match-making (especially the job transferring
  from RB to CE)
• Understanding the inputsandbox effect
• Understanding if or how interaction with DLI will
  slow down the match-making speed