S pe a t Ethernet e eta d

1
S????????? ?p????????e? ??a ???t?? Ethernet µe
?eta???? ?µ?d??

• ?a???s?as? ??eµµ?d?? ?????
• S??ßa??da? ?????

2
?e??e??µe?a

• Ge????
• ?p?ßa???
• ?e????af? d??t???
• ????? s?ed?as?? a??????µ??
• ?e?t?st?p???µ???? a??????µ??
• ???????µ?? ??a ??PC
• ???????µ?? ?????p????aµµat?sµ?? f?se??
• ???????µ?? ?????? a?ta??a???
• ??t?µat? pa?a???? ???t????
• ?p??tas? t?? MPI ??a s????????? ep????????e?
  (CC-MPI)
• Open MPI ?a? s????????? ep????????e?

3
Ge???? (1/2)

• S????????? ep????????e?
• ??a?-????-????? / ??a?-????-???????
• ??p?µp? (MPI_Bcast)
• ??as???p?s? (MPI_Scatter)
• ????-????-????? / ????-????-??????? (AAPC)
• ????? a?ta??a?? (MPI_Alltoall)
• ????? s?????? (MPI_Allgather)

4
Ge???? (2/2)

• ?e????af? p??ß??µat??
• ?p?????se? ???p???se?? de? ?aµß????? ?p???
  a???te?t????? s?st?µat??
• ?? ap?d?t???? ???t??e?
• ??t?µet?p?s? p??ß??µat??
• ??as??des? ??µß?? se Switched Ethernet
• ????? ap?d?s? ???? ap?f???? s?µf???s??
• ?µpe?????? p??se???se?? ??a pa?a???? ap?d?t????
  ???t????
• ?p?d?t??? p?a?s?? ???p???s?? ap? t? Open MPI

5
?p?ßa??? (1/2)

• ???t?? Switched Ethernet
• ??µß?? ?a? switches
• ???e ??µß?? s??d?eta? se µ?a ?a? µ??? µ?a ???a
  e?s?d??/e??d?? t?? switch
• Full duplex s??desµ??
• ?e??pt?se?? Switched Ethernet
• ??µß?? s??dedeµ???? se ??a µe???? switch
• ??µß?? s??dedeµ???? se d?af??et??? switches ?a?
  switches s??dedeµ??a se µe?a??te?a switches
  d?µ???????ta? ?e?a???a (?e?a????? µ??t???)

6
?p?ßa??? (2/2)

• ????? s?ed?as?? a??????µ??
• ?p?f??? s?µf???s?? ?a? ad?e??d??
• ??a??st?p???s? µet?d?s?? ?d??? µ???µ?t??
• ?a??????? µet?d?s?
• ????p???s? ???? t?? s??d?sµ??
• ?a?a?????p???s? µ???µ?t?? p?? p???????ta? ap?
  d?af??et????? ??µß???

7
?e?t?st?p???µ???? a??????µ?? (1/2)

• ???????µ?? t?? Bruck
• ???????µ?? ??a µ???µata µ????? µe??????
• logp ß?µata
• ??µa n d?e??as?a i st???e? st?? i2n ta µp??? µe
  n-?st? bit 1

8
?e?t?st?p???µ???? a??????µ?? (2/2

• ???????µ?? t?? da?t?????
• ???????µ?? ??a µ???µata µe????? µe??????
• ??e??as?e? s??dedeµ??e? se e??????? da?t????
• p-1 ß?µata
• ??µa n d?e??as?a i st???e? st?? (in)mod(p)
• ?a?a??a???
• ???s? light barriers (ap?f??? a?ta????sµ??)
• ???s? MPI_Barrier (ap?f??? s?µf???s?? d??t???)

9
???????µ?? ?????p????aµµat?sµ?? f?se?? (1/3)

• ?e????af? ???????a?
• ?p?d??d?? switch µe ??µß???
• ???a switch ?????te??? ep?p?d?? p?? s??d?e?
  ?p?d??d?a
• F?s? s????? µ???µ?t?? p?? de? pa???s???e?
  s?µf???s?
• ?a?????? µ???µata µ???µata µeta?? ?p?d??d???
• ??p??? µ???µata µ???µata µ?sa sta ?p?d??d?a

10
???????µ?? ?????p????aµµat?sµ?? f?se?? (2/3)

• ??µ??????a f?se??
• ???es? ???a?
• ???sd????sµ?? f?se??
• ????µata ap? ??a ?p?d??d?? p??? ??a ????
  t?p??et???ta? se d?ad?????? f?se??
• ???s?µ?p???s? a??????µ?? da?t????? ??a
  p??sd????sµ? f?se??

• ????es? t?p???? ?a? ?a??????? µ???µ?t??
• ?tete?µ???st???? t??p?? a???es??
• ????es? ?a??????? µ???µ?t?? se f?se??
• ????es? t?p???? µ???µ?t?? se f?se??

11
???????µ?? ?????p????aµµat?sµ?? f?se?? (3/3)

• ??µ??????a µ????? a???µ?? f?se??
• ?e? ?p???e? s?µf???s? µ?sa se ???e f?s?
• ???e µ???µa eµfa???eta? µ?a f??? se ???e f?s?

12
???????µ?? ?????? a?ta??a???

• ????p??e? s??d?sµ??? p???p?????ta? µ???µata
• ???s?µ?p??e? s??µa e?????? s?µf???s??
• Se ???e ??µß? a?at??eta? p???a????sµ??? ????
  ????s?? (pa??????)
• ??µß?? de? ep?t??peta? ?a ?epe????e pa??????
• ???e ??µß?? ep???????e? µe ??µß??? p?? ß??s???ta?
  se d?af??et??? switch se ???e ß?µa (s??µa
  a?ad??ta???)

13
??t?µat? pa?a???? ???t???? (1/2)

• S?st?µa a?t?µat?? pa?a????? ???t???? ??a
  s????????? ep????????e?
• ??µß?? s??dedeµ???? se d??t?? Ethernet
• ???s? eµpe?????? p??s????s?? ??a ?a????sµ?
  ß??t?st?? a??????µ?? ??a ???e s????????
  ep????????a
• ??µ??????a ???t???? p??sa?µ?sµ???? ap???ta st??
  apa?t?se?? t?? e??st?te p?atf??µa? ?a? efa?µ????

14
??t?µat? pa?a???? ???t???? (2/2)

• ??µa 1 ??s?d?? ap? t? ???st?
• 1.1 S???????? ep????????a p?? ?a ???µ?ste?
• 1.2 Ge???t??a p?? ?a ???s?µ?p????e?
• 1.3 ???a??sµ?? ?????µ?t??s?? p?? ?a
  ???s?µ?p???????
• 1.4 ???e?? pe????af?? t?? t?p?????a? t?? d??t???
• 1.5 ?p????? ??a t? ???s? e?d???? ??a t??
  t?p?????a ???t????
• ??µa 2 ?a?a???? ???t???? p?? ape??????ta? st??
  t?p?????a
• ??µa 3 ???µ?s? a??????µ?? µe pa?aµ?t????
• 3.1 ???es? t?µ?? ??a t?? pa?aµ?t????, ??a ??a
  s????? ap? µe????
• µ???µ?t??
• 3.2 ???es? e?e???? t?? µe?e??? ??a ta ?p??a ??
  t?µ?? t?? pa?aµ?t???
• a??????? (d?ad??? a?a??t?s?)
• 3.3 ?a?a????? ???t??a? µe t?? ß??t?ste? t?µ??
  pa?aµ?t???
• ??µa 4 ?e???? ???µ?s?, pa?a???? te????? ???t??a?
• 4.1 ???es? ?a??te??? a??????µ?? ??a ta µe???? t??
  s??????
• 4.2 ???es? e?e???? t?? µe?e??? ??a ta ?p??a ??
  ß??t?st??

15
?p??tas? t?? MPI ??a s????????? ep????????e?
(CC-MPI) (1/4)

• ??a?-????-????? / ??a?-????-???????
• ???p???s? MPI_Bcast, MPI_Scatter
• ??a?µat?p???s? µe multicast ???p???se??
• ????p?st?a ???s? a???p?st?? p??t???????
  multicast µe ACKs
• ??a?e???s? ?µ?d?? t??a s??µata
• S??µa stat???? d?a?e???s?? ?µ?d??
• S??µa d??aµ???? d?a?e???s?? ?µ?d??
• S??µa d?a?e???s?? ?µ?d?? ?p?ß?????µe?? ap?
  compiler

16
?p??tas? t?? MPI ??a s????????? ep????????e?
(CC-MPI) (2/4)

• S??µa stat???? d?a?e???s?? ?µ?d??
• ???e multicast ?µ?da s?et??eta? µe ??a
  communicator (?µ?da d?e??as??? p?? s?µµet??e?
  st?? ep????????a)
• ?µ?da ?atas?e???eta?/?atast??feta? ?ta?
  communicator ?atas?e???eta?/ ?atast??feta?
• S??µa d??aµ???? d?a?e???s?? ?µ?d??
• ?t?sµ??? p??? ap? s??µa stat???? d?a?e???s??
• ?µ?da ?atas?e???eta?/?atast??feta? ?ta? p??pe?
• S??µa d?a?e???s?? ?µ?d?? ?p?ß?????µe?? ap?
  compiler
• ?p??tas? MPI d?epaf?? ??a d?a?e???s? ?µ?d?? ap?
  ???st? (pa???? ?at??????? ???t????)

17
?p??tas? t?? MPI ??a s????????? ep????????e?
(CC-MPI) (3/4)

• ????-????-????? / ??????-????-???????
• ???p???s? MPI_Alltoall
• ??a?µat?p???s? µe unicast ???p???se??
• ??µ??????a f?se?? ??a ap?f??? s?µf???s??
• ?????p????aµµat?sµ?? µ???µ?t??
• ?p??st?? a??????µ??
• ?a????µe? µ???µata µe ß?s? t? µ??e???
• ?a???e? µ???µata (ap? µe?a??te?? se µ????te??)
  p?? de? ????? ?????p????aµµat?ste? ?a? ta
  t?p??ete? se f?se?? µe p???p??es? ?a µ?
  d?µ?????e?ta? s?µf???s?

18
?p??tas? t?? MPI ??a s????????? ep????????e?
(CC-MPI) (4/4)

• ????d?? ??a ep????????e? ??????-????-???????
• ?p?? ep????????a ???s? ???t????
  s?µe???-µe-s?µe??
• ?p????????a µe f?se?? ?????p????aµµat?sµ?? ?a?
  µet?d?s? µ???µ?t??
• ???p???s? µe??d?? ap? CC-MPI
• ???t??e? µet?d?s?? ded?µ???? ??a
• ep????????a s?µe???-µe-s?µe??
• ???p???s? ep????????a? µe f?se??
• ???t??e? e?????? d??t??? ??a
• e?t??es? t?? MPI_Allgather
• ???s? ???t??a? ?????p????aµµat?sµ?? µ???µ?t??

19
Open MPI s????????? ep????????e?(1/3)

• Open MPI
• ??a ???p???s? t?? MPI-2
• ?a???e? ??a p?a?s?? e??as?a? p?? ep?t??pe? t??
  ap?d?t???te?? ?e?t?????a t?? s?????????
  ep?????????? coll
• coll pa???e? e??a?e?a ??a s?ed?as?, ???p???s?
  ?a? pe??aµat?sµ? ???? a??????µ?? ??a s?????????
  ep????????e?
• ??e? ???t??e? t?p??et???ta? se a?e???t?ta tµ?µata
  (components) p?? a?a????????ta? ap? t?? ???p???s?
  t?? MPI se run-time

20
Open MPI s????????? ep????????e?(2/3)

• ??????e? p?? pa?????ta? ap? t? Coll
• M? t??p?p???s? ??d??a ??a p??s???? ???? ???t????.
• ???s???? ???? ???t???? se compile-time ?a?
  run-time
• ???p???s? ???t???? p??? ap? ???t??e?
  s?µe???-µe-s?µe??
• ?p?t??pe? t?? t?p???t?s? tµ?µ?t?? p??
  pe???aµß????? ???t??e? p??? se ???a tµ?µata µe
  ???t??e?.
• ??e? ???t??e? µp????? ?a e?µeta??e???ta? t??
  t?p?????e? t?? d??t???.
• ???s?µ?p???s? ???? ???t???? ????? e? ????
  µeta???tt?s? ?a? epa?as??des? t?? efa?µ????
• ?p?t??pe? se µ?a MPI d?e??as?a ?a ???s?µ?p??e?
  p???ap?? a?e???t?ta tµ?µata p?? pe???aµß?????
  ???t??e?.

21
Open MPI s????????? ep????????e?(3/3)

• ?????? ???? e??? coll module
• Selection d?µ??????a communicator ?a? ep?????
  ?at??????? component
• Initialization ???s? ???t??a? a?????p???s?? t??
  component ?a? ap????e?s? p????f????? st??
  communicator
• Checkpoint/restart d??at?t?ta epa?e?????s?? t??
  efa?µ????
• Normal usage ?? ???t??e? t??e?ta? se ?e?t?????a
  µet? ap? s?et??? ???s? st?? communicator
• Finalization ?atast??f? communicator ?a?
  ?atast??f? module

22
???t?se??

• ?