GLAST Charts

1
Peta-Cache Electronics Discussion
II Presentation Ryan Herbst, Mike Huffer,
Leonid Saphoznikov Gunther Haller haller_at_slac.sta
nford.edu [phone]
2
Flash Storage Option

• Skim Builder as in option discussed earlier
• Event server (cache box) as in other option
• Shown as two boxes for simplicity, could be in
  one box (there are pros and cons)
• Issue is again interconnect speed
• Up to 16 1-Terra-Byte Flash boxes for each event
  server
• Each lane PCI-E 256 MByte/sec
• 16-lanes gives total of 4-Gbytes/sec bandwidth
• Each Flash box has only fraction of total event
  store
• Flash has limited write-cycles so cant
  frequently rewritten (need to enforce with some
  policy which is most important)
• But dont really want to burn results of skim
  in flash, since goal is to make own lists (and
  flash cant be reburned at will anyways)
• Flexibility
• One event server can have a sub-set of the list
  and events go to client
• Or, better, have total of event server as one
  cache and event store is managed so that parts
  of the list which are in other pizza boxes are
  kept in that cache as opposed to discarded
• Question is again how to populate the Flash most
  effectively
• Decompression in event server
• Flash bad-block management in event server
• Reed-Solomon EDAC in event server
• Can consider without cache box 4000 clients
  going after the same block, the last one to get
  data is 300 msec later.

Disk Storage
Tape
16
16
1
1
Flash Storage
Disk Storage
Flash Storage
PCI-E
Event Server
Skim builder (s)
Event Server
Ethernet/PCI-E/etc
Optionally direct IO
Client (1, 2, or 4 core)
Client (1, 2, or 4 core)
Client (1, 2, or 4 core)
Up to 1,500 cores in 800 units?
3
Flash Box
16
16
1
Flash-Chip
Flash-Chip
Xilinx
Xilinx
e.g. PCI-E

• Just as an example, Flashs connect directly to
  Xilinx so page access is not serialized.
• Xilinxs up to 900 user-IOs
• Page aess 20usc for 2kbytes
• Random Access 20 usec
• Serial access 50 nsec (for 4-Gbyte device)

4
Flash-Box, Event Box

• Flash Memory Box
• 8-, 16-, or 32-Gbit NAND devices
• For 1 Terra-Byte need 250 each 32-Gbit devices
• All on board, or
• 32 G-Byte memory cards (DIMM)
• Need 30 DIMMs
• Preliminary placement on 19-inch rack PCB shows
  that we can fit 1 Terra-byte on single board
• PCI-E to PCI-X bridge (to get 64-bit addressing
  space )
• No smarts in here
• Event (Pizza) Box
• 8 F40 Xilinx (each has 2 450-MHz PPCs)
• 16 GBytes of RLDRAM2
• 8 PLX8508 PCI-E switch 5-ports
• 2 PLX8532 8-port switch (32 lanes)

5
Pizza box block diagram (needs some modification)
(In) PCI Express
x16
PLX8532
x4
PPC 405
PLX8508
RLDRAM II IGbyte
x4
x16
PLX8508
XILNIX XC4VFX40
(Out) PCI Express
PLX8532
6
Flash Event Server Boxes

• Flash Box

• 4Gbit chips 30, 8Gbit 60
• 4 Gbyte device quote 110 min qty 1000 (is 4-1GB
  die stack)
• 1 Peta-Byte 1,000 boxes total 27 Mill

• Event Server