GRAPES3 ASET SERIES

1
GRAPES-3 ASET SERIES

• Overview 03-Oct-2008 Atul Jain
• HPTDC I 17-Oct-2008 K.C. Ravindran
• HPTDC II 24-Oct-2008 Ajai Iyer
• MCA 31-Oct-2008 P. Jagadeesan
• Monitoring tools 07-Nov-2008 S.
  Muragapandian
• Computer Cluster I 14-Nov-2008 R. Suresh
  Kumar
• Computer Cluster II 21-Nov-2008 B. Rajesh
• e-Administration 28-Nov-2008 V.
  Viswanathan

1
2

• Plan of talk
• What is a Time to Digital Converter (TDC) ?
• Need of TDC in GRAPES-3 Experiment
• Development of TDC32 module
• High Performance TDC (HPTDC) specification
• Techniques used in HPTDC
• Results

K.C.Ravindran, CRL, TIFR, OOTY
3
What is Time To Digital Converter (TDC)

• Stop watch, ticks in sec
• Used to count lapsed time
• Not possible to distinguish between
• 10.3 sec and 10.4 sec
• Faster clock needed which ticks every 1/10
• of second
• Better accuracy faster clock
• TDC used for high energy experiments
• Used to find small differences in time
• Time difference to be measured is of the
• order of 10 -9 seconds (1 ns) or better

4
Extensive Air Shower

• A high energy primary particle entering the
  atmosphere produces secondary particles through
  interactions in the atmosphere
• Secondary particles travel almost at the speed
  of light (relativistic particles)
• Light takes 3.3 ns to travel 1m
• Observer at Ground

5
Role of TDC in Extensive Air Shower

• If the Air Shower is vertical then the arrival
  time of particles in the detectors are same.
• If the Air Shower is inclined at an angle then
  the particles arrive at the detectors at
  different times
• Time difference Dsin(?)/c where D is
  distance between detectors, ? is incidence angle
  and c is speed of light
• Using Time to Digital convertors the time
  difference can be found which allows us to
  calculate the incidence angle of the shower
• For example if ?30 degrees and D 8m then the
  time difference is 13.3 ns

6
Basic elements for each scintillation detector at
GRAPES-3 Experiment
CAEN QDC
Philips TDC
Philips TDC
Philips TDC
TRIGGER SELECTION
GRAPES TDC
GRAPES TDC NEEDS JTAG
GRAPES TDC NEEDS JTAG
6
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
7
TDC Operation Modes
START/STOP MODE
t 0
t1
TRIGGER MODE
t 0
Trigger generated from Start. Programable match
window No delay cables
and multi-hit feature
7
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
8
Delay cables used for delaying stops in
Start-Stop mode
9
Technique used in TDC32

• Simple counter based TDC

Start
For 500 ps resolution the clock has to be 2 GHz

• Technique used in TDC32

• 1 clock period 1/ (60 X 106) 16.6 nS
• Using 32 delay elements in the DLL this 1 clock
  period is divided into 32
• slots 520.8 ps (16.666ns /32 520.8 ps)
• Which slot out of 32 the stop comes encoded to
  5 bit vernier
• Resolution of 520.8 ps working at 60 MHz
  frequency

10
Delay Locked Loop
Clock Out
Charge Pump
Voltage Controlled delay lines
Clock
Phase Detector

• Clock delayed by delay lines
• Clock Out is compared with Clock
• If Phase difference is there a correction voltage
  generated till
• the total delay introduced is equal to 1
  clock cycle
• Takes time to reach this state and is called
  locking time and depends
• upon the design ( few milli seconds)
• TDC32 has 32 such delay elements

11
Specifications of TDC32 ASIC

• Number of channels 32 1 Common start
• Clock frequency 20-60 MHz
• Time bin size 520ps _at_ 60MHz
• Dynamic range 21 bits
• Power supply 4.75-5.25Volts, 100mA _at_ 60MHz
• Temperature range -40 to 80oC
• Hit input Standard TTL

• Chip designed by
• JÖrgen Christiansen
• Microelectronics Group
• CERN, Geneva

12
Calibration of GRAPES TDC32
Resolution 520.8 ps/count
13
Non-linearity of TDC
Phillips TDC
GRAPES TDC
33 ps rms
14
GRAPES and PHILLIPS TDCs with real data!
EAS DATA OF SAME CHANNEL
15
GRAPES TDC32 Module
60 MHz CRYSTAL
JTAG
CAMAC CONTROL
INPUT OUTPUT BUFFERS
TDC32
16
TDC32 versus HPTDC

• TDC32
• Resolution freq. dependent (max 60 MHz)
• At 60 MHz 520 ps
• 86 pin PLCC package
• TTL standard
• Power 5 V
• 127 bits for JTAG

• HPTDC
• Basic clock 40 MHz
• Programmable resolution
• 25, 100, 200 780 ps
• 225 pin pBGA package
• LVDS LVTTL
• Power 2.5 V and 3.3 V
• 1000 bits for JTAG
• Highly flexible programming

16
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
17
Phased Lock Loop (PLL)
PLL for frequency multiplication

• It is basically a feedback contol system that
  controls the phase of a voltage controlled
  oscillator(VCO)
• The input signal is applied to one input of a
  phase detector. The other input is connected to
  the output of a divide by N counter. Normally the
  frequencies of both signals will be nearly the
  same. The output of the phase detector is a
  voltage proportional to the phase difference
  between the two input
• This signal is applied to the loop filter. It is
  the loop filter that determines the dynamic
  characteristics of the PLL. The filtered signal
  controls the VCO. Note that the output of the VCO
  is at a frequency that is N times the input
  supplied to the frequency reference input.
• This output signal is sent back to the phase
  detector via the divide by N counter.

18
HPTDC Block Diagram
18
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
19
High Performance Time to Digital Converter
Specifications

• 225 pin pBGA package
• 1000 bits for JTAG programming
• Programming Flexibility
• Number of channels 32
• Clock frequency 40 MHz external
• 40MHz / 160 MHz / 320 MHz internal
• Time bin size 781,195, 98 24ps
• Event buffer size 4 x 256
• Read-out buffer size 256
• Trigger buffer size 16
• Power supply Core 2.3 - 2.7 volt
• TTL IO 3.0 - 3.6 volt
• Power consumption 450mW - 2000 mW depending on
  operation mode.
• LVDS or LVTTL inputs for hits, clock , trigger,
  reset, serial readout, parallel readout

20
Programming parameters

• Resolution
• Channel offsets
• Leading/trailing/pair
• Channel enable/disable
• LVDS/LVTTL hit inputs
• Channel dead time ( 10 100ns)
• Encoding of Triggers and resets
• Trigger matching or no trigger matching
• Trigger latency
• Matching window

• Limiting number of hits/event
• Reject latency
• Readout FIFO size (lt256)
• Readout of buffer occupancies per event
• Serial, byte or parallel readout
• Readout via JTAG
• Serial readout speed
• Use of headers and trailers
• Token passing scheme
• Test modes

21
Parallel port JTAG Controller
TAP Controller state Machine
Joint Test Action Group (JTAG) TCLK Test
Clock TDI Test Data In TDO Test Data
Out TMS Test Mode Select TRST Test Reset
(Optional)
21
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
22
JTAG functions for HPTDC and the c program
For HPTDC Control bits 40 5 sets of
Initialisation namely PLL DLL reset, PLL lock,
DLL lock, Global reset, Enable output data
bus Setup bits 647 once to setup parameters
23
Setup bits 31 to 44 out of 647 bits
32 bit Leading edge measurement data
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
24
Time resolution using delay cable test
HPTDC Manual At 781ps resolution rms 265ps
195ps resolution rms 86ps 98ps
resolution rms 64ps

• Trigger mode operation
• Start delayed by 5m cable
• and given as stops
• Data taken at different resolution

Resolution 98ps
Resolution 195ps
Resolution 781ps
rms 60ps
rms 51ps
rms 190ps
Frequency
Frequency
Frequency
TDC Bin
TDC Bin
TDC Bin
24
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
25
HPTDC CALIBRATION, Resolution 98 ps, Clock used
40 MHz
RMS of deviation 31.26 ps
Deviation from straight line ns
TDC counts
Y 10.2397x 120.8717
Resolution 97.7 ps / count
Delay in ns
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
26
Dynamic Range of HPTDC
HPTDC Counts
Resolution 97.7 ps/count
Time in µs
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
27
Plot on multi-hit capability of HPTDC for A
channel ( 98ps/count)
Only four hits recorded by HPTDC to avoid noisy
channels occupying the data bus bandwidth
12.25ns
10.68ns
10.78ns
Frequency
TDC bin
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
28
Code density Test-TDC Distribution for random
events at 98ps resolution
The spikes every 32 bin in TDC distribution is
due to internal crosstalk caused by logic core
running at 40MHz
Frequency
The distribution repeats every 25 ns (40MHz TDC
Clock)
At 98ps resolution the distribution repeats every
256 counts
256 X 98 ps 25.088ns
TDC Bin
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
29
Differential non linearity (DNL) results-Code
density test
Differential nonlinearity (DNL) is the variation
of any code from an ideal 1 LSB step
DNLi (Actual i / Expected i ) - 1
DNL Counts
HPTDC Bin
DNL Plot we obtained at 98ps resolution
1.25,-0.1 bin
DNL Counts
HPTDC Bin
DNL Plot at 98ps resolution from manual
0.6,-0.25 bin
30
Integral non linearity (INL) Results
Integral nonlinearity (INL) is the deviation of
the transfer function from a reference line
measured in fractions of 1 LSB using a best
straight line. INL is simply the integral of the
DNL
HPTDC bin
INL (counts)
INL Plot at 98ps resolution 0.5,-2.5 bin
INL (counts)
HPTDC bin
INL Plot at 98ps resolution from manual 0.6,
-1.4 bin
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
31
INL and correction in timing

• HPTDC MANUAL
• A fixed pattern in the INL is clearly seen in the
  high resolution and the very high resolution
• modes. This is caused by 40 MHz cross talk from
  the logic part of the chip to the time
  measurement part and is believed to come from
  power supply and substrate coupling.
• As this crosstalk comes from the 40MHz clock,
  which is also the time reference of the TDC, the
  integral non linearity have a stable shape
  between chips and can therefore be compensated
  for by a simple look up table using the LSB bits
  of the measurements as an entry

Effective HPTDC resolution based on cable delay
measurement
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
32
PROTOTYPE MODULE INSTALLED IN GRAPES-3 DATATAKING
40 MHz clock
Redundant
JTAG
HPTDC
Control part in CPLD
33
GRAPES-3 ASET series dedicated to our friend and
colleague Mr.S. Karthikeyan who passed away on
June 20, 2008 in an accident
33
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
34
GRAPES-3 ASET SERIES

• HPTDC II 24-Oct-2008 Ajai Iyer
• MCA 1-Oct-2008 P. Jagadeesan
• Monitoring tools 07-Nov-2008 S. Muragapandian
• Computer Cluster I 14-Nov-2008 R. Suresh
  Kumar
• Computer Cluster II 21-Nov-2008 B. Rajesh
• e-Administration 28-Nov-2008 V.
  Viswanathan

THANKS
34
35
THANKYOU
36
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
37
HPTDC Technical Specification

• Number of channels 32 / 8
• Clock frequency 40 MHz external
• 40MHz / 160 MHz / 320 MHz internal
• Time bin size 781 ps low resolution mode
• 195 ps medium resolution mode
• 98ps high resolution mode
• 24 ps very high resolution mode (8 channels)
• Differential non linearity Typical values
• /- 0.2 bin 0.08 bin RMS low resolution mode
• /- 0.3 bin 0.09 bin RMS medium resolution mode
• 0.60, -0.25 bin0.10 bin RMS high resolution
  mode
• 0.35, -0.25 bin0.08 bin RMS high resolution
  mode (DLL corr.)
• 1.3, -0.7 bin 0.21 bin RMS very high resolution
  mode
• Integral non linearity Typical values
• /- 0.25 bin 0.11 bin RMS low resolution mode
• /- 0.50 bin 0.24 bin RMS medium resolution mode
• 0.6,-1.4 bin 0.50 bin RMS high resolution mode
• 3.5,-5.0 bin 2.1 bin RMS very high resolution
  mode

37
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
38
HPTDC Technical Specifications

• Time resolution Typical values
• 0.34 bin RMS (265 ps) low resolution mode
• 0.44 bin RMS (86ps) medium resolution mode
• 0.65 bin RMS (64ps) high resolution mode
• 2.4 bin RMS (58ps) very high resolution mode
• 0.72 bin RMS (17 ps) very high resolution mode
  (table corr)
• Difference between channels Maximum /- 1 ns
  offset
• Variation with temperature Maximum 100ps change
  with 10 Deg. change of IC temperature
• Cross talk Maximum 150 ps from concurrent 31
  channels to one channel.
• Dynamic range 12 5 17 bit low resolution
  mode
• 12 7 19 bit medium resolution mode
• 12 8 20 bit (19) high resolution mode
• 12 8 2 22 bit (21) very high resolution
  mode.
• Numbers in parenthesis are number of bits read
  out
• Double pulse resolution Typical 5 ns. Guaranteed
  10ns

38
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
39
HPTDC Technical Specifications

• Max. recommended Hit rate Core logic at 40 MHz
• 2 MHz per channel, all 32 channels used
• 4 MHz per channel, 16 channels used.
• 2 MHz per channel in very high resolution mode
• Event buffer size 4 x 256
• Read-out buffer size 256
• Trigger buffer size 16
• Power supply Core 2.3 - 2.7 volt
• TTL IO 3.0 - 3.6 volt
• Power consumption 450mW - 2000 mW depending on
  operation mode.
• Temperature range -40 - 70 Deg. Cent.
• Hit inputs LVDS or LV TTL
• Clock input LVDS or LV TTL
• Trigger and reset inputs LVDS or LV TTL
• Serial readout LVDS or LV TTL
• Parallel readout LV TTL

39
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
40
40
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
41
TDC32 versus HPTDC

• TDC32
• Resolution freq. dependent (max 60 MHz)
• At 60 MHz 520 ps
• 86 pin PLCC package
• TTL standard
• Power 5 V
• 127 bits for JTAG

• HPTDC
• Basic clock 40 MHz
• Programmable resolution
• 24, 98, 195 781 ps
• 225 pin BGA package
• LVDS LVTTL
• Power 2.5 V and 3.3 V
• 1000 bits for JTAG
• Highly flexible programming

41
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
42

• The ispLSI 1016 signals were incorporated and we
  moved into PCB artwork.
• The HPTDC chip being a BGA package was only
  surface mount and could not be fanned out on
  single layer or two layer with power and ground
  so we went for a four layer PCB with two power
  planes and two routing layers
• Before PCB fabrication we wanted to be sure of
  our design so it was extensively checked by me
  Shri.Ajai several times independently
• The power plane had to support four different
  powers namely 3.3v, 2.5v, 5v, -5v and it was a
  challenge to place and route
• For the fan out of signals from HPTDC we had to
  use reduced track width and special vias which
  could fit in between the pads of the BGA. Ball
  dia 0.75mm and ball to ball spacing 1.5 mm
• The final PCB design (Artwork) was again checked
  independently by myself Mr.Ajai and Mr.Manjunath
  before giving for fabrication
• After fabrication the PCB was tested by us and
  again send to Bangalore with the HPTDC for BGA
  mounting

K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
43

• JTAG Programming
• Setup 647 bits and 40 control bits 5 times
• Setup read from a text file in hex.
• Reflecting a change takes time
• Mr Ajai made a excel sheet
• We also faced the problem of data hitches from
  the HPTDC which we solved by probing using the
  JTAG interface built in and using JTAG commands
  to see status of internal registers Data
  readout via JTAG
• For this purpose we had to write a special
  program to send the JTAG commands to the HPTDC
  and read the register values
• The first test we did was calibration for which
  we used the offline calibration circuit for TDCs
• The calibration circuit generates a start and
  stop after a known delay. Each delay is repeated
  for 100 counts in our case and the average value
  is used to plot the TDC distribution ( Delay
  versus counts)

K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
44
(No Transcript)
45
CABLE ATTENUATION/100 m
46
(No Transcript)
47
Why amplifier is necessary ?

• 230m of 5D2V cable is used to bring the signal to
  control room
• Attenuation of 5D2V is 8 dB/100 m at 100 MHz.
• The signal gets attenuated by 18.4 dB in 230m.
• Discriminator set to -30 mV to trigger at 1/3 rd
  of single particle.
• Without amplifier the PMT pulse height has to
  cross -250 mV and with an X10 amplifier it
  reduces to -25 mV at the PMT anode.
• Without amplifier the PMT has to be operated at
  much higher HT to trigger the discriminator where
  noise is more and reduces the life of the PMT
• Dynamic range of ADC

K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
48
PCI Interface card

• The PCs which are available now does not have
  ISA bus support
• ALS systems manufactures prototyping PCI cards
• The local address bus , data bus and control
  signals like read and write are available to user
• We could read data at 1 µs / word using this card
  in our DAS
• A common PCI Interface card designed and wired
  and used in all our DAS at GRAPES-3
• This card costs around Rs 5,500/- and the card
  from Japan PCI-7200 which is a PCI, I/O card
  costs around Rs 12,000/-

48
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
49
Comparison of power requirement for Phillips and
CRL module
50
Trigger Rate at GRAPES-3 with 400 scintillation
detectors

• Level-0 100 Hz
• Level-1 28 Hz
• Calibration 5 Hz
• Pedestal 1 Hz

51
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
52
INL and correction in timing
A fixed pattern in the INL is clearly seen in the
high resolution and the very high
resolution modes. This is caused by 40 MHz cross
talk from the logic part of the chip to the time
measurement part and is believed to come from
power supply and substrate coupling). As this
crosstalk comes from the 40MHz clock, which is
also the time reference of the TDC, the integral
non linearity have a stable shape between chips
and can therefore be compensated for by a simple
look up table using the LSB bits of the
measurements as an entry
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
53
Dynamic Range of HPTDC
HPTDC Counts
Resolution 97.657 ps/count
Time in µs
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
54
DYNAMIC RANGE TEST FOR HPTDC CHANNEL 1
TDC Counts
Conversion gain 1/10239.953 µs / count
97.657 ps
Time in µs
K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY
55
Test setup to check Differential non linearity
(DNL) and Integral non linearity (INL) using code
density test
TDC Clk 40 MHz
Start
50 Hz
Trigger latency 500ns
Matching Window 450 ns
Trigger
10 MHz Clock
STOPS

• Matching window programmed to 450ns to see 4
  stops inside the window
• HPTDC operated in enable-relative mode ( relative
  time to trigger time tag)
• TDC Clk and Stops generated with different
  crystals (uncorrelated in time)

K.C.RAVINDRAN,GRAPES-3 EXPERIMENT,OOTY