As already mentioned in the previous sections the IBL requires a new data readout chain to be capable to transmit the increased data rate from the IBL modules towards the ROS. The current Pixel Detector readout chain is able to transmit the data of six FE-I3 Modules at a bandwidth of 160 Mb/s each. For the IBL the data of 16 FE-I4 Modules with a bandwidth of 160 Mb/s
1Most Significant Bit
2Last Significant Bit
6.5 The IBL readout chain
Figure 6.4: Schematic view of the Pixel Register. The Pixel Shift Register writes into one of 13 Pixel Register latches of a complete double column and in total into 672 latches. Thereby, the latch of the last pixel of the second column is written first and the latch of the last pixel of the first column is written last. The DCs to be written are selected by the Global Register configuration and the specified address within the “Write Front End” command [86].
each has to be handled and would exceed the capabilities of the current readout chain. Thus, a new readout chain is needed to fulfil the requirements of the IBL.
The Fast Track IBL schedule demands to have a fully integrated chain for the new detector layer before the Phase 0 shutdown in 2013. To accomplish the demands for IBL within the short time scale a readout chain similar to the one of the present detector was planned. With this decision basic board layouts, the firmware and also the gained knowledge of the previous detector operation can be used as a sophisticated starting point to develop a readout chain for the IBL. Furthermore, the necessary high level of backward-compatibility to the current Pixel Detector readout can be provided. This is done by a readout chain on the basis of the current ROD-BOC pair using VME interface communication.
It consists of a ROD-BOC pair inside a VME crate and an Optoboard followed by a hybrid module as sketched in Figure 6.5. Control and configuration of the IBL ROD and IBL BOC as well as the IBL Module are still done via VME interface and the corresponding SBC-host connection. The same TIM used for the current readout chain is also used for IBL. Hence, the VME pin-out to the TIM has to be exactly the same as before and was considered during the IBL ROD-BOC development. More details on how the individual parts are upgraded for the IBL are explained in the following. As the IBL ROD is the major topic of this thesis, more details on the upgrade can be read in Section 6.6.
The main tasks of the IBL BOC is the data preparation such that it can be handled by the IBL ROD. It receives the clock from the TIM and adjusts it for a proper clock delay and phase
6 The Insertable B-Layer Upgrade
Figure 6.5: The IBL readout chain. The main components of the IBL readout chain are indicated. The analysis of the pixel hit data is done off-ROD by a new introduced Fit Server and the DSP tasks are carried over to a PPC. Optical transmission between the IBL BOC and the IBL Optoboard is done via SNAP-12 transceiver and transmitter.
to be synchronous to the global LHC bunch crossing clock of 40 MHz. This clock is further distributed to the IBL ROD and the on-detector readout parts. One major change of the IBL BOC is the 8b10b encoded module data which is decoded on the IBL BOC. While decoding the data the IBL BOC simultaneously checks the 8b10b stream for errors like disparity errors. This kind of error is specific for 8b10b coding and occurs if the predetermined sequence of 1’s and 0’s is not kept. During data taking such an error would cause the IBL BOC to discard the current event as the exact location of the provoking SEU cannot be determined.
The IBL BOC has to be upgraded to be able to satisfy the higher data throughput. Two equivalent data paths are implemented which each manage the data of 16 FE-I4 chips. To transmit commands within one data path one SNAP-123 transmitter (SNAP-12 Tx) is used.
The commands are transmitted to two FE-I4 chips at a bandwidth of 40 Mb/s. To receive data two SNAP-12 Receiver (SNAP-12 Rx) per data path have been introduced onto the IBL BOC, see Figure 6.6. The two receivers handle the data of two FE-I4 chips at a bandwidth of 160 Mb/s each. The optical power of SNAP-12 was not sure to be enough in the first stages of the board development. Hence, a possibility to place additional Tx plugins from the current BOC design on the IBL BOC was given. Meanwhile, it was shown that the SNAP-12 plugins fulfil the requirements as stated in [87]. One SNAP-12 Rx receives twelve optical links: eight data channels, two address and two control channels. As the optical receiver automatically senses the average light level the best working point can be automatically determined to receive the incoming data without errors. This is an improvement as on the Pixel Detector BOC the best working point had to be determined regularly and an extra scan, the “BOC scan” had to be implemented. The data of one data path is handled by a Spartan6 “BOC Main FPGA” (BMF) where it is 8b10b decoded, multiplexed and further sent to the IBL ROD [88].
To be able to transmit the increased amount of data towards the ROS and to send a copy of the data to the future Fast Track Trigger four S-Links are used. The space consuming HOLA card used for the current BOC was removed and the protocol logic needed for the S-Links
3A SNAP-12 is a commercial Snap-On Plugin for twelve optical channels [90].
6.5 The IBL readout chain
Figure 6.6: The second IBL BOC prototype with its main components and communica-tions [91].
moved inside the FPGAs. As no final decision was made the first IBL BOC prototypes contain two different implementations of S-Links for evaluation. Four individual SFPs4 and one smaller Quad-SFP (QSFP), which has the same functionality as the single SFPs but is less space consuming, were chosen to transmit the data to the ROS [89].
To control the data handling on the IBL BOC a Spartan6 “Board Control FPGA” (BCF) is used. It programs the individual FPGAs with the firmware stored inside the flash memories and manages the ROD-communication as well as the clock distribution. Moreover, all three FPGAs have their own 512 Mb DDRII memory. For a possible future upgrade the two BMF FPGAs each have a connection to an additional Low Pin Count FPGA mezzanine card (LPC FMC) [88]. For the DCS communication an Embedded Local Monitoring Board (ELMB) is foreseen such that PiN currents and voltages can directly be read out.
The IBL Optoboard serves as optical-electrical interface between the on-detector and off-detector components. It receives optical bit streams from the IBL BOC and transmits them into an electrical signal. Vice versa it converts the electrical signals from the FEs into an optical signal. The new board has eight command links and 16 data links and thus is connected to 16 FE-I4 chips. Due to failures present in the current Optoboard some urgent improvements
4Small Form-Factor Pluggable Transceiver
6 The Insertable B-Layer Upgrade
have to be implemented for IBL. One problem is the solder connection between the so called
“opto-pack” which contains the VCSEL and the PiN diode and the PCB5and leads to problems in the transmission of signals [74]. This problem is solved for IBL by using a wire bond instead of a solder connection. Furthermore, the currently used custom made connector for the optical fibres is fragile and difficult to mount and dismount and hence replaced by a commercial one.
During the past Pixel Detector operation the appearance of the Optoboard failures increased and led to configuration or readback problems in 1% of the modules [74]. So far, the source of the problem could not be figured out as the Optoboards are inside the detector and not accessible.
The IBL Optoboards will be placed at the end caps outside the detector such that they can be accessed.
One IBL ROD-BOC pair processes the data of two half staves where each half stave has 16 FE-I4 chips. And one IBL Optoboard transmits streams of half a stave. Thus, in total there are 14 staves and 14 ROD-BOC pairs with 28 IBL Optoboards and 448 FE-I4 chips.
To be prepared for the IBL installation several testing phases for the IBL detector chain are foreseen in 2012 and 2013. These system tests are the first trial to check, validate and improve the interconnection between all readout components. This comprises the FE-I4 chips loaded onto a stave and connected to a readout system. Furthermore, cooling and the DCS system is implemented. Currently a readout board for test issues is used but it is planned to integrate the IBL ROD-BOC chain as soon as the major development steps are completed.
5Printed Circuit Board