Back-end readout and Data Acquisition (DAQ)

The data handling hybrid (DHH) system forms the backend of the PXD readout and is an essential part of the PXD data acquisition. Apart from receiving the data stream from the front-end electronics it also generates the control signals required to configure and trigger the ASICs on the PXD half-ladders. This section gives an overview of its most important components and their functions. A detailed description of the full DHH system can be found elsewhere [62]. A simplified version of the DHH system is also used in the laboratory for the readout of a single half-ladder.

4.6.1. DHH Components

The DHH System has three main components. Following the readout stream the next part in the chain after the DHP is the DHE (Data Handling Engine), a readout board based on a Xilinx Virtex-6 LX130T field-programmable gate array (FPGA). All four DHPs of one half-ladder are connected to one DHE with highspeed data links using the Aurora Protocol with 8B/10B encoding. The DHE sends the trigger signal to the DHPs, receives their data and merges it into one DHE data frame.

The DHI (Data Handling Interface) works in parallel to the DHE and is responsible for the configuration of the ASICs on the half-ladder using the JTAG protocol. Prototype versions of the DHE used in the laboratory combined the functionality of the DHI and the DHE on one FPGA board which was also referred to as DHE. Figure4.16 shows a single DHE/DHC unit. The units are built in the advanced mezzanine card (AMC) form factor. These units are mounted on carrier board to be operated in a stand-alone manner (laboratory) or together with other units in an ATCA (Advanced Telecommunications Computing Architecture) shelf.

4.6. Back-end readout and Data Acquisition (DAQ)

Figure 4.16.: Photograph of a DHH module (either DHE or DHC). The unit shown here has additional DDR3 RAM attached (blue card) to buffer incoming data and function as a DHC. From [62].

The data from 5 DHEs is collected in one DHC (Data Handling Concentrator). The DHC merges the 5 DHE data frames into a single DHC frame before it is send out further. The inner PXD sensors (and thereby the inner DHEs) are expected to have a higher data rate since they are closer to the interaction point where they are effected more strongly by the background than the outer ones. If the inner and outer sensor are distributed to the DHCs in the right way, the data rate of the DHCs can be averaged, making the subsequent data processing and load balancing easier.

4.6.2. Full DHH System and Global DAQ Integration

The DHH system functions as the connection of the PXD system to the rest of Belle II.

Apart from transmitting the received data to the global Belle II DAQ system, it also ensures that the PXD is synchronised with the SuperKEKB accelerator by propagating the signals from the Belle II trigger and time distribution system (B2TT) [63]. From the B2TT clock (127.21 Mhz) the DHH system generates the 76.33 Mhz clock that is further propagated to the ASICS on the PXD half-ladders.

The full DHH system consists of 40 DHE/DHI pairs and 8 DHCs to process the data of all 40 PXD half-ladders. Even with the zero-suppression of the DHPs the data rate of the PXD is still too high for the global Belle II Data acquisition system (Belle II DAQ).

Assuming a trigger rate of 30 kHz and an occupancy of 3 % (which is the maximum possible occupancy for the DHP), the data rate of the full system is ≈ 22GB/s. To reduce the data rate, the DHCs send their events to the ONSEN system [64], where they are buffered.

The Belle II DAQ system uses data from all subsystems but the PXD to construct an initial event when the (low level) hardware trigger is activated. The events are passed to the Belle II high level trigger (HLT) system [65] (high performance parallel computer farm), where a first tracking procedure is started. The reconstructed tracks

are extrapolated to the PXD sensors and a region of interest (ROI) around the predicted hit position is determined. These ROIs are passed together with the corresponding event numbers to the ONSEN system, which filters the PXD data and returns all registered PXD hits within the ROI of the given event. The data rate reduction factor because of the high level trigger and the ROIs is around 30 [64]. Together with the added PXD information a second event building process (including a new track reconstruction with the PXD data) is started before the data is stored to disk. Figure 4.17 shows a sketch which summarises this workflow.

CDC + SVD + ... PXD DHH

Event Builder 1 HLT ONSEN

Event Builder 2

ROIs

Figure 4.17.: Sketch of the dataflow of the Belle II DAQ system. PXD data is not used in the first reconstruction of an event but buffered and reduced in the ONSEN system. In a second event building step the PXD data is added to the global data stream.

There is a second outgoing data stream from the DHH system which sends a copy of the raw PXD data out to a connected computer using UDP (UserDatagramProtocol). This data stream is used for the local PXD DAQ, which is used for data quality monitoring but also serves as the main data source for calibration measurements of the PXD. It is also the DAQ system that is used in the laboratories.

4.7. Power Supply and Sevices