Before going into implementation details in chapter 5, this chapter describes the approach for the implementation of a radiation mitigated CBM-ToF read-out controller and further-more the basic setup for in-beam testing of the mitigation efficiency is presented. As the previous chapter, this one is also organized in the three sections that represent the same three topics.
First, section 4.1 presents the GET4 read-out controller that follows the modular design approach of the CBM read-out controller family. The general advantages of this modular design approach are discussed. Then, in section 4.2, the choice of radiation mitigation techniques and the necessary adaptations for the techniques to fit for the CBM use case are discussed. Section 4.3 finally describes the basic ideas to verify the efficiency of the implemented techniques at in-beam tests.
4.1. Modular Firmware Concept for the CBM Read-Out
Figure 4.1.: The CBM experiment at FAIR/GSI consists of several detector systems as illustrated here in its muon configuration. Picture as published in [MAGK10].
The different front-end chips require different interfacing logic in the read-out controller (ROC). With theSysCore Boardsbeing very flexible regarding these kind of interfaces, the goal is, of course, to support as many of the front-end electronics interfaces as possible.
The first chip that was read out by aSysCore Boardwas the nXYTER in 2007 [Abe07]. This read-out firmware, developed by Norbert Abel, quickly became the standard solution for nXYTER read-out and is still widely used in CBM community. With the work of this thesis, the read-out of the first prototype of the GET4 followed in 2009. As of 2012, further iterations of the GET4 with slightly different interfaces are supported. Later on, theSysCore Board Version 3was used to interface the SPADIC [Gar11] and the STS-XYTER [Sch14].
Plans foresee to read out some of the front-end electronics with ASICs instead of FPGA-based ROCs, e.g. in case of the STS. The reason is the extreme radiation level in those regions. Nevertheless, even if the read-out of some of the detector front-ends is planned to be accomplished by an ASIC in the final detector setup, the prototyping of those read-out ASICs is still done on an FPGA.
In the end, several different firmwares for interfacing the different front-end electronics are required. However, the different firmwares have very common requirements regard-ing the interface towards the DAQ. The transport logic towards the DAQ can be the same, and with a modular design and a clear interface, the same logic can easily be reused.
Different Requirements for Data Transport TheSysCore Boardsare designed to support the research and development of the CBM read-out chain and also to providing a work-ing data read-out solution for detector development. The challenge here is to develop
towards an implementation that meets the full list of future requirements, while at the same time permanently providing a working read-out solution for the smaller laboratory setups.
Final experiment Laboratory
many ROCs involved ↔ few ROCs involved
synchronization of all the ROCs is very complicated
↔ rather simple synchronization method that does not scale to more than a few ROCs is also acceptable huge data bandwidth required ↔ significantly lower data bandwidth is
sufficient data is processed on a high
perfor-mance cluster
↔ data is processed on a commodity PC significant amount of radiation is
af-fecting the electronics
↔ no radiation issue different detector ground levels
re-quire electrically isolated readout
↔ only one ground level needs to be available at the end, when
the experiment is constructed
↔ needs to be available as early as possi-ble, during R’n’D
Table 4.1.: The different requirements on the transport logic during the research and development phase and for the final detector setup. Detector picture from [FHK+11]
The final experiment setup and the smaller laboratory setups are two different sce-narios and both have different requirements. The requirements for the final experiment setup are tighter than the requirements for laboratory setups. In the final experiment many read-out boards will be operated in parallel, which requires a sophisticated time distribution and synchronization scheme. Also a much higher data throughput has to be handled than in small laboratory setups. The radiation level in the final experiment will be high enough to cause serious trouble for the electronics. Implementing all these features is a complex task that requires quite an amount of development time.
However, a working data read-out solution for laboratory setups needs to be available as early as possible to allow detector developers to analyze their detector data. The fi-nal experiment setup, on the other hand, is required much later, at the time when the experiment is constructed. The two scenarios constitute different requirements on the
transport logic. The final experiment setup requires a complex logic that supports all specified features, and in the early days a subset of all features is sufficient.
Table 4.1 lists the different requirements on the transport logic in early R’n’D and for the final detector setup.
The bottom line is that a lot of different firmwares are needed by different detector groups, but the firmwares only differ in certain aspects and much of the logic can be shared among those firmwares. A modular design of the firmware is the obvious ap-proach.
In 2009, the first prototypes of the GET4 chips were available. At that time, the decision was made to split the existing monolithic nXYTER read-out firmware in two modules, one to interface the front-end electronics and one to handle data transport. The existing logic for data transport via Ethernet could be used to quickly develop a firmware for GET4 read-out. The read-out logic for the GET4 was urgently needed to test the first GET4 prototype and push its integration in the CBM read-out chain.
At the same time, the development of the CBMNet transport protocol reached a state mature enough to start the process of integration in the CBM read-out chain. While the GET4 was put into operation, the Optics module for integration of the CBMNet transport protocol could be developed in parallel. When the Optics module reached a mature state, it could be used as transport logic for both, GET4 and nXYTER.
4.1.2. Interface Requirements
Once the decision for a modular approach is made, a specification of the interface be-tween the modules is needed. Figure 4.2 illustrates the separation of the firmware into transport and front-end modules and the interface between them. Naturally, such an in-terface should meet the requirements of the CBM experiment. The CBM requirements are:
• high data throughput
• reliable control
• precise time synchronization
• relaxed latency requirements
The interface clock is provided from the transport module to the front-end module.
This allows to use the recovered clock from the optical connection, which is the global CBM clock that is distributed via CBMNet to all front-end electronics, as system clock.
Together with CBMNet’s deterministic latency interface this enables a precise time syn-chronization over multiple ROCs.
If the Ethernet logic is used as transport module, the local board clock is used and synchronization over multiple ROCs is not possible. This is not problematic as Ethernet
Deterministic Latency Interface for Sync
Transport Module Front-End
Module PC
ROC
FEE
Clock
Fifo Interface
for Data Bus Interface for Controls
Figure 4.2.: Basic illustration of the CBM read-out controller firmware design based on two modules. The interface between the modules reflects the requirements of the CBM experiment.
is not intended to be used in complex setups with more than a few ROCs, and they can be synchronized as described in section 3.1.3 (“Poor Man’s Sync”, page 42).
A standard FIFO interface is chosen for the data path as it is easy to use, well known, and it does provides a high data throughput. The latency induced by a FIFO buffer is not relevant for CBM because the experiment is designed to operate without low-latency triggers.
The controls interface is designed as a bus structure, derived from the OPB Bus [IBM01]
that was already in use in the early, Ethernet-based read-out logic for the nXYTER chip.
Every read and every write of a register in the design is acknowledged, providing a robust controls channel.
The functionality of the different modules is briefly discussed in section 5.1.1, details of the modules and the interface are given in appendix B.
4.1.3. Towards Radiation Tolerance
Radiation tolerance is not required for all use cases. Only the optical transport module and for the GET4 read-out module are planned to finally be operated on SRAM technol-ogy and in a radiation environment. Here, radiation mitigation techniques need to be implemented. Everything that is not part of the GET4 read-out via optical fibers is either for intermediate solutions, prototyping for ASIC solutions, or will be operated outside the radiation environment in the final setup, and thus does not need to address radiation effects.