8. Statistical Analysis and Results 109
8.3. Results
8.3.3. Fit to Data
In order to understand better the fit behaviour when performing the fit to data, the fit result to data in the six fit regions under the signal-plus-background hypothesis, where signal in this case ist¯tV, is shown in comparison with the fit to the three fit regions in the 2`OSZveto region in Figure 8.18, and the fit to the three fit regions in the 2`OSZ region in Figure 8.19. By splitting
8. Statistical Analysis and Results
0.036 -0.11 0.022 -0.3 0.0018 -0.027 0.37 -0.0057 -0.44 -0.021 1
0.079 0.012 -0.082 -0.0052 -0.0007 -0.074 0.24 -0.097 0.27 1 -0.021
0.063 -0.076 -0.085 -0.16 -0.0039 -0.18 -0.17 -0.27 1 0.27 -0.44
-0.045 0.026 -0.12 -0.00038 0.0031 -0.21 0.28 1 -0.27 -0.097 -0.0057
0.0077 0.082 0.12 0.13 -0.0055 0.26 1 0.28 -0.17 0.24 0.37
0.099 0.42 0.3 0.21 0.3 1 0.26 -0.21 -0.18 -0.074 -0.027
0.0015 -0.016 -0.0087 -0.0081 1 0.3 -0.0055 0.0031 -0.0039 -0.0007 0.0018
-0.046 -0.051 0.0023 1 -0.0081 0.21 0.13 -0.00038 -0.16 -0.0052 -0.3
-0.09 0.083 1 0.0023 -0.0087 0.3 0.12 -0.12 -0.085 -0.082 0.022
0.049 1 0.083 -0.051 -0.016 0.42 0.082 0.026 -0.076 0.012 -0.11
1 0.049 -0.09 -0.046 0.0015 0.099 0.0077 -0.045 0.063 0.079 0.036
V)t(tµ BTAGBREAK5 JetModel1 LUMI Zjets_XS Zjets_ktfac_Norm -DataRw-IFSRtt _HFtt _MG_Normtt _PartonShower_Normtt _XStt
_XS t t _PartonShower_Norm t
t
_MG_Norm t t
_HF t t -DataRw-IFSR t
t
Zjets_ktfac_Norm Zjets_XS LUMI JetModel1 BTAGBREAK5 V) t (t µ
Correlation
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Figure 8.17.: Correlation matrix showing the correlations between the nuisance parameters built in the fit model, and with the signal strength,µttV¯ . Only nuisance param-eters with at least one correlation above 25% are shown.
the fit by region, a shift or constraint of a fitted nuisance parameter can be associated with a given region. The black points and error bands correspond to the combined six-regions fit, and the red ones correspond to the fit in the 2`OSZveto region in Figure 8.18, and to the fit in the 2`OSZ region in Figure 8.19.
There are no over-constraints in the fit to data that are not expected from the fit to the Asimov dataset. In general, the fit output shows very little movement of the nuisance parameters’ central value with respect to 0.
When performing the fit in the 2`OSZveto and 2`OSZ regions separately, it can be seen that the constraints on the systematic uncertainties related to tt¯originate from the data in the 2`OSZveto region, while the constraints on the systematic uncertainties related to Z+jets originate from the data in the 2`OSZ region. Similarly, it can be seen that the leading pulls originate from the analysis regions where the t¯tand Z+jets are dominant backgrounds.
The large data statistics available in the 2`OSZveto (4j,1b + 2b) and (≥5j,1b + 2b) fit regions allow to constrain the t¯t Madgraph+Pythia uncertainty to 40% of its initial value. The reduced post-fit uncertainties, as well as the modified background and signal yields according to the fitted nuisance parameters’ central values and signal strength µt¯tV, can be seen in the post-fit distributions in Figure 8.20. After the fit, the agreement between data and prediction improves significantly, and the differences are covered by the post-fit uncertainty band.
134
8.3. Results
Figure 8.18.: Pull distributions of the fitted nuisance parameters in theS+B hypothesis to the data for the combined six-regions fit (black) and for the fit in the 2`OSZveto region only (red). The fitted nuisance parameters are shown with their pulls and constraints compared to the nominal central value of 0 and standard deviation of 1. If a nuisance parameter appears with no red line, it means that the nuisance parameter did not pass the pruning selection in the 2`OSZveto region, and is therefore not included in the corresponding fit.
8. Statistical Analysis and Results
Figure 8.19.: Pull distributions of the fitted nuisance parameters in theS+B hypothesis to the data for the combined six-regions fit (black) and for the fit in the 2`OSZ region only (red). The fitted nuisance parameters are shown with their pulls and constraints compared to the nominal central value of 0 and standard deviation of 1. If a nuisance parameter appears with no red line, it means that the nui-sance parameter did not pass the pruning selection in the 2`OSZ region, and is therefore not included in the corresponding fit.
136
8.3. Results
100 200 300 400 500 600 700
Events / bin
100 200 300 400 500 600 700
Data / Pred 0.5
100 200 300 400 500 600 700
Data / Pred 0.5
Figure 8.20.: Post-fit distributions for each of the six fit regions in the OS dilepton channel.
The left (right) column shows the distributions in the 2`OSZveto (2`OSZ) re-gions. The red dashed lines show thettV¯ signal normalised to the background yield. The top left plot shows the (3j,1b + 2b) region, where only the event
8. Statistical Analysis and Results
The post-fit event yields for the combinedee+jets,µµ+jets andeµ+jets samples in the six fit regions are summarised in Table 8.5 for the 2`OSZveto region, and in Table 8.6 for the 2`OSZ region.
3 j, 1 b + 2 b 4 j, 1 b + 2 b ≥5 j, 1 b + 2 b
t¯tV 52 ± 3 46 ± 2 36 ± 2
t¯t 21000 ± 200 8300 ± 100 3660 ± 90
Z+jets 510 ± 60 190 ± 10 80 ± 20
t¯tH 11 ± 2 13 ± 2 17 ± 2
Single top 800 ± 100 260 ± 40 100 ± 20
Diboson 60 ± 10 26 ± 7 12 ± 3
tZ 4 ± 2 3 ± 1 1.9 ± 1.0
MisID lepton 160 ± 80 90 ± 40 50 ± 20
Total Bkgd 22600 ± 200 8900 ± 100 3910 ± 80
Data 22585 8909 3901
Table 8.5.: Post-fit event yields under the signal-plus-background hypothesis for signal, back-grounds and data in each of the three fit regions in the 2`OSZveto region. The quoted uncertainties are the sum in quadrature of post-fit statistical and total sys-tematic uncertainties on the yields, taking into account correlations among nuisance parameters and among processes.
3 j, 2 b 4 j, 2 b ≥5 j, 2 b t¯tV 4.0 ± 0.3 6.8 ± 0.4 14.5 ± 0.9
t¯t 830 ± 40 340 ± 20 163 ± 8
Z+jets 1900 ± 70 670 ± 30 290 ± 20
t¯tH 0.5 ± 0.07 0.64 ± 0.08 0.9 ± 0.1
Single top 20 ± 3 7 ± 1 3 ± 1
Diboson 25 ± 7 14 ± 4 9 ± 3
tZ 4 ± 2 4 ± 2 3 ± 3
MisID lepton 4 ± 2 2 ± 1 1.4 ± 0.7
Total Bkgd 2790 ± 50 1040 ± 30 470 ± 20
Data 2806 1031 471
Table 8.6.: Post-fit event yields under the signal-plus-background hypothesis for signal, back-grounds and data in each of the three fit regions in the 2`OSZ region. The quoted uncertainties are the sum in quadrature of post-fit statistical and total system-atic uncertainties on the yields, taking into account correlations among nuisance parameters and among processes.
The agreement between data and prediction is also checked in the kinematic variables, which are not used in the fit. Figure 8.21 shows the post-fit distributions for the two highest-ranked input variables used for the NN discriminant in each signal region: the 2`OSZveto (4j,1b + 2b) and (≥5j,1b + 2b) regions, and the 2`OSZ (≥5j,2b) region. The small discrepancies remaining between data and prediction are covered by the post-fit uncertainty band.
138
8.3. Results
Figure 8.21.: The post-fit distributions of the highest two ranked variables in (top): the 2`OSZveto (4j,1b+2b) region, (middle): the 2`OSZveto (≥5j,1b+2b) region, and (bottom): the 2`OSZ (≥5j,2b) region. The red dashed line shows t¯tV signal normalised to the background yield. The error band includes both statistical and systematic post-fit uncertainties. The arrow in the bottom panel indicates that the central value of the ratio between data and total
8. Statistical Analysis and Results
In order to identify the systematic uncertainties with the largest post-fit impact on the fit-ted signal strength, µt¯tV, a ranking of the nuisance parameters is performed, and shown in Figure 8.22. The procedure to obtain such a ranking consists of repeating the fit fixing one nuisance parameter at a time to its post-fit +1σ, and then to the post-fit−1σ uncertainty. The post-fit impact of the nuisance parameter, shown with the blue dashed band, corresponds to the difference of the fittedµvalues. If the same is performed but fixing the nuisance parameters to the pre-fit±1σ values, the pre-fit yellow band is obtained. The upperx−axisshows the abso-lute difference in the fittedµ. The lower x−axis and the black lines show the aforementioned pull distribution of the nuisance parameters in the fit result.
The nuisance parameters corresponding to the leading modelling systematics on the main backgrounds are ranked in the first positions, as expected. The high ranking of the nuisance parameter related to the alternative DS approach for single top background can be understood given the “signal-like” shape variation in the NN distribution in the 2`OSZveto (4j,1b + 2b) and (≥ 5j,1b + 2b) regions, as shown in Figure 8.11. Moreover, it has a significant normalisation variation per fit region, which is not constrained by the fit.
The fitted signal strengths and its total uncertainty with respect to the NLO QCD calcu-lation described in Sections 2.4 and 5.2, as well as the corresponding expected and observed compatibility with the background-only hypothesis in each case, are summarised in Table 8.7 for options 1-3. From the expected significances, it can be seen that the OS dilepton analysis is more sensitive to the t¯tZ signal process, than to the t¯tW process. This can be understood by the presence of the 2`OSZ region in the fit, which has the most discriminating NN in the signal region (≥5j,2b).
Process Signal Strength Observed σ Expectedσ
t¯tV 0.77+0.63−0.56 1.4 1.7
t¯tZ 0.77+0.69−0.59 1.4 1.5
ttW¯ 0.57+2.48−2.30 0.3 0.4
Table 8.7.: The observed signal strength and its total uncertainty relative to the correspond-ing NLO QCD prediction, and the observed and expected significance of the t¯tV, t¯tW and t¯tZ signals, expressed in terms of the standard deviations from the background-only hypothesis.
Table 8.7 shows that the total fitted uncertainties are slightly asymmetric. This follows from the negativelog of the profile likelihood ratio curve as a function of µ,−logλ(µ). Figure 8.23 shows in red the −logλ(µttV¯ ) curve, and the black dashed lines indicate the µt¯tV values for which the likelihood curve has a value of 0.5, corresponding to the ±1σ fitted uncertainty on µttV¯ . The “real” asymmetric curve is compared to the symmetric blue dashed profiled likelihood curve, when imposing a symmetric uncertainty onµ. The divergence between the two curves is more pronounced the more theµvalue is shifted from the minimum of the curve, for instance at µ= 0. The “asymmetric” nature of the profile likelihood curve comes mainly from the statistical component of the total uncertainty, as can be seen in Table 8.8.
Table 8.8 provides a breakdown of the uncertainties on the measured signal strengths of the t¯tZ andt¯tW processes. It provides equivalent information to the post-fit impact bands from the systematic uncertainties on the fitted signal strength in Figure 8.22, grouped into categories.
Furthermore, the statistical uncertainty (data and template statistics) is shown in the table. The
140
8.3. Results
-4 -2 0 2 4 6
JetFlavComp JEFF _HF t t MisIDlepton_Norm -DataRw-Fragmentation t
t
BTag1 _MG_Norm t
t
JetMu JetModel1 JVF _PartonShower_Norm t
t
Dibosons_HF Zjets_NoZpT _MG_Shape t
t
Zjets_ktfac_Norm SingleTop_DS Zjets_sherpa_Norm
µ
∆ -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
θ )/∆ θ0
θ - (
-1.5 -1 -0.5 0 0.5 1 1.5 2
Pull
Pre-fit Impact on µ µ Post-fit Impact on
Figure 8.22.: Ranking of the nuisance parameters based on the fit to data according to their effect on the fitted signal strength. The top 18 parameters are shown. The yellow band corresponds to its pre-fit impact, and the dashed blue band, to its post-fit impact. They are plotted relative to the top horizontal axis in terms of the difference between the fitted signal strength values, ∆µ. The ranking from top to bottom is based on the post-fit impact onµ. The bottom axis shows the pull of the nuisance parameters from the fit result.
main contribution to the total uncertainty for thet¯tZ measurement arises from the limited data statistics, and from systematic variations for thet¯tW measurement. The dominating systematic uncertainty in both cases comes from the modelling of the backgrounds.
The results of the simultaneous fit of the t¯tZ and t¯tW processes are presented together with the combined results with the trilepton and same-sign dilepton channels in Section 8.4.
8. Statistical Analysis and Results
V t t
0 0.5 1 1.5 2 2.5 µ
) Vttµ(λ-log
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Figure 8.23.: Profiled likelihood curve−logλ(µttV¯ ) =−(logL(µt¯tV)−logL(ˆµt¯tV)) as a func-tion of the ttV¯ signal strength, µt¯tV. The red solid curve corresponds to the
“real” asymmetric likelihood, and the blue dashed curve corresponds to the symmetric likelihood, when imposing a symmetric uncertainty on µttV¯ . The minimum value of the curve corresponds to the “best-fit” signal strength, ˆµt¯tV. The µt¯tV values corresponding to the intersection between −logλ(µttV¯ ) = 0.5 and the profiled curve, indicated with the horizontal and vertical black dashed lines, correspond to the error on the signal strength, ˆµt¯tV ±1σ.
Uncertainty µt¯tZ µt¯tW
Detector 0.11 0.36
Background from simulation 0.20 1.28
MisID lepton 0.02 0.32
Signal modelling <0.01 0.10
Total systematics 0.33 2.06
Statistics +0.59−0.51 +1.20−1.18
Total +0.69−0.59 +2.48−2.30
Table 8.8.: Breakdown of uncertainties on the measured signal strength of t¯tZ and t¯tW pro-cesses from the 1-parameter-of-interest fit. Due to (anti)correlations between the different sources of uncertainties, the total systematic uncertainty can be different from the sum in quadrature of the individual sources. Systematic uncertainties are given symmetrised.
142