6. Results 85
6.1.1. Fit to Powheg + Herwig Pseudodata
In this setup, the parton-shower (PS) uncertainty and its correlations to other uncer-tainties are studied in detail. To better understand the output of the fit, the difference between the value that was estimated by the fit ( ˆΘ) and the input value (Θ), normal-ized to the uncertainty (∆Θ) of the input value, is compared as shown for example in Figure 6.1 for three sets of uncertainties that were used in three different fits. The uncertainty of the estimated value is given by the black line around each data point.
In the scenario when only the parton-shower uncertainty is included, by definition, the difference between the nominal prediction and the pseudodataset corresponds exactly to the PS uncertainty. Therefore, as shown in the upper plot in Figure 6.1 (a), the pulls on the PS uncertainties are close to 1σ (green area) away from the nominal prediction which is centred around 0. Furthermore, the uncertainties areconstrained to less than 1 σ (black error bars) since the fit is able to determine the uncertainty more precisely than the uncertainty that is provided by the comparison of the nominal and the alterna-tive samples. From the difference in the constraints it is concluded that the fit is most sensitive to the tt+¯ ≥2b component of the PS uncertainty.
In the lower part of Figure 6.1 (a), the PS uncertainties have been decorrelated by splitting each uncertainty in migration (Mig), corresponding to the difference in yields between different regions, and shape effects, corresponding to the slope of each uncer-tainty for a given variable, in each region. This allows to alter the normalisation while keeping the shape constant and vice versa for each component and hence adds more de-grees of freedom to the fit. However, too many dede-grees of freedom can lead to overfitting which means that smaller parameters get adjusted to describe statistical fluctuations which can lead to a bias of the parameter of interest (POI) in the final fit to data. By studying this behaviour, it is found that the decorrelation in migration and shape is only
6.1. Results from Fits to Pseudodata in the 1L+OS Channel required for the PS uncertainty. In general, the pulls are compatible to the upper case where shape and migration are correlated. The differences in the constraints indicate that the fit is more sensitive to the normalisation than to shape effects. Fort¯t+≥1c, the difference between migration and shape indicates that mostly the migration is relevant for this component.
In the next setup, shown in Figure 6.1 (b), all other modelling systematics have been included in the fit. It is observed that the constraints slightly decrease while the pulls stay similar but not identical. This is explained by the correlations among the system-atics which are given in the correlation matrix in Figure 6.2. For example, as indicated by the red-dashed line, by increasing the t¯t+≥ 1c cross section a similar effect, corre-sponding to a correlation of 45%, can be achieved as by altering thet¯t+≥2bmigration.
What is not shown in Figure 6.1 are the absolute values of the pulls which have to be taken from the corresponding configuration. For example, the pull on thet¯t+≥1ccross section corresponds to 0.13. However, the initial value of the cross-section uncertainty is approximately 40% (50% before applying additional SFs). Therefore, the absolute pull is 0.13×0.4=5% of the nominal prediction. However, the absolute values are typically not as relevant as the relative pulls and constraints to judge the stability and outcome of the fit.
In conclusion, the PS systematic uncertainties cover the difference to the pseudodataset as expected. However, due to correlations they are not observed as 1σ pull when system-atic uncertainties are decorrelated or more uncertainties are added to the fit. Therefore, correlations among systematics are monitored in detail while studying the setup as these might hide or introduce additional pulls and constraints that alter the fit outcome.
An important information that is gained from the correlations is that the fit is, if at all, only slightly biased towards larger values of the signal strength. Since exactly the SM prediction is used in the fit, ideally a signal strength of µt¯ttt¯= 1 is obtained. How-ever, due to statistical uncertainties, certain values around unity are acceptable. The measured signal strengths for the three scenarios in Figure 6.1 are shown in Table 6.1.
While all values are reasonably close to unity within uncertainties, the absolute value of µt¯ttt¯increases by 0.3 when more modelling uncertainties are included. This indicates that, due to the correlations to the systematic uncertainties, the signal strength is in-creased to compensate effects that occur only by adding more uncertainties to the fit.
To further study the bias, 500 toy experiments (TE) are performed in the setup with all modelling uncertainties. For each TE, thePowheg+Herwigpseudodata is varied within its uncertainties. The obtained distributions for the signal strength (left) and its uncertainty (right) are shown in Figure 6.3. They are parametrized by a Gaussian to extract the mean value and the width. The mean of µt¯tt¯t = 1.08 is 1σ away from an unbiased result when only the statistical uncertainties of the pseudodataset are consid-ered. However, considering the full uncertainties of about ∆µt¯tt¯t = 0.77σ, which is in good approximation constant for all TEs, the bias is considered small in this setup.
From these studies the following conclusions are drawn for the fit to data. It is impor-tant to measure the background distributions and related uncertainties independent of
−2 −1 0 1 2
B PS choice t t
b PS choice t
+light PS choice t
TTB_ttB_PhHerwig Mig TTB_ttB_PhHerwig Shape TTB_ttb_PhHerwig Mig TTB_ttb_PhHerwig Shape TTB_ttgeq2b_PhHerwig Mig TTB_ttgeq2b_PhHerwig Shape TTC_PhHerwig Mig TTC_PhHerwig Shape TTL_PhHerwig Mig TTL_PhHerwig Shape
(a) PS uncertainty
−2 −1 0 1 2
B cross section t t
b cross section t t
2b cross section
≥
1b hdamp scale
≥
F scale µ
≥ 1b + t t
R scale µ
≥ 1b + t t
TTB_ttB_PhHerwig Mig TTB_ttB_PhHerwig Shape TTB_ttb_PhHerwig Mig TTB_ttb_PhHerwig Shape TTB_ttgeq2b_PhHerwig Mig TTB_ttgeq2b_PhHerwig Shape
αS
TTC_PhHerwig Mig TTC_PhHerwig Shape
1c hdamp scale
≥ + t t
1c cross section
≥
F scale µ
≥ 1c + t t
R scale µ
TTL_PhHerwig Mig TTL_PhHerwig Shape
+light hdamp scale t
t +light ME choice t t
F scale µ +light t t
R scale µ
(b) Modelling uncertainties
Figure 6.1.: The estimated (ˆθ) value for the fit to thePowheg+Herwigpseudodataset compared to the input value (θ) for each nuisance parameter. The green (yellow) area corresponds to a 1 (2) σdeviation fromθ. Figure (a) compares the pulls and constraints when only the parton-shower (PS) uncertainties are used in the fit while in the fit for figure (b) other modelling uncertainties have been included as well.
the signal strength to reduce correlations and hence a possible bias. This is achieved by decorrelating the shape and migration of the PS uncertainty and by splitting the t¯t +≥ 1b jet component into sub-categories. By choosing fit regions with only few signal events to measure background contributions and their uncertainties, the signal can be extracted in parallel in regions with a better separation. Furthermore, it is motivated to useHTall as fit variable in the lower (b-)jet multiplicities: the PS uncertainties appear rather flat in the BDT-score while they show a trend in HTall as representatively shown in Figure 6.4 where the difference in event yields (y-axis) for the BDT-score andHTall are compared. The range of the variables (x-axis) has been normalized in both cases to allow for a better comparison. By treating the PS uncertainties correlated among all regions, assuming that there is no additional underlying effect related to the jet multiplicity, the shape effect inHTall is measured in lower jet multiplicities and is then propagated to the signal enriched regions allowing for a better estimate of the shape of the parton-shower uncertainty.
6.1. Results from Fits to Pseudodata in the 1L+OS Channel
-28.0 0.3 -0.3 35.7 9.6 -21.1 -8.3 -6.4 2.0 10.0 -0.4 -11.9 6.7 -36.5 -6.6 3.5 -5.2 -16.0 -2.6 22.4 -8.2 -0.3 -6.9 -4.9 12.6 -4.8 100.0
-4.8 -8.7 -15.0 7.7 -9.5 -14.2 6.1 1.3 1.1 8.8 7.4 1.2 6.0 11.3 3.8 10.0 -7.1 5.0 -20.4 -18.3 0.4 11.6 18.8 4.4 -0.0 100.0 -4.8
-10.4 -27.3 38.5 0.7 25.2 -25.3 1.4 3.0 -10.8 -0.7 -3.1 3.3 -7.6 -4.0 -3.4 -0.4 29.3 -6.3 21.6 38.3 0.6 -8.4 -0.4 -27.9 100.0 -0.0 12.6
-2.4 13.3 -18.3 6.3 -10.0 8.6 3.6 1.9 2.3 -2.9 0.9 7.7 3.1 9.4 0.5 12.4 -9.9 1.1 -8.2 -20.9 6.3 -3.8 3.0 100.0 -27.9 4.4 -4.9
2.6 6.2 -13.6 4.6 -5.7 -5.4 2.3 2.8 11.2 -1.5 14.7 1.3 8.4 6.6 4.2 11.5 -14.4 1.8 -12.2 -19.0 12.0 0.8 100.0 3.0 -0.4 18.8 -6.9
-18.2 1.5 2.2 3.1 -3.4 9.9 4.3 2.4 -4.3 3.0 -5.3 -3.4 -6.1 2.7 -5.3 3.5 9.5 4.7 9.8 9.8 0.3 100.0 0.8 -3.8 -8.4 11.6 -0.3
-1.9 -1.7 -7.1 14.1 -3.5 -23.0 2.3 -1.2 7.8 1.8 1.6 -5.3 -3.1 4.8 -1.1 -12.9 -9.6 -2.5 -10.9 -9.2 100.0 0.3 12.0 6.3 0.6 0.4 -8.2
3.4 -45.8 42.4 9.8 7.7 -13.2 -5.1 0.0 4.4 -0.0 12.7 6.0 -44.5 -7.1 2.7 -19.5 -5.8 -2.4 36.6 100.0 -9.2 9.8 -19.0 -20.9 38.3 -18.3 22.4 18.9 27.4 6.9 -10.1 -10.4 -3.0 -5.8 -3.9 4.9 2.0 -0.4 -1.1 -8.1 -7.0 0.2 0.9 -29.1 -10.0 100.0 36.6 -10.9 9.8 -12.2 -8.2 21.6 -20.4 -2.6
4.2 -6.5 0.4 -0.7 0.1 2.9 1.8 -0.1 -1.5 -0.5 1.1 -9.2 -5.6 -2.6 -1.6 1.1 2.1 100.0 -10.0 -2.4 -2.5 4.7 1.8 1.1 -6.3 5.0 -16.0
-14.7 -30.8 11.4 -14.8 5.7 12.5 3.9 6.9 -18.1 2.8 -31.6 1.8 7.9 7.1 4.8 2.4 100.0 2.1 -29.1 -5.8 -9.6 9.5 -14.4 -9.9 29.3 -7.1 -5.2
10.4 -6.0 -3.6 -9.6 2.8 5.1 3.8 0.9 0.5 -2.6 14.8 -6.7 4.6 -12.5 0.0 100.0 2.4 1.1 0.9 -19.5 -12.9 3.5 11.5 12.4 -0.4 10.0 3.5
-6.5 2.3 10.0 5.7 1.3 4.2 0.5 -3.0 -0.9 4.0 -2.6 -0.5 6.9 4.6 100.0 0.0 4.8 -1.6 0.2 2.7 -1.1 -5.3 4.2 0.5 -3.4 3.8 -6.6
20.3 -13.3 -4.1 -28.1 -9.7 -20.8 10.1 21.4 -1.5 -7.3 -0.5 -10.7 6.4 100.0 4.6 -12.5 7.1 -2.6 -7.0 -7.1 4.8 2.7 6.6 9.4 -4.0 11.3 -36.5
-7.4 21.3 -20.2 -24.9 -7.3 11.1 6.5 3.4 1.2 9.8 -25.7 -5.8 100.0 6.4 6.9 4.6 7.9 -5.6 -8.1 -44.5 -3.1 -6.1 8.4 3.1 -7.6 6.0 6.7
26.7 -18.6 7.1 9.9 6.5 -0.7 3.6 4.1 -1.8 -1.6 0.1 100.0 -5.8 -10.7 -0.5 -6.7 1.8 -9.2 -1.1 6.0 -5.3 -3.4 1.3 7.7 3.3 1.2 -11.9
11.5 -9.3 13.1 -10.5 -5.8 8.4 -0.5 -0.7 -18.9 0.4 100.0 0.1 -25.7 -0.5 -2.6 14.8 -31.6 1.1 -0.4 12.7 1.6 -5.3 14.7 0.9 -3.1 7.4 -0.4
20.3 -13.6 -0.3 -6.3 -1.8 -3.5 -0.9 1.9 2.4 100.0 0.4 -1.6 9.8 -7.3 4.0 -2.6 2.8 -0.5 2.0 -0.0 1.8 3.0 -1.5 -2.9 -0.7 8.8 10.0
-1.8 -7.3 11.6 -8.7 5.3 7.7 -0.0 -0.5 100.0 2.4 -18.9 -1.8 1.2 -1.5 -0.9 0.5 -18.1 -1.5 4.9 4.4 7.8 -4.3 11.2 2.3 -10.8 1.1 2.0
-13.6 5.4 2.4 -6.8 0.8 -6.7 -16.5 100.0 -0.5 1.9 -0.7 4.1 3.4 21.4 -3.0 0.9 6.9 -0.1 -3.9 0.0 -1.2 2.4 2.8 1.9 3.0 1.3 -6.4
-5.7 -0.5 -0.6 -10.0 -1.9 -4.8 100.0 -16.5 -0.0 -0.9 -0.5 3.6 6.5 10.1 0.5 3.8 3.9 1.8 -5.8 -5.1 2.3 4.3 2.3 3.6 1.4 6.1 -8.3
-0.7 -1.1 -4.8 -34.6 -19.3 100.0 -4.8 -6.7 7.7 -3.5 8.4 -0.7 11.1 -20.8 4.2 5.1 12.5 2.9 -3.0 -13.2 -23.0 9.9 -5.4 8.6 -25.3 -14.2 -21.1
16.4 14.3 85.3 14.2 100.0 -19.3 -1.9 0.8 5.3 -1.8 -5.8 6.5 -7.3 -9.7 1.3 2.8 5.7 0.1 -10.4 7.7 -3.5 -3.4 -5.7 -10.0 25.2 -9.5 9.6
6.8 -10.9 -8.7 100.0 14.2 -34.6 -10.0 -6.8 -8.7 -6.3 -10.5 9.9 -24.9 -28.1 5.7 -9.6 -14.8 -0.7 -10.1 9.8 14.1 3.1 4.6 6.3 0.7 7.7 35.7 13.9 -9.4 100.0 -8.7 85.3 -4.8 -0.6 2.4 11.6 -0.3 13.1 7.1 -20.2 -4.1 10.0 -3.6 11.4 0.4 6.9 42.4 -7.1 2.2 -13.6 -18.3 38.5 -15.0 -0.3 -35.5 100.0 -9.4 -10.9 14.3 -1.1 -0.5 5.4 -7.3 -13.6 -9.3 -18.6 21.3 -13.3 2.3 -6.0 -30.8 -6.5 27.4 -45.8 -1.7 1.5 6.2 13.3 -27.3 -8.7 0.3 100.0 -35.5 13.9 6.8 16.4 -0.7 -5.7 -13.6 -1.8 20.3 11.5 26.7 -7.4 20.3 -6.5 10.4 -14.7 4.2 18.9 3.4 -1.9 -18.2 2.6 -2.4 -10.4 -4.8 -28.0
B cross sectiontt b cross sectiontt 2b cross section≥+tt Fµ1b FSR ≥+tt 1b hdamp scale≥+tt 1b ME choice≥+tt F scaleµ1b ≥+tt R scaleµ1b ≥+tt TTB_ttB_PhHerwig Mig TTB_ttB_PhHerwig Shape TTB_ttb_PhHerwig Mig TTB_ttb_PhHerwig Shape TTB_ttgeq2b_PhHerwig Mig TTB_ttgeq2b_PhHerwig Shape Sα1b ISR ≥+tt Fµ1c FSR ≥+tt TTC_PhHerwig Mig TTC_PhHerwig Shape 1c hdamp scale≥+tt 1c cross section≥+tt 1c ME choice≥+tt Fµ+light FSR tt TTL_PhHerwig Mig TTL_PhHerwig Shape +light hdamp scalett +light ME choicett ttttµ tt
µtt +light ME choice tt +light hdamp scale tt TTL_PhHerwig Shape
TTL_PhHerwig Mig F +light FSR µ tt
1c ME choice +≥ tt
1c cross section +≥ tt
1c hdamp scale +≥ tt TTC_PhHerwig Shape
TTC_PhHerwig Mig F 1c FSR µ +≥ tt
S 1b ISR α +≥ tt TTB_ttgeq2b_PhHerwig Shape
TTB_ttgeq2b_PhHerwig Mig TTB_ttb_PhHerwig Shape TTB_ttb_PhHerwig Mig TTB_ttB_PhHerwig Shape TTB_ttB_PhHerwig Mig R scale 1bµ +≥ tt
F scale 1b µ +≥ tt
1b ME choice +≥ tt
1b hdamp scale +≥ tt
F 1b FSR µ +≥ tt 2b cross section +≥ tt
b cross section tt B cross section tt
Figure 6.2.: Correlation matrix for the fit to thePowheg+Herwigpseudodataset with all modelling systematics. The red-dashed line indicates one of the highest correlations between thett+¯ ≥1ccross section and thet¯t+≥2bmigration.
Table 6.1.: Signal strength µ and the total fit uncertainty for three fits to Powheg +Herwig.
Systematics µt¯tt¯t ∆µttt¯¯t
PS corr. 0.87 0.53 PS decorr. 0.79 0.60 PS+modelling 1.09 0.77
0.8 0.9 1 1.1 1.2 1.3 1.4
t t t
µt
0 10 20 30 40 50 60
# PE mean: 1.08
width: 0.08 Powheg+Herwig7
0.75 0.755 0.76 0.765 0.77 0.775 0.78 0.785 0.79 0.795 0.8
tt
µtt
Δ 0
10 20 30 40 50 60 70 80 90
# PE mean: 0.770
width: 0.005 Powheg+Herwig7
Figure 6.3.: Distribution of the mean (left) and uncertainty (right) of 500 toy-experiments for the PS+modelling setup. The uncertainties are statistical.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Norm. range [a.u]
0.1
− 0.08
− 0.06
− 0.04
− 0.02
− 0 0.02 0.04 0.06 0.08 0.1
l+jets 8j 3b
allT
H BDT-score Y
nomY
nomY
syst-Figure 6.4.: Comparison of the Yields (Y) for the PS systematic (syst) with the nominal (nom) sample for the distribution of the BDT-score and the HTall variable.
Both variables are normalised to the range [0,1] to allow for a better com-parison where the origin value of the BDT-score (HTall) was [-1,1] ([0-2000]).
6.1. Results from Fits to Pseudodata in the 1L+OS Channel