atlas_conf_2013_031 Higgs 3615.0 3450.0 180.0 0.0 0.055 484.0 363.0 0 0
atlas_conf_2013_036 SR1Z 3.0 1.3 1.0 0.0 0.055 6.5 4.5 0 0
atlas_conf_2013_049 SR_mT2_110_elmu 5.0 4.4 2.0 0.0 0.053 7.105 6.699 0 0 atlas_conf_2013_061 SR0L7JA 22.0 22.5 6.9 0.06 0.053 15.3 14.6 0 0 atlas_conf_2013_062 SoftLep1BHigh 6.0 4.0 1.1 0.01 0.053 7.9 6.3 0 0 atlas_conf_2013_089 SR1OF 87.0 103.0 15.0 0.0 0.053 24.0 31.0 0 0
atlas_conf_2014_014 SRa 59.0 53.0 10.0 0.0 0.053 30.2 27.0 0 0
atlas_conf_2014_033 emu 5067.0 4376.0 281 0.0 0.053 1176.0 566.0 0 0 atlas_conf_2014_056 sig 60424.0 60000.0 3600.0 0.0 0.053 6902.0 6717.0 0 0
atlas_conf_2015_004 M1 539.0 578.0 48.4 0.0 0.053 73.0 96.0 0 0
cms_1301_4698_WW combined 1111.0 1000.0 60.0 0.0 0.009 240.4 135.7 0 0 cms_1303_2985 23j_0b_875 25.0 16.1 1.7 7.98 0.447 18.5 10.1 0.383 0.701 cms_1405_7570 Zjj_030 20991.0 21364.0 859.0 0.0 0.051 1379 1595 0 0 cms_1408_3583 550 519.0 509.0 66.0 10.7 0.694 129.0 123.0 0.073 0.077 cms_1502_06031 SR01_GE2jets_c.highMET 7.0 12.8 4.3 0.0 0.051 7.6 7.6 0 0
cms_1504_03198 SR1 18.0 16.4 3.64 0.0 0.052 12.9 11.4 0 0
cms_smp_12_006 0e 557.0 487.8 40.0 0.0 0.052 151.62 88.98 0 0
cms_sus_12_019 For_OF 155.0 155.0 16.4 0.0 0.051 31.8 31.8 0 0
cms_sus_13_016 SR1 1.0 1.2 1.05 0.0 0.051 4.0 3.9 0 0
This file is helpful in getting a good overview of which analyses yield a nonvanishing r value and hence generally show sensitivity to the input model. For our example, one would expect those analyses to be most sensitive which are designed to find events with large jet multiplicity and missing transverse energy in the final state. Indeed one examines three such analyses with sizable r-values: atlas 1405 7875 [183], a zero lepton multijet search, the ATLAS monojet12 search atlas 1502 01518 [185] and the CMS search cms 1303 2985 [186] which uses the αT
variable to identify BSM events with large hadronic activity.
CheckMATE then again chooses the most sensitive region among these. The corresponding observed result will be used to finally conclude whether the input can be considered excluded or not, i.e. in the case of the r-limit if robscons is larger than 1. In the above example, this best signal region would be SR02 3jit in analysis atlas 1405 7875 which with anrobscons value of about 1.7 yields excludes the input signal. This is exactly the result which was printed on-screen at the end of our originalCheckMATE run.
With that, we have illustrated howCheckMATE can be used to test various input formats for BSM physics and which content can be found in all the produced output files. This knowledge should be sufficient for standard users to test their models of interest without much effort.
After finishing this detailed explanation of the main part of CheckMATE the user typically gets into contact with when running the program, we continue the discussion with some inter-nal details which are required to make CheckMATE work and which are thus also worthwhile mentioning. This dicussion also illustrates for which other purposes the program can be used.
4.3 Detector Tunings
We already discussed the general concept of a detector simulation in Section 3.3.1. Within CheckMATE, each of the given event files will be processed with the fast multipurpose detector simulationDelphes. This tool attempts to reproduce the experimental resolutions and efficien-cies of the two LHC multipurpose detectors, ATLAS and CMS by parametrising a large list of detector effects. Most importantly it
simulates track reconstruction,
12Despite the name, this analysis allows for events with up to three hard jets in the final state and hence is also sensitive to our expected multijet signature.
determines hadronic and electromagnetic energy deposits of all particles,
applies identification efficiencies for photons and leptons,
clusters jets,
performs energy/momentum smearings of all reconstructed objects,
evaluates total missing energy,
checks isolation conditions for photons and leptons and
appliesb-/τ-tagging on jets.
Due to the use of efficiency maps instead of proper simulation of individual particles in detec-tor material, the time scale of a typical Delphes run is about an order of magnitude smaller compared to the overall computational time required for the generation, showering and hadro-nisation of the same events. For more details on Delphes in general, its implemented features and performance tests we refer to Ref. [165].
CheckMATE requires extra functionalities which lie beyond those in the public version of Delphes. Most importantly, as already explained in Section 4.1.4, the simultaneous testing of various analyses at once requires many different sets of final state objects passing different identification algorithms to be determined and stored in parallel. This forking of the same reconstructed to different identified final state objects is not possible within Delphes, which is why we have outsourced these parts of the detector simulation into the AnalysisHandlers objects discussed before. At the same time, we re-parametrised many of the efficiencies which are already present inDelphes’ standard implementations of the ATLAS and the CMS detector.
In the remainder of this section we describe these changes in detail. We show the implemented efficiency distributions and the data points these were fitted to. The functional forms of these distributions are listed in Appendix B.3.
4.3.1 Improved Description of Lepton Reconstruction
In order to properly estimate the measurement of leptons inside a detector, there are two main effects which have to be taken into account:
1. Inaccuracies in tracker, calorimeters and muon spectrometer lead to an uncertainty in the reconstruction of the kinematical properties of electron and muon candidates. These can be accounted for in the simulation by applying a Gaussian smearing on every candidate in dependence on its energy and position in the detector.
2. Algorithms to reconstruct electrons inside the calorimeter and to identify muons by as-sociating tracks to hits in the muon chamber might fail. Hence a given generated lepton should only appear in the list of reconstructed lepton objects with a probability given by the reconstruction and identification efficiency`≤1.
There exist sophisticated experimental studies which provide quantitative statements for these effects, e.g. in the form of probability functions. InCheckMATE, the following parameterisations are included:
4.3 Detector Tunings
50 100 150 200 250 300 350 E(e)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(e)
0 2 4 6 8 10
smearingσEinGeV
(a) Discretised version of continuous 2D func-tion used for smearing electron energies.
50 100 150 200 250 300 350 pT(µ)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(µ)
0 10 20 30 40 50
smearingσpTinGeV
(b) Discretised version of continuous 2D func-tion used for smearing muon momenta,
0 10 20 30 40 50 60 70≥80 pT(e)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(e)
0.5 0.6 0.7 0.8 0.9 1.0
total’medium’efficiency
(c) Discretised version of continuous 2D effi-ciency map for ’medium’ ATLAS electrons.
0 10 20 30 40 50 60 70≥80 pT(e)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(e)
0.5 0.6 0.7 0.8 0.9 1.0
total’tight’efficiency
(d) Discretised version of continuous 2D effi-ciency map for ’tight’ ATLAS electrons.
2.5 1.5 0.5 0.5 1.5 2.5 η(µ)
−π
−π/2 0 π/2 π
φ(µ)
0.90 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1.00
’combined+’efficiency
(e) Discrete 2D efficiency map for ’combined plus’ ATLAS muons.
2.5 1.5 0.5 0.5 1.5 2.5 η(µ)
−π
−π/2 0 π/2 π
φ(µ)
0.90 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1.00
’combined’efficiency
(f) Discrete 2D efficiency map for ’combined’
ATLAS muons.
Figure 4.2: Lepton smearing and efficiency distributions used inCheckMATE.
A Gaussian smearing of the muon momenta and electron energies is applied by using position- and energy/momentum-dependent widths taken from results in Refs. [187–191]
and for high-energetic muons fitted13 to match results in Ref. [192]. They are shown in Figs. 4.2aand 4.2b. Note that to improve readability we show the results in the form of a 2D histogram even though we use continuous functions internally.
Detailed pT- and η-dependent reconstruction and identification efficiency functions for
‘medium’ and ‘tight’ electrons14are determined from results in Refs. [189,193]. They are shown in Figs.4.2cand4.2d. Again, we show histogrammed results to improve readability even though continuous functions are used.
Muon reconstruction efficiencies for ’combined’ and ’combined plus segment tagged’ muons are implemented in dependence of the detector component that will measure the candi-date. We use aφ–ηmap of the muon spectrometer from Ref. [188] and show the resulting dicrete two-dimensional efficiency maps in Figs. 4.2eand 4.2f.
4.3.2 Improved Description of Jet Tagging
Jets which originate fromb-quarks or hadronically decaying τ-leptons can to a certain degree be distinguished from other, non-flavoured jets by e.g. measuring vertex displacements or by reconstructing distinctive track and calorimeter signatures. The experimental collaborations have determined sophisticated algorithms which are applied on every reconstructed jet object in order to identify whether they are likely to have originated from ab-quark or aτ-lepton. Since under different circumstances it is sometimes a high signal rate and sometimes a low background contamination which is targeted, these algorithms are defined with diferent discrete or even continuous working point efficiencies. The resulting working point dependent probabilities of correctly and incorrectly tagging jets which did or did not originate from a b/τ have been published and are used in fast detector simulation frameworks like ours. For the background, a typical quantity is the so calledrejection which is simply the inverse of the background tagging efficiency.
We consider pT-dependent b-quark identification efficiencies and mis-tagging probabili-ties according to Refs. [194–197]. Results for ROC-curves, i.e. relations between signal-and background efficiencies, signal-and the momentum dependence of these efficiencies are given in Fig. 4.3. The typically higher mistagging rate of jets which originate from c-quarks are considered separately. Various target working point efficiencies are used in different analyses andCheckMATE uses the Receiver Operating Characteristic (ROC) curve to de-termine the absolute background efficiency for a given working point and then uses the momentum-dependent functions to re-scale this probability, see also Appendix B.3.4.
Note that the efficiencies in the above references have been determined to generally over-estimate the number of Monte-Carlo b-jets which pass the tagging filter. Therefore the efficiencies have been scaled down in such a way that the cutflow validation procedure yielded globally better agreements between CheckMATE and the respective experimental result.
13 This fit has been performed by Florian Jetter.
14 ‘loose’ electrons are considered in a special manner, see AppendixB.3.
4.3 Detector Tunings
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 signal efficiency
10−1 100 101 102 103 104 105
lightjetrejection
CheckMATE ATLAS
(a) b-tag ROC curve. Rejection considers jets from light quarks only.
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 signal efficiency
100 101 102
c-jetrejection
CheckMATE ATLAS
(b)b-tag ROC curve. Rejection considers jets fromc quarks only.
0 50 100 150 200 250 300 pT(b)in GeV
0.0 0.2 0.4 0.6 0.8 1.0
signalefficiency CheckMATE
Fit
ATLAS (Muon Data) ATLAS (Top Data)
(c) pT dependence of signal efficiencies for b-tagging for a working point efficiency of 70 %, determined by downscaling (see text) the com-bined information of two different search chan-nels for [197,198].
0 20 40 60 80 100 120 140 160 pT(c)in GeV
0.2 0.3 0.4 0.5 0.6
D∗-jetefficiency
CheckMATE ATLAS
(d)pT dependence ofb-tagging background effi-ciency of jets containingD∗mesons for a work-ing point efficiency of 70 %, determined from Ref. [197]. The inclusive efficiency onc–jets is assumed to be 40 % of thisD∗ efficiency.
0 100 200 300 400 500 600 700 800 pT(j)in GeV
0.01 0.02 0.03 0.04 0.05
lightjetefficiency
CheckMATE Fit
ATLAS
(e) b-tagging efficiency of jets containing light quarks for a working point efficiency of 70 % and|η|<1.3, see Ref. [196].
0 100 200 300 400 500 600 700 800 pT(j)in GeV
0.01 0.02 0.03 0.04 0.05 0.06 0.07
lightjetefficiency
CheckMATE Fit
ATLAS
(f) same as Fig.4.3efor 1.3≤ |η|<2.5.
0 50 100 150 200 250 pT(τ>1p)in GeV
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
signalefficiency
Loose Medium Tight CheckMATE ATLAS
(a)τ-tagging efficiencies for multiprongτ can-didates.
25 50 75 100 125 150 175 200 pT(τ1p)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(τ1p)
0.0 0.2 0.4 0.6 0.8 1.0
total’loose’signalefficiency
(b) ’loose’τ-tagging efficiency for 1-prongτ can-didates,
0 50 100 150 200 250
pT(j1p)in GeV 10−2
10−1 100
backgroundefficiency Loose
Medium Tight CheckMATE ATLAS
(c) τ-tagging background efficiencies for 1-prong jets.
25 50 75 100 125 150 175 200 pT(τ1p)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(τ1p)
0.0 0.2 0.4 0.6 0.8 1.0
total’medium’signalefficiency
(d) ’medium’ τ-tagging efficiency for 1-prong τ candidates.
0 50 100 150 200 250
pT(j>1p)in GeV 10−3
10−2 10−1
backgroundefficiency Loose
Medium Tight CheckMATE ATLAS
(e) τ-tagging background efficiencies for 3-prong jets.
25 50 75 100 125 150 175 200 pT(τ1p)in GeV
−2.5
−2.0
−1.5
−1.0
−0.5 0 0.5 1.0 1.5 2.0 2.5
η(τ1p)
0.0 0.2 0.4 0.6 0.8 1.0
total’tight’signalefficiency
(f) ’tight’τ-tagging efficiency for 1-prongτ can-didates.
Figure 4.4: Distributions used to perform phenomenologicτ-tagging inCheckMATE