In most practical cases, the Fisher Information Matrix (FIM) is put in operation to quantify the uncertainty about estimated model parameters, ˆθ∈Rl. For a set of mea-surement sample time points,ydata(tk), ∀k= 1, . . . , K, and the associated measurement (co)variance matrix,Cy(tk), the FIM is defined by
F IM =
K
X
tk
SMtTk·Cy(t−1
k)·SMtk (3.1)
Here, the sensitivity matrix,SMtk, reads
SMtk =
∂y1(tk)
∂θ1
ˆ
θ1
∂y1(tk)
∂θ2
ˆ
θ2
· · · ∂y∂θ1(tk)
l
ˆ
θl
∂y2(tk)
∂θ1
ˆ
θ1
∂y2(tk)
∂θ2
ˆ
θ2
· · · ...
... ... . .. ...
∂ym(tk)
∂θ1
ˆ
θ1
· · · ∂y∂θm(tk)
l−1
ˆ
θl−1
∂ym(tk)
∂θl
ˆ
θl
(3.2)
In case of ODEs, a matrix differential equation system for the sensitivities has to be solved in parallel to the original model
SM˙ = ∂f
∂y ·SM+ ∂f
∂θ ; SM(0) = 0m×l (3.3)
Subsequently, the inverse of FIM in relation to the Cramer-Ra´o inequality [Kay93]
provides a lower bound of the parameter covariance matrix, Cθˆ, according to Cθˆ≥ ∂g(θ)
∂θ F IM−1∂g(θ)T
∂θ , (3.4)
whereg(θ) =θ+Bi(ˆθ). Here, the bias term, Bi(ˆθ) =E[ˆθ]−θ, describes a systematic deviation of the estimated to the true parameter values. The equality only holds, if (i) the measurement errors are additive, and (ii) the model is linear in its parameters.
Moreover, the FIM does not give any information onE[ˆθ]. Therefore, in many cases it is additionally assumed that (iii) the estimates are unbiased, E[ˆθ] =θ → Bi(ˆθ) = 0.
Consequently, the parameter covariance matrix is approximated by
Cθˆ≈F IM−1 (3.5)
In practical applications, however, the FIM approach may lead to poor approximations of Cθˆ. Obviously, the Fisher Information Matrix is based on the same linearisation principles as shown in Sec. 2.2.2.1. Here, too, linear surrogates of non-linear problems might be poor representatives, i.e., results which are based on linearisation are likely to fail. Moreover, the derivatives in FIM have to be evaluated at parameter values, θ, which are actually unknown. A remedy might be the following procedure which is frequently implemented in practice:
“
... a prior guess ˆθ0 for θ is used to design the experiment, with the hope that the local optimal design for ˆθ0 will be close to the optimal one for the unknown θ. When the alternation of estimation and design phases is possible, sequential design permits to progressively adapt the experiment to an estimated value ofθthat (hopefully) converges to its unknown true value.A. P´azman & L. Pronzato [PP07]
”
To avoid, or at least to reduce, costly reiterations of parameter estimation and design phases, robust OED strategies based on min-max optimisation principles have been derived [KKBS04, TLDI12a, TLDI12b]. Here the parameter uncertainty is addressed explicitly, i.e., the parameters are characterised by confidence intervals. Hence, the FIM is not evaluated at a single parameter vector of estimates exclusively. Instead, a bounded parameters space is explored in parallel searching for parameter values at which OED has the worst performance in spite of optimal operating conditions. That means, the worst-case scenario in relation to the unknown model parameters,θ∈Θ, is solved for OED.
In the particular case of FIM-based OED, however, even a proper choice of model pa-rameters is no guarantee of optimally designed experiments for non-linear problems.
As shown in Sec. 3.2, the approximation error due to linearisation may provide sub-optimal results as well, see [BW04, JSMK06, VG07] for confirming references. Thus, an improvement in the approximation accuracy is likely to provide more suitable op-erating conditions, i.e., to provide more informative data. Nevertheless, in the field of
systems biology, even today, the implementation of the Fisher Information Matrix is state-of-the-art in OED.
Obviously, the Unscented Transformation approach is an appropriate alternative to determine the uncertainty about the estimated model parameters, ˆθ. As demonstrated in Sec. 2.2.3, the UT method provides approximations about the mean, E[ˆθ], and the covariance matrix, Cθˆ, respectively. Thus, the mean square error matrix of the estimated parameters, M SEθˆ[Kay93], can be determined via
M SEθˆ=E[(ˆθ−θ)(ˆθ−θ)T]≈Cθˆ+Bi(ˆθ)Bi(ˆθ)T (3.6) Consequently, instead of analysing the covariance matrix exclusively, M SEθˆ is pro-cessed in OED. But independently of the analysed uncertainty matrix, Cθˆ orM SEθˆ, numerical optimisation routines require an adequate representation of these matrices.
Thus, for the purpose of numerical implementation, a scalar utility function of the ap-proximated parameter uncertainty matrix has to be derived. Instead of solving a multi-objective optimisation problem which aims to minimise every element ofCθˆ orM SEθˆ, respectively, a one-dimensional compromise function is used. Therefore, well known optimality criteria exist in literature and are frequently applied in practice [WP97], e.g.,
A−optimal design ΦA(Cθˆ) =trace(Cθˆ) (3.7) D−optimal design ΦD(Cθˆ) =det(Cθˆ) (3.8) E∗−optimal design ΦE∗(Cθˆ) = λmax(Cθˆ)
λmin(Cθˆ) (3.9) with λmax (λmin) as the maximum (minimum) eigenvalue of Cθˆ. Please note that Cθˆ might be replaced by M SEθˆ in case of the UT approach. In general, however, the choice of the design criterion influences the outcome of OED and it is not clear in advance which criterion will produce the best result, i.e., which criterion will produce the most precise parameter estimates. Naturally, at this point, it raises the question whether it is really necessary to identify all model parameters with the same accuracy.
As stated previously, the objective of most mathematical models is to provide reliable simulation results, i.e., to provide meaningful model-based inferences. Therefore, a cost function which is based on the parameter precision exclusively is usually not an ideal
option for non-linear models. Admittedly, in the special case of linear problems there might be no differences between these two objectives. Both are equivalent objectives under stringent conditions, see [KW60, Won94, Won95] and references therein. For non-linear problems, however, this equivalence is not any longer a valid assumption [BHC+04, GWC+07]. Only a subset of the unknown model parameters and combina-tions thereof determine the qualitative behaviour of the model. Thus, when the main interest lies in obtaining a predictive model but not in identifying certain parameter values, it is not necessary to reduce all parameter uncertainties by the same amount.
In consequence, OED should provide models with tight confidence intervals of simula-tion results, which are indirectly influenced by the imperfect parameter identificasimula-tion process. Here, the universal concept of the UT method provides also an appropriate and elegant way to take these considerations into account, see Sec. 2.2.3.3. In detail, the UT approach provides the mean and the (co)variance matrix of the states of the simulated time interval. A suitable cost function taking the uncertainty aboutx(t) into account is
ΦU TM SEx =trace
tend
Z
t0
M SE(x(t,θ, u))dtˆ
. (3.10)
Here, the mean square error matrix of the states is approximated according to
M SE(x(t,θ, u)) =ˆ E[(x(t,θ, u)−x(t, θ, u))(x(t,ˆ θ, u)−x(t, θ, u))ˆ T]≈Cx(t)U Tˆ +Bix(t)ˆ BiTx(t)ˆ (3.11) In doing so, the M SE(x(t,θ, u)) is evaluated at validation conditions, i.e., conditionsˆ which have not been part of a former parameter identification process. Hence, by minimising ΦU TM SE
x the uncertainty about the most sensitive parameters and param-eter combinations is reduced automatically. The correlation of global paramparam-eter sen-sitivities and the mean square error of prediction has been analysed empirically in [LML+09, LMM10].