4.2.3 Multi-parameter regularization
If the inversion deals with the joint retrieval of several gas species (and optionally of some auxiliary parameters), the state vectorxis a concatenation of the vectors xp corresponding to the concentration profile of thepth molecule. Moreover, all individual regularization termsLp are assembled into a global matrix Lwith a block-diagonal structure based on the assumption that the components of the state vector are independent:
L =
√λ1L1 0 · · · 0
0 √
λ2L2 · · · 0 ... ... . .. ... 0 0 · · · √
λNLN
, (4.32)
whereNis the number of target molecules. Note that the regularization parameters in Eq. (4.32) are included in the expression of the global regularization matrix. Multi-parameter regulariza-tion schemes can be classified into two types according to the objective of the inverse problem:
• complete multi-parameter regularization scheme, in which the regularized solution corre-sponding to the entire state vector is computed;
• partial multi-parameter regularization scheme, in which only some components of the state vector are retrieved with a sufficient accuracy, e.g. the joint fit of one molecule considered as a main target and an auxiliary atmospheric profile or instrument parameter considered as a contamination.
In this regard, the regularization parametersλkcan be selected by minimizing certain objective functions with respect to the entire state vector or corresponding to the main target of the state vector, respectively [Doicu et al., 2010].
Figure 4.2: Data flow diagram for illustrating the different type of errors.
In this context, we can say that the instrument is sensitive over the “entire” spectral domain to a ±ε-variation in the kth component of the state vector about the a priori, if|[∆Fk]i|> σ for all i= 1, . . . , m, whereσ2 is the noise variance.
By using these diagnostic techniques, the sensitivity of the limb radiances to the unknowns of the inverse problems (i.e. vertical distributions of the molecular concentration or the temper-ature) can be studied. For a finer altitude grid, the unfavorable situation is that the retrieval of these quantities is essentially based on information coming from the a priori knowledge and not from the measurement itself. To remediate this deficiency, we can choose a coarser retrieval grid or employ higher-order interpolation schemes.
4.3.2 Error analysis and characterization
When the iterative process converges, it is important to assess the reliability of the retrieval products through an error analysis and characterization. An elaborate discussion covering the topic of error analysis and characterization can be found in Rodgers [2000] and Doicu et al.
[2010]. To explain the different type of errors, we illustrate the data flow diagram in Fig. 4.2.
The state vector x and the forward model parametersb are mapped by the radiative transfer model into the forward model functionF(x,b). Similarly, the signal measured by the instrument sδ is mapped by the instrument model together with the instrument model parameters c into the noise data vector yδ. The noisy data vector yδ sums the contribution of the instrument model function and the measurement (or the instrument) noiseδ, wheresis the signal measured by the instrument in the noise-free case. Finally, in the inversion model, the residual vector f(x) =F(x,b)−yδ is used to compute the Tikhonov function.
Given the regularized generalized inverseK†λ, the averaging kernel (or the model resolution) matrix is defined by
A = K†λK. (4.36)
Providing that the higher-order terms at the solution are neglected, the retrieval error is straight-forwardly represented by
eλ = xλ−xt = es+ey+eb+ec , (4.37) where the smoothing (or null-space) error
es = (A−In) (xt−xa) (4.38)
quantifies the loss of information due to the regularization in the inversion model,
ey = K†λδ (4.39)
is the noise error quantifying the loss of information due to the measurement noise δ,
eb = K†λδb (4.40)
is the forward model error in the state space resulting from the forward model error in the data space δb, and
ec = K†λδc (4.41)
is the instrument model error in the state space resulting from the instrument model error in the data spaceδc.
Although in practice the smoothing error is not a computable quantity, it can be alternatively approximated by
es ≈ (A−In) (xλ−xa) . (4.42)
Evidently, keeping the smoothing error small requires a small regularization parameter λ, whereas the stability of the inversion process requires a reasonably large regularization pa-rameter. Thus, the optimal value of λ has to be determined through a compromise between stability and accuracy.
As can be inferred from Eq. (4.38), the deviation of the averaging kernel matrix A from the identity matrix In characterizes the smoothing error. In atmospheric remote sensing, the averaging kernel matrix provides more information than just a characterization of the smoothing error. The rows of A, namely the averaging kernels, theoretically tend to peak at the diagonal value with a width which is a measure of the vertical resolution of the instrument. In the ideal case, A would be a unit matrix, but in practice, the regularization degrades the vertical resolution, that is reflected by the peak value and the full width at half maximum (FWHM) of this peak. The measurement response is given by the sum of the elements of each averaging kernel row:
Mi =
n
X
j=1
[A]ij , i= 1, . . . , n , (4.43) and high values around unity assure that the retrieved information comes mostly from the measurement and the contribution of the a priori profile is almost negligible. Furthermore, the trace of the averaging kernel matrix yields the degree of freedom (DOF) for the signal, that is interpreted as the number of useful independent quantities in a measurement, or a measure of information for brevity.
The forward model error eb is caused by inaccurate knowledge of the forward model pa-rameters b (atmospheric and spectroscopic parameters in the forward model). If ∆b are the uncertainties inb, the forward model error can be approximated by
δb = Kb∆b ≈ F(xt,b+ ∆b)−F(xt,b), (4.44)
Figure 4.3: Instrument model error.
whereKb is the Jacobian matrix with respect to the forward model parameters, ∂F/∂b.
The instrument model error ec is due to inaccurate knowledge of the instrument model parameters c (Fig. 4.3). The signal delivered by the instrument is given by the inverse of the instrument model functionR−1acting on the exact data vectoryand the instrument parameters c, plus the additive measurement noiseδ, i.e.sδ=s+δ=R−1(y,c) +δ. If in the Level-1 data processing step, the instrument model parameters c0 match the true instrument parametersc, the output of the instrument model read as
yδ = R(s,c) +δ = R(R−1(y,c),c) +δ = y+δ . (4.45) If this is not the case, i.e. if c0 6=c, we have
yδ = R(s,c0) +δ = R(R−1(y,c),c0) +δ = y+δc+δ, (4.46) whereδc is the instrument model error in the data space defined by
δc = R(s,c0)−R(s,c) = R(R−1(y,c),c0)−R(R−1(y,c),c) . (4.47) Thus, assuming that ∆care the uncertainties inc, the instrument model error in the data space can be computed as
δc = R(s,c+ ∆c)−R(s,c) . (4.48)
Because in the Level-2 data processing, we deal only with the noisy data vectoryδ=y+δc+δ, it is a common practice to employ an additional correction step to account for the instrument model error δc = δc(c). This situation is depicted in Fig. 4.4, and note that the instrument parameters can or cannot be included in the retrieval.
In a semi-stochastic framework, the smoothing and model parameter errors are deterministic, whereas the noise error is stochastic with zero mean and covariance matrix Sy. The quality of the regularized solution can be estimated through the mean square error matrix computed as
Sλ = Eh
(xλ−xt) (xλ−xt)T i
≈ Ss+Sy+Sb+Sc , (4.49) where under the assumption that the measurement noise δ is a white noise with variance σ2,
Figure 4.4: Correction step for the instrument model error.
we have
Ss = eseTs , (4.50)
Sy = σ2K†λK†Tλ , (4.51)
Sb = ebeTb , (4.52)
Sc = eceTc . (4.53)
The square root of the diagonal elements of Sλ gives the retrieval error, while the off-diagonal elements quantify the correlations between the components of the regularized solution xλ,
Cλ(i, j) = Sλ(i, j)
pSλ(i, i)Sλ(j, j) , (4.54)
whereiand j are the row and column index, respectively.
Note that the sum of the square root of the diagonal elements of Sλ is the expected value of the retrieval error, i.e.
Eh keλk2i
= kesk2+Eh keyk2i
+kebk2+keck2 , (4.55) where E is the expected value operator. As stated previously, the model parameter errors are deterministic. More precisely, if for example, ∆b=εbfor some scalar ε <1, we have
Sb = ε2K†λKbbbTKTbK†Tλ . (4.56) Defining the diagonal matrixB by [B]ii= [b]i,i= 1, . . . , n, the matrix of all ones1 by
1 =
1 · · · 1 ... . .. ...
1 · · · 1
, (4.57)
and the symmetric and positive definite matrixCb by
Cb = ε2B1BT , (4.58)
i.e.
[Cb]ij = ε2[b]i[b]j , i, j= 1, . . . , n . (4.59) yields
Sb = K†λKbCbKTbK†Tλ . (4.60) In a stochastic framework, when b is assumed to be a stochastic quantity, Cb defined by Eqs. (4.58) and (4.59), is interpreted as the covariance matrix of b. Thus, in view of Eq. (4.22) with l → ∞, Eqs. (4.57)–(4.59), a deterministic treatment of b is equivalent to a stochastic treatment, in which εis the model parameter standard deviation, and all components of bare perfectly correlated. If the components of b are not correlated, then the matrix of all ones 1 should be replaced by the identity matrixIn.