Alfred Wegener Institute Helmholtz Center for Polar and
Marine Research
The Error-subspace Transform Kalman Filter
Lars Nerger, Wolfgang Hiller, and Jens Schr ¨oter
Alfred Wegener Institute, Helmholtz Center for Polar and Marine Research, Bremerhaven, Germany Contact: Lars.Nerger@awi.de
·
http://www.awi.deEnsemble square-root Kalman filters are currently the computationally most efficient ensemble-based Kalman filter methods. In particular, the Ensemble Transform Kalman Filter (ETKF) [1] is known to provide a minimum ensemble transformation in a very efficient way. In order to further improve the computational efficiency, the Error-Subspace Transform Kalman Filter (ESTKF) was developed [2]. The ESTKF solves the estimation prob- lem of the Kalman filter directly in the error-subspace that is represented by the ensemble. As the ETKF, the ESTKF provides the minimum ensemble transformation, but at a slightly lower cost. Both, the ETKF and ESTKF are related to the SEIK filter [3]. This filter shows small deviations from the minimum transformation, but is similarly efficient as the ESTKF.
•
The Error Subspace Transform Kalman filter (ESTKF) is an efficient ensemble square-root filter that com- putes the weights for the ensemble transformation di- rectly in the error subspace. The transformations are identical to those of the ETKF.•
The compute performance of the ETKF can be im- proved by using a projection matrix of sizeN × N
T ˜
i,j=
1 −
N1for i = j
−
N1for i 6 = j
to compute
Z
f= X
fT ˜
.•
When the symmetric square root is used, the SEIK filter shows very similar results to those of the ETKF and ESTKF. With Cholesky decompositions, the qual- ity of the SEIK filter deteriorates.•
An implementation of the ESTKF is available in the release of the Parallel Data Assimilation Framework (PDAF) [5, 6].The figure below shows a comparison of the weight matrices used for the ensemble transformation for a single analysis step.
All matrices are projected to be of size
N × N
, e.g. for ESTKFW = TCT
T and for ETKFW ˜ = T ˜ C ˜
. ETKF’sW
is closest to the Identity, the transformation of the ESTKF is identical up to numerical precision. SEIK’sW
differs more from the identity in case of a Cholesky square-root (SEIK-chol
). With the symmetric square-root (SEIK-sym
), the transformation in SEIK is minimally different from that of the ETKF and ESTKF, but depends on the ensemble ordering.ETKF: Transformation matrix
-0.1 -0.05 0 0.05 0.1
SEIK-chol: Transformation matrix
-0.1 -0.05 0 0.05 0.1
SEIK-sym: Transformation matrix
-0.1 -0.05 0 0.05 0.1
transformation matrices difference: SEIK-ETKF
-4 -3 -2 -1 0 1 2 3 4 x 10-3
ETKF ESTKF SEIK
Z
f= X
f− X
f, Z
f∈ R
n×NS
f= X
fT , S
f∈ R
n×(N−1)L
f= X
fT ˆ , L
f∈ R
n×(N−1)T
i,j=
1 −
N1 √1 1N+1
for i = j , i < N
−
N1 √1 1N+1
for i 6 = j , i < N
−
√1Nfor i = N
T ˆ
i,j=
1 −
N1for i = j , i < N
−
N1for i 6 = j , i < N
−
N1for i = N
Notation:
State vector
x
f∈ R
n; Ensemble ofN
membersX
f=
x
f(1), . . ., x
f(N); Matrix of ensemble means
X
f=
x
f, . . . , x
fThe error subspace has a dimension of
N − 1
. The ETKFuses an ensemble representation of the error subspace of
N
ensemble perturbations. The ESTKF and the SEIKfilter directly use a basis of the error subspace of dimen- sion
N − 1
. The difference between ESTKF and SEIK is caused by the distinct projection matricesT
andT ˆ
.ETKF ESTKF SEIK
Analysis covariance matrix
P ˜
a= Z
fA ˜ ( Z
f)
TP
a= S
fA ( S
f)
TP ˆ
a= L
fA ˆ ( L
f)
Twith transformation matrix
A ˜ ∈ R
N×NA ∈ R
(N−1)×(N−1)A ˆ ∈ R
(N−1)×(N−1)A ˜
−1= ( N − 1 ) I + ( HZ
f)
TR
−1HZ
fA
−1= ( N − 1 ) I + ( HS
f)
TR
−1HS
fA ˆ
−1= ( N − 1 ) T ˆ
TT ˆ + ( HL
f)
TR
−1HL
fEnsemble transformation
X ˜
a= X
a+ √
N − 1Z
fC ˜ X
a= X
a+ √
N − 1S
fCT
TX ˆ
a= X
a+ √
N − 1L
fCT ˆ
Twith square-root
C ˜ ˜ C
T= A ˜ CC
T= A C ˆ ˆ C
T= A ˆ
The symmetric square root
C = U Λ
−1/2U
T from the singular value decompositionU Λ V
T= A
−1 can be used in all cases.All filters compute a square root of the transform matrix (
A ˜
,A
,A ˆ
). These matrices are distinct, but the ensembletransformations in ETKF and ESTKF are identical if the symmetric square root is used for both filters.
Twin experiments were conducted using the nonlinear Lorenz96 model [4] implemented in PDAF [5, 6]. Syn- thetic observations of the full state were generated from a model run. Observations were assimilated at each
time step over 50000 time steps. For SEIK, configura- tions with either symmetric square root or with a square- root based on Cholesky decomposition were used. The global formulations of the filters were used.
10 20 30 40
0.9 0.92 0.94 0.96 0.98 1
ESTKF, determin.
forgetting factor
ensemble size
10 20 30 40
0.9 0.92 0.94 0.96 0.98 1
ETKF, determin. Λ (Λ=I)
forgetting factor
ensemble size
10 20 30 40
0.9 0.92 0.94 0.96 0.98 1
SEIK-sqrt, determin.
forgetting factor
ensemble size
10 20 30 40
0.9 0.92 0.94 0.96 0.98 1
SEIK-orig, determin.
forgetting factor
ensemble size
0 0.51 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.17 0.175 0.18 0.185 0.19 0.195 0.2 0.205 0.21 0.22 0.23 0.24 0.25 0.3 0.4 0.5 0.6 0.8 1
The figure shows mean RMS errors as functions of the ensemble size and forgetting factor (covariance infla- tion). As expected, the results from ESTKF and ETKF are almost identical. The differences are only caused by the finite precision of the numerical computations.
The SEIK filter with symmetric square root provides very similar results. Errors from the SEIK filter using a Cholesky square root of
A ˆ
are larger. This is caused by an inferior ensemble quality in which a small number of ensemble members carry most of the variance.ESTKF ETKF
SEIK-sym SEIK-chol
RMS
W
: ETKFW
: SEIK-cholW
: SEIK-sym W∆ W
: SEIK-sym - ETKF W[1] CH Bishop et al.. (2001). Adap- tive Sampling with the Ensemble Transform Kalman Filter. Part I: The- oretical Aspects. Mon. Wea. Rev.
129: 420–436
[2] L Nerger et al. (2012) A Uni- fication of Ensemble Square Root Kalman Filters. Mon. Wea. Rev.
140: 2335–2345
[3] DT Pham et al. (1998). Singu- lar evolutive Kalman filters for data assimilation in oceanography. C. R.
Acad. Sci. Series II 326: 255–260
[4] EN Lorenz. (1996). Predictabil- ity - a problem partly solved. Pro- ceedings Seminar on Predictability, ECMWF, Reading, UK, 1–18.
[5] Parallel Data Assimilation Frame- work (PDAF) – an open source framework for ensemble data assim- ilation. http://pdaf.awi.de
[6] Nerger, L. and W. Hiller (2013).
Software for Ensemble-based Data Assimilation Systems – Implementa- tion Strategies and Scalability. Com- puters & Geosciences. 55: 110–118