Correction of spurious trends in climate series caused by inhomogeneities

(1)

Correction of spurious trends in climate series caused by

inhomogeneities

Ralf Lindau

(2)

Break detection

Consider the differences of one station compared to a neighbor reference from a surrounding network of stations.

The dominating natural variance is cancelled out, because it is very similar at both stations.

Breaks become visible by abrupt changes in the station-reference time series.

Internal variance (Noise) within the subperiods External variance (Signal)

between the means of different subperiods

Break criterion:

Maximum external (explained) variance

(3)

Break-aware idea

Breaks are only detected, but not corrected.

Calculate the mean trend over all homogeneous subperiods

(omitting the known breakpoints).

This trend should reflect the true trend.

Dipdoc Seminar – 12. November 2015

� ��= �²−1

�² �_��+ 1

�² �_��

Is not promising, because:

(4)

Correction is important

Known break positions alone are not sufficient to derive reliable trends.

The correction part of homogenization determines a fraction of of the trend, when N is the number of subsegments.

Even for only one break this is equal to ¾.



(5)

Station or network trend?

From my point of view, the network-mean (regionally averaged) trend is per se more interesting.

Moreover, trend corrections for individual stations are easy to derive, if the network-mean trend is known.

Therefore, we concentrate on corrections of the network-mean trend.

(6)

Network-mean trend error

� _� ₀= � + �_� ₀

Observed = True + Spurious station trend

1

� ∑

�=1

�

(^�_�₀⁻^� _�)⁼ ¹

� ∑

�=1

�

(^�⁺^�_�₀⁻^� ⁻^�_�)⁼^�_�₀⁻ ¹

� ∑

�=1

�

�_�

Observed trend anomaly = Spurious station trend against the network-mean – network-mean trend error trend

(7)

Two ways of correction

1. Break-by-break method:

Consider an individual break of one candidate station.

Compare the two homogeneous subperiods before and after this break with homogeneous subperiods of suited neighbor

stations, which preferably long overlap periods with the candidate break.

2. ANOVA method:

Minimize the variance of the entire network.

Discussed in the following, because it is better defined.

Dipdoc Seminar – 16. June 2014

(8)

ANOVA correction scheme (1/3)

Observation b (station, year) Climate signal c (year)

Inhomogeneity a (station, year) Noise e (station, year)

Minimize the squared difference between theory and observations.

Derivation with respect to c(j) leads to m equations, one for each c(j).

�(�, �)=�( �)+�(�, �)+� (�, � )

∑

�=1

�

∑

�=1

�

(�( _�)₊_�(_�_, _�)₋_�(_�_, _�))²=min

�( _�)₌ ¹

�∑

�=1

�

(�(_�_, _�)₋_�(_�_, _�))=0

(9)

Analogously, we get h equations, one for each of the homogeneous sub-periods.

Altogether we have m+h equations for m+h unknowns.

Thus a (m+h) x (m+h) matrix has to be solved.

E.g. 115x115-matrix for 100 years and 15 subperiods.

We can insert the m climate equations for each year into the h equations for each subperiod, so that only a hxh-matrix has to solved.

(10)

n = 5 stations with data from 100 years.

h = 15 homogeneous sub-periods in total.

Consider sub-period no. 8:

It has an overlap of 32 years with a₂, of 13 years with a₄,

of 19 years with a₅, etc.

The overlap periods constitutes the non- diagonal matrix elements.

The diagonal is given by (n-1) length(a_i)

(11)

Simulated data

Test the performance of the ANOVA correction scheme with simulated data.

The simulated data consist of three superimposed signals:

1. The climate signal, which identical for all stations of a network.

2. Noise, which mimics the difference between the stations, e.g. due to weather.

3. Inhomogeneities inserted at random timings and with random strengths.

(12)

Test under perfect conditions

1000 networks

10 stations with 5 breaks each No mean trend error

No noise

Known break positions Result:

Perfect skill.

- We made no programming errors.

- The method works perfectly.

Inserted yearly station inhomogeneity

Detected yearly station inhomogeneity

(13)

Signal / noise = 1

Still: No mean trend error

Perfectly known break positions Equal break and noise variance:

SNR = 1

Result:

As expected no longer perfect, but r = 0.955

Inserted yearly station inhomogeneity

Detected yearly station inhomogeneity

(14)

Network-mean trends

From individual inhomogeneities to network-mean trends.

Both regressions are calculated.

That taking the x-axis data as independent is in all three cases equal to the 1-to-1 line.

What does this mean?

Inhomogeneities Station trends Network trends

r = 0.9550 r = 0.9525 r = 0.8092

(15)

What does it mean?

It means that:

y is equal to x plus random scatter e.

Because in this case, the following equation chain holds true:



(16)

Remaining trend error

It is convenient to display not the detected (y), but the remaining (y-x) trend error.

As shown the inserted and the remaining quantities are

uncorrelated.

The remaining errors are smaller, but comparable in size.

This is valid for SNR = 1

Inserted network-mean trend error

Remaining network-mean trend error

s_x² = 0.265 s_y² = 0.141

(17)

Preliminary conclusion I

For perfectly known breakpoints, time series lengths of 100 with 5 breaks, SNR = 1, and no mean trend introduced by the breaks the situation is:

The result after homogenization is slightly better than the original.

The mean trend is zero, the uncertainty before is: 0.265 and afterwards: 0.141

Ratio q: (Improvement)



(18)

Improvement ratio (1/2)

Ratio q depends on the SNR.

Upper panel:

Doubled noise (SNR = ½ ) leads to doubled remaining trend error. The inserted trend error is unchanged.

Doubled q Lower panel:

Doubled signal (SNR = 2) leads to doubled inserted trend error. The remaining trend error is unchanged.

Halved q

Inserted and remaining errors are independent.

The inserted error is determined by the break (signal) variance.

The remaining error is determined by the noise variance.

Dipdoc Seminar – 12. November 2015 s_x² = 0.265

s_y² = 0.563

Remaining

Inserted

Remaining

SNR = ½

s_x² = 1.060 s_y² = 0.141 SNR = 2

q = 1.46 (0.73)

q = 0.365 (0.73)

(19)

Improvement ratio (2/2)

Further parameters (besides SNR) that may also affect q are:

break and station number.

It shows:

The ratio does mainly depend on break number and not on station number.

For 6 breaks the ratio is about 1.

Break number

Station number

0.5 1.0 1.5

(20)

Preliminary conclusion II

Does the correction act neutrally if no correction is necessary?

Yes, depending on SNR,

for SNR=1, break number=6, length=100

the data is neither upgraded nor downgraded.

But,

the obtained homogenized data is mutually dependent.

Standard statistical techniques (using data independency) cannot be applied.

All variances are underestimated.

(21)

And IF there is a trend error?

To simulate this, a fourth signal is added:

The heights of the inhomogeneities are shifted by DI upward or downward

depending on the middlemost year of the subperiod j_m.

Due to boundary effects the mean introduced trend error is decreased from 1 (as naively expected) to 0.873 K/cty



Year

Mean insertedDI

(22)

Non-zero mean trend error

The scatter is conserved, compared to zero mean trend error.

The data cloud is shifted as a whole to the right.

The uncertainty remains, but the mean trend error is well corrected.

x_mean = 0.873 y_mean = 0.010

(23)

Including break position errors

Question:

What happens, if the break

positions have errors and are not perfectly known (as in reality).

Simulation:

Scatter the correct positions by adding noise with standard deviation of 2 years.

Result:

Only 80% of the trend error is corrected, 20% remains.

x_mean = 0.873 y_mean = 0.161

(24)

Conclusion

If the original data contains no trend error (if the inhomogeneities have by chance no overall effect) the trend error for individual networks is (under realistic conditions) not improved.

However, mutually dependent data results from homogenization.

This is a disadvantage.

Mean trend errors (due to inhomogeneities) are corrected perfectly, if the break positions are known perfectly.

For realistic position errors (s = 2 years) the trend error is only partly corrected, 20% remains.