The break signal in climate records:Random walk or random deviations?

(1)

The break signal in climate records:

Random walk or random deviations?

Ralf Lindau

(2)

Break signal

Climate records are affected by

breaks resulting from relocations or changes in the measuring

techniques.

For the detection, differences of neighboring stations are considered to reduce the dominating natural variance.

Homogenization algorithms identify breaks by searching for the

maximum external variance (explained by the jumps).

(3)

Benchmark datasets

Benchmarking data sets are used to assess the skill of homogenization algorithms.

These are artificial data sets with known breaks so that an evaluation of the algorithms is possible.

However, benchmark datasets should reflect as much as possible the statistical properties of real data .

An important question is how to model the breaks:

1. As free random walk (Brownian motion)

2. As random deviation from a fixed level (random noise)

Dipdoc Seminar – 30. May 2016

(4)

Conceptual model

Same signal, two approaches:

Which of the two DT is assumed to be an independent random variable?

The deviations or

the jumps?

Depending on our choice

different statistical properties of break signal will result.

Random deviations

Brownian motion

(5)

Different effects of identical s

Difference:

The introduced break variance of random deviations is larger by a factor of 2 compared to Brownian motion.

Reason:

All jumps are created by the sum of two random numbers, while it is only one in case of Brownian motion.

Preliminary:

Random deviations

Brownian motion

(6)

Linearly growing variance of BM

The variance of a Brownian motion grows linearly in time.

At the end of a BM time series the variance is:

Var = k s² with k: number of breaks and s²: break variance

The average variance (over time) is Var_BM = k/2 s²

For RD the variance is (shown before):

Var_RD = 2 s²

k is in the order of 5, for difference time series twice: 10.

Thus k/4 is in the order of 2.5.

Brownian motion created by the same s is much easier to detect.



(7)

Which type is more realistic?

There are indications for both of the two break types:

For random deviations:

Relocations are bound to fixed position.

Stations have geographical names and their positions are not free to fluctuate away.

For random walk:

Changes in measuring techniques can be seen as elimination of error sources one after the other.

Ideal case: Today most errors are eliminated.

Then the break signal can be seen as Brownian motion backward in time.

(8)

Different “schools”

For a long time we were not aware that there are these two approaches.

Williams et al. (2012) modelled random walk.

Venema et al. (2012) modelled random deviations.

Only the standard deviations applied were communicated.

But these are not comparable for RD and BM.

(9)

Platforms & Stairs

Venema et al. (2012) analyzed the statistics of the retrieved signal to decide whether breaks are BM or RD type.

Platforms Stairs

p (RD) = 0.67 p (actual) = 0.59 p (BM) = 0.50

But, the result was hardly significant due to the small number.

And (more important):

The result is dependent on the performance of the homogenization algorithm.

T₃ T₁

T₂ T₃

T₁ T₂

(10)

Platforms are difficult to detect

Running a homogenization algorithm with artificial pure RD data results in 0.62 – 0.64 platform frequency ( < 0.67 ).

In the retrieved signal, the platforms are underestimated.

The detected frequency is not suited as independent indication parameter to distinguish RD from BM.

Therefore, it would be convenient to be independent from the retrieved break signal and instead able to derive break parameters directly from the data.

(11)

Two superimposed signals

We assume that the climate time series consists of two superimposed signals:

Inhomogeneities and noise

Each yearly value can be thought as the sum of two random numbers, e_b and e_n, where e_b depends on segment number S, which is defined as the number of breaks lying temporally behind.



(12)

Random deviation breaks

In case of random deviation breaks we calculate the

Lag-covariance C(L):

For internal pairs E(C(L)) = 0 For external pairs E(C(L)) = s_b²



(13)

Probability of internal pairs



Probability of a specific year to belong to segment of length l:

Probability of a specific year to have sufficient spacing to the next break:

Probability of internal pairs is the sum over all length of the product.

The probability for internal pairs increase with segment length l and decrease with time lag L.

(14)

Probability of internal pairs



The long version of the product :

By some purely arithmetic transformations we get:

By some further approximations we get:

(15)

Lag covariance for RD

The covariance is an

exponential function of the time lag.

C(L) = a exp (-bL) break

a = s_b² strength s_b b = k/(n-k) number k

As byproduct we have a nice method to retrieve also

strength and number of breaks directly from the data.

Input:

s_b = 1.000 k = 5.000 Output:

s_b = 1.000 k = 4.984

(16)

Brownian motion type

For Brownian motion type breaks the covariance depends only on the segment number of the earlier of the two years , because they have all random numbers e_b constituting the break signal at x(i) in common.

The segment number is a stochastic variable growing linearly in time:

Consequently, also the covariance grows linearly with time:



(17)

Time dependent Cov for BM

The covariance is a linear function in time.

C(i) = a i + b

a = k/(n-1) s_b² b = ( 1 - k/(n-1)) s_b²

Input:

s_b = 1.000 k = 5.000 Output:

s_b = 1.005 k = 4.920

(18)

Conclusion

Brownian motion and random deviation break types can be distinguished by calculating:

1. Lag covariance C(L)

2. Time dependent covariance C(i)

For Random deviations C(L) is decreasing with L.

For Brownian motion C(i) is increasing with j.

The two other combinations remain constant.

As byproduct we get an estimate for break size and number without running a full homogenization algorithm.

(19)

Platforms & Stairs

Venema et al. (2012) analyzed the statistics of the retrieved signal to decide whether breaks are BM or RD type.

They distinguish platforms:

from stairs:

T₃ T₁

T₂

T₃

T₁

T₂

(20)

Platform probability for RD

For RD break types T₁, T₂, T₃ are iid random variables (not the case for BM).

There are 6 possibilities of rank order, which all have the same probability:

T₁ < T₂ < T₃ T₁ < T₃ < T₂ T₂ < T₁ < T₃ T₂ < T₃ < T₁ T₃ < T₁ < T₂ T₃ < T₂ < T₁

Upward and downward stairs have both the probability 1/6.

Every other combination is a

platform. (Either T₂ is the smallest or T₂ is the largest element of the

triple.)

Downward stair Upward stair

For RD break types the probability of platforms is 2/3.