The following gives a short introduction into the basic concepts of statistics, see [81, 100, 145] for more information.
3.2.1 Error Types
This entire chapter would not have been necessary if it were not for the fact that any observation will always contain errors. These errors are traditionally grouped into three categories: random errors, systematic errors, and blunders.
Blunders — or outliers, as they are customarily called in computer vision — are gross errors generally due not to the observed process or variable, but to the observer. If at all possible, they should be removed from the set of observations. How to reliably classify outliers is unfortunately still an open question, the reader is referred to [46, 67, 140, 151] for examples from nearly 20 years of outlier removal in computer vision. Particularly en vogue is cur-rently once more a method called RANSAC — Random Sample Consensus
— which was introduced in 1981 by Fischler and Bolles [46].
Outliers are ignored in the following unless otherwise stated.
Systematic errors — orsystematic effects, as they are commonly named in the recent literature — are not really errors in the observations, but rather in the underlying model. It is therefore usually possible to remove or avoid systematic effects if an appropriate model is chosen, and part of Section 4 is dedicated to the process of model-selection. An example of systematic effects often encountered in computer vision are radial distortions of the image due to an imperfect lens, see Section 4.2.1. It is well known how to model this effect (usually by an odd polynomial of the distance to the principal point, compare e. g. [139]), and therefore easy to account for it. This is usually not done by incorporating the model of the radial distortion into that of (perspective) projection — which would lead to rather intractable equations
— but by correcting the observations for this particular effect. In computer vision, such corrections are often part of (partial) camera calibration.
Random errors are the only kind of effects with which traditional statistics is concerned, although the use of the term “error” is deprecated in modern literature, and the term statistical properties used instead. This captures the fact that from a statistical standpoint, observations can be considered
60 Basic Concepts in Statistics
samples of an unknown probability distribution of a random variable. Dis-crepancies between several observations are therefore not due to errors, but simply serve to describe the particular probability distribution. It is statis-tics’ task to gain as much information as possible about this distribution from the observations.
So how can we describe the properties of our unknown probability distribution?
A very concise description can often be given by the use of moments, as described in the next section.
3.2.2 Mean and Central Moments
Probability distributions can often be described in terms of their mean and central moments. The population mean of a random variablex, also called first moment or expectation, is denoted byE(x) and is defined (if it exists) as the average value µxof the variable over all possible values, weighted by their respective probabilities Px∈IRor probability density function2px∈IR, it is
E(x) =µx= Xn
i=1
xiPx(xi) (3.1)
or
E(x) =µx= Z ∞
−∞
x px(x) dx (3.2)
in the continuous case. Given two random variablesxandyand three constants a, b, c, the following rules hold [100]:
E(E(x)) =E(x) (3.3)
E(x+y) =E(x) +E(y) (3.4)
E(c) =c (3.5)
E(c·x) =c·E(x) (3.6)
=⇒ E(a·x+b) =a·E(x) +b. (3.7) Ifxandyare independent random variables, it is also true that
E(x·y) =E(x)·E(y). (3.8)
Note, however, that, in general,E(x2)6= (E(x))2.
2In the following denoted by pdf.
3.2.2 Mean and Central Moments 61
Central moments, which can be used to describe most pdfs, are expectations with respect to the mean, where the kthcentral moment is defined as
mk=E
(x−E(x))k
. (3.9)
One particularly important central moment is the second moment or varianceσx2. It is:
σx2=m2=E
(x−E(x))2
=E
(x−µx)2
(3.10)
=E x2−2xµx+µ2x
(3.11)
=E x2
−2µxE(x) +µ2x (3.12)
=E x2
−µ2x. (3.13)
The variance’s positive square rootσ >0 is called standard deviation. Note that Equation (3.13) can easily lead to numeric problems for big values ofµ2x. The equations corresponding to Equations (3.4)–(3.8) are:
σ2x+y=σx2+σ2y (3.14)
σ2c= 0 (3.15)
σc2·x=c2·σx2 (3.16)
=⇒ σa2·x+b=a2·σx2. (3.17) Related to the concept of the variance is that of the cofactorq, which could be viewed as a relative variance. It is
q2x= σ2x
σ20 (3.18)
for a possibly unknown value ofσ20, the reference variance.
The Equations (3.2), (3.10), and (3.13) can only be used if the pdfpx is already known. In order to estimate the pdf from observations alone, we have to approx-imate the population mean and population variance (and possibly higher order moments) by the sample mean and sample variance. GivenN measurementsxi, (i= 1. . . N) the sample or empirical meanxis defined as the arithmetic mean:
x= 1 N
XN
i=1
xi. (3.19)
62 Basic Concepts in Statistics
It isE(x) =µx. The sample variance is defined as s2x= 1
N−1 XN
i=1
(xi−x)2 (3.20)
and it isE(s2x) =σ2x. Higher order moments can be approximated similarly.
3.2.3 Normal Distribution
The most important probability distribution, and one uniquely defined by mean and variance, is the normal or Gaussian distribution
N(µx, σx2) = 1 p2πσ2xe−12
(x−µx)2 σ2
x . (3.21)
3.2.4 Multidimensional Extension
The above can easily be extended for multi-dimensional random variables. If x∈IRnis a vector ofN(not necessarily independent) random variables, so is the expectation simply
E(x) =µx= XN
i=1
xiPx(xi) (3.22)
or
E(x) =µx= Z ∞
−∞
xpx(x) dx (3.23)
in the continuous case, where the summation (or integration) can be performed separately for each vector-element. Note thatPx, px∈IR.
The central moments of order (k1+· · ·+kn) can be calculated as E
(x1−µx1)k1· · · · ·(xn−µxn)kn
. (3.24)
Of particular importance are again the second order central moments ofx∈IRn, which can be set up as all combinations between each two elements of the vector.
The result can be arranged as a matrixMxx∈IRn×n. This matrix is customarily called the matrix of second central moments, the variance-covariance matrix or
3.2.4 Multidimensional Extension 63
simply the covariance matrix. It is
Mxx=E((x−µx)(x−µx)T) =
mx1x1 mx1x2 · · · mx1xn
mx2x1 mx2x2 · · · mx2xn
... ... . .. ... mxnx1 mxnx2 · · · mxnxn
=
σ2x1 σx1x2 · · · σx1xn
σx2x1 σ2x2 · · · σx2xn
... ... . .. ... σxnx1 σxnx2 · · · σ2xn
=Σx. (3.25)
Note that this is a square symmetric matrix since
σxixj =E((xi−µxi)·(xj−µxj)) =E((xj−µxj)·(xi−µxi)) =σxjxi. (3.26) Using Equation (3.24) it is always possible to construct higher order central mo-ments. The equations corresponding to Equations (3.3)– (3.7) and (3.14)– (3.17) are (with random variablesx,y∈IRn, constant vectorsb,c∈IRmand a constant matrixA∈IRm×n):
E(E(x)) =E(x) (3.27)
E(x+y) =E(x) +E(y) (3.28)
E(c) =c (3.29)
E(Ax) =AE(x) (3.30)
=⇒ E(Ax+b) =AE(x) +b (3.31)
Σx+y=Σx+Σy (3.32)
Σc= 0 (3.33)
ΣAx=AΣxAT (3.34)
=⇒ ΣAx+b=AΣxAT. (3.35)
A cofactor matrix can be defined analogous to Equation (3.18), it is Qx= 1
σ20Σx. (3.36)
The sample meanx and sample covariance matrixSx are defined in analogy to
64 Error Propagation
Equations (3.19) and (3.20) as x= 1
N XN
i=1
xi (3.37)
Sx= 1 N−1
XN
i=1
(xi−x)(xi−x)T. (3.38)
Then-dimensional normal distribution is given by N(µx,Σx) = 1
p(2π)n|Σx| exp(−1
2(x−µx)TΣ−1x (x−µx)), (3.39) where|Σx|is the determinant ofΣx∈IRn×n. Additional distributions are given in Section 3.4 (theχ2-distribution, used for testing) and 3.5 (an adaption of the normal distribution to cyclic data), but the normal distribution is by far the most important distribution used in this thesis. Its theoretical and practical importance is due to thecentral limit theoremwhich states that the sumPn
i=1xi ofn inde-pendent random variablesx1, . . . , xnwill be asymptotically normally distributed asn→ ∞. Normal distributions are encountered very often in practical applica-tions; in particular, random variables that representindependent measurements in photogrammetry, geodesy, or surveying are often nearly normally distributed [100].
Another reason for the normal distribution’s prominence is its simple form which is completely described by mean and variance. This makes it particularly well suited for the propagation of statistical properties as described in the next section.