The tools developed in the thesis have become part of an early cognitive vision framework [Pugeault et al., 2006]
mainly developed in the European ECOVISION project that otherwise makes use of only edge-like structures. This work has been supported by the European Drivsco [Drivsco, 2007] and PACO-PLUS [PACO-PLUS, 2007] projects.
8.2 Outlook
Depth extraction makes use of monocular or multi-view depth cues in order to recover the third dimen-sion from a set of images. This ill-posed inverse problem is challenging since each depth cue bears ambiguities, or is reliable only for certain types of scenes. Moreover, how the different cues should be integrated or fused together is itself a difficult and an open question because different cues might carry conflicting interpretations of a scene. Because of these reasons, current computer vision algorithms or applications are limited to only certain types of scenes and are not general.
The experiments with babies suggest that depth cues which are not directly based on correspondences evolve rather late in the development of the human visual system. For example, pictorial depth cues are made use of only after approximately 6 months [Kellman and Arterberry, 1998]. This indicates that ex-perience may play an important role in the development of these cues,i.e., that we have to understand depth perception as a statistical learning problem [Knill and Richards, 1996, Purves and Lotto, 2002, Rao et al., 2002], where attention and the utilization of statistical regularities play an important role.
In view of the above mentioned problems, the following need to be tackled in order to build a biologically-motivated fully-functional machine vision system:
1. Which visual abilities and depth cues are humansequippedwith at birth, and what abilities and cues do welearnand in what sequence, if there is an ordering between different depth cues? A more important question is, of course, “how do we do it?”
2. How do we experiment with the world to (1) build representations of objects, to (2) exploit the statistical regularities of natural scenes, and to (3) integrate the information from other senses such as haptic and sound.
The current thesis is relevant to these questions, and any extension in view of these questions are valuable contributions to the field. To name a few:
• Using vision to build a database of object models, and building mechanisms for providing context to the visual processing through attention or other feedback mechanisms.
Appendix A
Algorithmic Details of Intrinsic Dimensionality
This chapter provides the algorithmic details of intrinsic dimensionality which can be used to implement it. There are two different ways to computeiD:
1. As proposed in [Felsberg and Kr¨uger, 2003, Kr¨uger and Felsberg, 2003], which computes the ori-gin and line variance explicitly to compute the coordinates of a signal in theiDtriangle.
2. As proposed in [Felsberg et al., 2007a], which implicitly computes the origin and line variance by mapping the magnitude and orientation of signals to a cone.
The first approach is slower than the second one; therefore, the second method is used in this the-sis. For this reason, only the second approach is detailed here; the interested reader is directed to [Felsberg and Kr¨uger, 2003, Kr¨uger and Felsberg, 2003] for the first approach.
In the cone model, the coordinates are constructed from the magnitude m and the orientationθ.
Averaging the coordinates locally inside the cone implicitly computes the line variance.
The overall algorithm for an image pointu=(u1,u2) is as follows:
1. Gradient information: Extract the (complex) gradient datag=m(u) exp(iθ(u)),mbeing the mag-nitude andθthe orientation at pixelu.
128
1
i0D i2D
1
Figure A.1: The cone constructed from the magnitude and the orientation of signals. Taken from [Felsberg et al., 2007a].
2. Magnitude normalization and double angle representation: Convert the gradient data to soft-thresholded double angle representationd(u)=s1(m(u)) exp(i2θ(u)),s1(·) being the soft threshold function.
3. Cone representation: Set the cone coordinatesc(u)= (c1,c2,c3) =(|d|,Re{d},Im{d}). The cone is exemplified in figure A.1. For different real examples, the cone coordinates of the points in the patch are shown in figure A.2.
4. Averaging: Average the cone coordinates locally: c0(u) =P
iwic(i) whereiruns over the neigh-borhood ofu, andwiis the two dimensional Gaussian with appropriateσ. In our implementation, sigmais set to be√
l/4,lbeing the patchsize.
5. Triangle representation:(x4(u),y4(u))=(c01,q
(c02)2+(c03)2)
6. Normalization of y values (optional): Set ( ˆx(u),y(u))ˆ = (x4(u),s2(x4(u),y4(u))) where s2 is a monotonic transform to spread the data more uniformly, mainly for the purpose of visualization.
7. Barycentric coordinates:Extract barycentric coordinates from ( ˆx,y) according to equation (2.1).ˆ The soft-threshold functions1and monotonic transforms2to spread theyvalues are as follows:
APPENDIX A. ALGORITHMIC DETAILS OF INTRINSIC DIMENSIONALITY 130
• The soft-threshold function s1 : R+ → [0,1) :m 7→ s1(m) maps the unbounded magnitudes to a bounded interval. Basically, we can make use of any activation function used in neural networks (see,e.g., [Bishop, 1995]) such as the logarithmic sigmoid function. However, we must adjust one constant in order to get an appropriate mapping. Our choice is:
s1(m)=tanh(αm), (A.1)
whereαis a parameter controlling the dynamics ofm(figure A.3(a)). This parameter can be esti-mated such that the empirical distribution of|d|follows as good as possible a particular predefined distribution,e.g., a uniform distribution. For the following experiments, we computedαsuch that the mean ¯mof the magnitudes is mapped to the empirically chosen value 0.35:α=atanh(0.35)/m.¯ An alternative soft-threshold function can be normalization by the maximum magnitude in the image:
s1(m)= m mmax
, (A.2)
wheremmaxis the maximum magnitude in the image.
• We apply the mappings2to obtain ˆyfor ensuring a reasonable spread of representations between i1D and i2D,i.e., we want to ensure that corners are mapped close to the i2D vertex while edge–like structures are mapped close to the i1D vertex. This is mainly for visualization and interpretation purposes and in practice one can omit this mapping. The subsequent illustrations were generated with the mappings2(x0,y0)=x0(y0/x0)βwithβ=5.
The normalization functionyβforβ=5 is shown in figure A.3(b).
b)
c)
d)
e)
f)
Figure A.2: Illustration of how the points in some image patches taken from real scenes map to the triangle and the cone. The patches are illustrated in the right-most column. The color of the points inside the triangle and the cone encode different orientations, whose values can be accessed using the color-bars.
APPENDIX A. ALGORITHMIC DETAILS OF INTRINSIC DIMENSIONALITY 132
0 5 10
0 0.5 1
m
tanh( α m)
α =1 α =4 α=1/4
(a) Soft-threshold function.
0 0.5 1
0 0.5 1
y
y
β(b) Spread function.
Figure A.3: (a) Functions1(m)=tanh(αm) for soft-thresholding the magnitude with different values of α. (b) Spread function,yβforβ=5.
This chapter describes the perceptual grouping relations that are used to group 2D primitives into con-tours. As the primitives are local contour descriptors, scene contours are expected to be represented by strings of primitives that are locally close to collinear. In the following, we will explain methods for grouping 2D primitives into contours.
In the following,c(li,j) refers to the likelihood for two primitivesπiandπjto belinked: i.e. grouped to describe the same contour.
Position and orientation of primitives are intrinsically related. As primitives represent local edge estimators, their positions are points along the edge, and their orientation can be seen as a tangent at such a point. The estimated likelihood of the contour described by those tangents is based upon the assumption that simpler curves are more likely to describe the scene structures, and highly jagged contours are more likely to be manifestations of erroneous and noisy data.
Therefore, for a pair of primitives πi andπj in imageI, we can formulate the likelihood for these primitives to describe the same contour as a combination of three basic constraints on their relative position and orientation — see [Pugeault et al., 2006].
B.1 Proximity (c
p[l
i,j])
A contour is more likely if it is described by a dense population of primitives. Large holes in the primitive description of the contour is an indication that there are two contours which are collinear yet different.
133