We have seen in the last section that the position of 4 points on a plane allows us to uniquely specify the position of each additional point on that plane indepen-dently of any projective transformation applied to that plane. These projective coordinates are, however, not a particularly intuitive way to describe most image features, and it is therefore often desirable to find some quasi-Euclidean represen-tation instead. This can easily be done if the entire object plane is transformed in such a way that the projective coordinates’ four reference-points (or lines) are transformed onto four points (or lines) in a fixed position, the so called canonical frame.
2.7.1 Motivation
Three possible uses for canonical frames are described below; all have been used within this thesis.
Verification and Recognition: Canonical frames have traditionally been used for verification and recognition purposes [127, 129][9], as they allow for the direct comparison of features within a quasi-Euclidean framework. Possible comparisons range from direct comparison of pixel-positions to the
calcula-2.7.2 Commonly used Frames 45
tion of higher order features of non-algebraic curves, where they considerably reduce the number of derivatives required (from up to seventh order to only first or second order [158]). It is noteworthy that all frames are mathemat-ically equivalent in the absence of errors. This is however not the case for practical applications, as we will see in, e. g., Section 4.4.2.
Recognition within a canonical frame can be implemented as simple as a comparison with different models, and as complicated as the extraction of invariants or the application of an index-function. Examples for both uses are given in Section 7.
Backprojection: Normally, image pixels or low-level features like edgels or lines are projected into the canonical frame in order to test a hypothesis. If instead the known contour (or other features) of a hypothesis are projected from the frame back into the image we talk about backprojection. This is often done for verification directly in the image, recognising the fact that in practice, and in the presence of errors, all canonical frames arenotequal.
Another use is the prediction of additional image features from a hypothesis, which, if found, would lend additional credibility to the particular hypothesis.
This is used in Section 5.
Fitting: Using the canonical frame to fit higher order structures to low-level fea-tures (mainly edgels) allows us to enforce additional constraints not easily enforced within the image. The basic idea here is to find the transformation from the image into the canonical frame which minimises the error between the transformed image features and a structure in the canonical frame. It is then possible to invert the transformation in order to calculate the structure’s position within the image.
This approach is of course only useful in the presence of errors (fitting would not be required otherwise) and therefore discussed in more detail in Sec-tion 4.4.2, where it is used.
2.7.2 Commonly used Frames
In the case of 4 points, commonly used frames include e. g. the unit square (0,0,1)T,(1,0,1)T,(0,1,1)T,(1,1,1)T (2.47) and the triangle of reference and unit point
(1,0,0)T,(0,1,0)T,(0,0,1)T,(1,1,1)T , (2.48)
46 Canonical Frames
a a
b
c
Figure 2.16: Some distinguished points: a. Bitangent-points, b. inflection, c.
casttangent-point.
where the first three points are called the vertices of the triangle of reference, while the last point is called the unit point [103, 138]. Other canonical frames are often based on the object’s appearance in the Euclidean world. Figure 2.15 shows an example where the unit square is also the object’s natural frame. In the absence of measurement errors, all canonical frames are of course mathematically equivalent.
2.7.3 Commonly used Image Features
A canonical frame describes a particular instance of a projectively transformed plane (or, more general, space). In order to define this particular instance, it is necessary to determine the projective transformation between the original space and its representation within the canonical frame. In the case of planar structures as discussed here, this transformation has 8 degrees of freedom, and it is clear that the position within the image and frame of any structure which fixes at least 8 degrees of freedoms can be used to describe the transformation between the two planes. In practice, however, this structure will nearly exclusively be made up of points and lines. The reason for this is that the use of points and lines leads to a set of linear equations (compare Section 2.7.4), while higher order algebraic structures generally do not. Also, points (edgels) and lines (straight edgel chains) are the most basic image features found.
Three different types of points are commonly used in computer vision:
1. Corners of grey-level discontinuities as found by corner detectors.
2. Intersections of higher-order algebraic features, usually lines fit to grey-level discontinuities.
3. Distinguished points. These are points on a curve which are easily distin-guishable from all the other points on the curve by order of contact, which
2.7.4 Calculation of Canonical Frames 47
is a projective invariant (compare Table 2.1). Examples are points of bitan-gency or inflections (see Figure 2.16), which are easily identified using only up to first or second order derivatives.
Once a point and a curve are identified it is often easy to create a number of addi-tional distinguished points. Examples are rays cast from one distinguished point and tangent to the curve at a second point, so called casttangents. This second point, the casttangent-point, is another distinguished point. Another example is the intersection of a line through two distinguished points and the curve (compare Item 2), again generating extra distinguished points (although of course collinear with the first two).
2.7.4 Calculation of Canonical Frames
Finding the transformation A ∈ IR3×3 such that N ≥ 4 image points Xi are mapped onto the corresponding frame-pointsxiis easily done by solving the equa-tion
AX=xk (2.49)
forA. X,x∈ IR3×N are two matrices, where each column represents one image or frame point respectively, andk∈IRN×N is a diagonal matrix of scale-factors accounting for the fact that the overall scale of each homogeneous coordinate can be chosen arbitrarily. The resulting equations are of the form:
a11Xi+a12Yi+a13Zi = kixi
a21Xi+a22Yi+a23Zi = kiyi
a31Xi+a32Yi+a33Zi = kizi
(2.50)
and it is always possible to eliminateki. We assume that w. l. o. g. zi = 1 (this is obviously not the case for the triangle of reference (2.48), but the underlying principle is the same) and get
a11Xi+a12Yi+a13Zi = a31Xixi+a32Yixi+a33Zixi
a21Xi+a22Yi+a23Zi = a31Xiyi+a32Yiyi+a33Ziyi. (2.51)
48 Canonical Frames
Furthermore, since the overall scale ofAis arbitrary we can choose w. l. o. g.a33= 1 and get, forN= 4
The existence of this linear system ensures the existence and uniqueness of a solution forA given four point correspondences, provided that no three of the points are collinear[103].
A more elegant implementation would use a singular value decomposition (SVD) approach to calculate the eigenvector to the smallest (zero!) eigenvalue of the system
This way it is not necessary to single out any particularaij = 1. In addition, such an approach will also work in the presence of errors, and givenN6= 4 point-pairs.
SVD is, however, computationally more expensive.
The same basic approach can be used when 4 lines are given, no three of which are coincident, solving forA−Tinstead (compare Equation (2.28)). Rearranging Equation (2.28) to readL = ATℓkit is even possible to combine the equations for points and lines into one system of equations; the equation for a line-pair
2.7.5 Semi-Frames 49
Closed form solutions for combinations with higher-order algebraic forms, e. g.
conics, are unfortunately not as easy to find.
2.7.5 Semi-Frames
So far, we have always assumed that a canonical frame fixes all of a planar trans-formation’s 8 degrees of freedom. For many applications, however, this is not necessary. Imagine a frame which solely consists of a set of horizontal lines with fixed distance from the origin. Neither an anisotropic scaling factor inxdirection, nor any skew nor translation in that direction would change the appearance of the frame. It would therefore be sufficient to solve for a 5 degrees of freedom transformation and arbitrarily fix the remaining 3 degrees. This could, e. g., look like