2. Closed-Form Registration 13
2.5. Evaluation
2.5.1. Accuracy
2.5. Evaluation
number of correspondences
4 10 30 100 300 1000 3000
rotation error (degrees)
10-3 10-2 10-1
rotation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
4 10 30 100 300 1000 3000
absolute translation error
10-4 10-3
10-2 translation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
4 10 30 100 300 1000 3000
relative scale error
10-4
10-3 scale using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
8 20 60 200 600 2000 6000
rotation error (degrees)
10-3 10-2
10-1 rotation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
8 20 60 200 600 2000 6000
absolute translation error
10-4 10-3
10-2 translation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
8 20 60 200 600 2000 6000
relative scale error
10-4
10-3 scale using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
3 7 20 60 200 600 2000
rotation error (degrees)
10-3 10-2 10-1
rotation using points gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
number of correspondences
3 7 20 60 200 600 2000
absolute translation error
10-4 10-3
10-2 translation using points gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
number of correspondences
3 7 20 60 200 600 2000
relative scale error
10-4
10-3 scale using points gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
Figure 2.3.: Mean errors of the estimated rotation, translation, and scale for general geometric configurations with varying numbers of input correspondences.
evaluations. In each run, we computed the mean of the singular values of the covariance matrix of the pointsxk
and the covariance of the noise. The’relative noise level’was defined as the square root of their ratios. Again, we evaluated general geometric configurations of the lines, planes, and points.
Fig. 2.4shows the results. Again, it can be seen that DLS/gDLS and UPnP do not succeed in correctly determining the registration parameters from point-to-point correspondences whereas our solvers do. For the other cases, we see that gDLS has slightly higher mean errors for very low noise levels.
2.5.1.2. Fixed Scale and the Inhomogeneous Case - General Configuration
We also analyzed the benefits of the GAPS and UPnP rotation solvers, because these are the only ones that accept the inhomogeneous matrixMi∈R10×10as in Eq. 2.15. The other two algorithms can only handle 9×9 input matrices. Again, we performed the evaluation on general geometric configurations by varying the number of input correspondences. The relative noise ratio was set to 0.0001. For our own hGAPS solver [WK17] and the
relative noise level 10-1510-1310-1110-910-710-510-310-1101
rotation error (degrees)
10-12 10-10 10-8 10-6 10-4 10-2 100
102 rotation using lines
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
absolute translation error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 translation using lines
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
relative scale error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 scale using lines
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
rotation error (degrees)
10-12 10-10 10-8 10-6 10-4 10-2 100
102 rotation using planes
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
absolute translation error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 translation using planes
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
relative scale error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 scale using planes
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
relative noise level 10-1510-1310-1110-910-710-510-310-1101
rotation error (degrees)
10-12 10-10 10-8 10-6 10-4 10-2 100
102 rotation using points
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
relative noise level 10-1510-1310-1110-910-710-510-310-1101
absolute translation error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 translation using points
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
relative noise level 10-1510-1310-1110-910-710-510-310-1101
relative scale error
10-15 10-13 10-11 10-9 10-7 10-5 10-3 10-1
101 scale using points
gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Umeyama
Figure 2.4.: Mean errors of the estimated rotation, translation, and scale for scaled Euclidean registration with 50 correspondences in general geometric configurations with varying noise. The relative noise level refers to the ratio between the standard deviations of the noise and the centered pointsxk.
DLS/gDLS solver, we computed the homogeneous matrixMh as in Sec. 2.3.2. Note that this corresponds to leaving the scale as free parameter.
Fig.2.5shows the results. Using GAPS and UPnP onMileads to more accurate results as expected, because the problem is modeled more appropriately. A rather surprising result is that this only affects the translation, whereas the rotation estimates are equally accurate for all solvers. Note also that only GAPS and UPnP can solve the minimal cases of six point-to-plane or three point-to-line correspondences.
Fig. 2.5does not include our evaluations when point-to-point correspondences are used exclusively. The matrixMiwill then have a very special block-diagonal structure (except the last row and columns) for which also our inhomogeneous solver GAPS has difficulties. At this point we note, however, that as soon as any mixture of correspondences is used, all of the considered solvers will succeed. Using only point-to-point correspondences remains a special case for which existing algorithms [AHB87,Hor87,HHN88,Ume91] are better suited, also regarding the runtime behavior.
2.5. Evaluation
number of correspondences
3 4 6 10 18 34 66 130 260 520
rotation error (degrees)
10-4 10-2 100
rotation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
3 4 6 10 18 34 66 130 260 520
absolute translation error
10-6 10-4
10-2 translation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
67 11 20 32 65 130 260 520 1040
rotation error (degrees)
10-4 10-2 100
rotation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
67 11 20 32 65 130 260 520 1040
absolute translation error
10-6 10-4
10-2 translation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
Figure 2.5.: Mean errors of the estimated rotation and translation for Euclidean registration with known and fixed scale (inhomogeneous case) versus number of used correspondences. The relative noise level was set to 0.0001. Only UPnP and GAPS can handle the minimal cases, i.e. 3 line correspondences or 6 plane correspondences.
2.5.1.3. The Degenerate Central Case for Lines and the PnP Problem
As we mentioned in Sec. 2.3.4, the central case, i.e. when all 3D lines have a common point of intersection, can be considered as a degenerate situation because the scale cannot be computed. This special case has a direct connection to the PnP problem, where all lines intersect at the camera center. For this reason - and also because the PnP problem has a very high relevance in computer vision - we use the PnP evaluation framework of Zheng et al. [ZKS∗13]12and includedUPnP,gDLS/DLSand our own solversGAPS/hGAPSinto it. Again, we replaced the linear parameter elimination of the algorithms by ourorthogonal complement based version (marked by
’(OC)’).Newton (OC)refers to the direct second order (Newton) based minimization of Eq.2.10.
For comparison we also include the well-known algorithmsLHM[LHM00],EPnP[LMNF09], andOPnP [ZKS∗13]. LMrefers to the solution obtained by minimizing the reprojection error in the image plane with Levenberg-Marquardt iterations, which represents the baseline for PnP problems.
The evaluation setup is as follows. Random pointsx0kin the camera coordinate frame are created inside the cube[(−2,−2,4)×(2,2,8)]and corresponding 2D image measurementsqk∈R2are generated by perspective division and by adding pixel noise with varying variance. Next, with a random ground-truth rotationRGT and translationtGT the corresponding reference pointsxk are obtained. The image points,qk, and the reference points,xk, are used as input for all PnP algorithms.
Fig. 2.6shows the results. It can be seen that the modified algorithms (OC) compare favorably with the state-of-the-art even for the minimal cases of three correspondences. Perhaps more important is the fact that
12The evaluation framework of Zheng and a collection of the most prominent PnP solvers are available at https://sites.google.com/site/yinqiangzheng/.
Number of Points
3 4 5 6 7 8 9 10 11 12 13 14 15
Rotation Error (degrees)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Mean Rotation Error
LHM EPnP+GN OPnP DLS/gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Newton (OC) LM
Number of Points
3 4 5 6 7 8 9 10 11 12 13 14 15
Translation Error (%)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Mean Translation Error
LHM EPnP+GN OPnP DLS/gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Newton (OC) LM
Gaussian Image Noise (pixels)
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Rotation Error (degrees)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Mean Rotation Error
LHM EPnP+GN OPnP DLS/gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Newton (OC) LM
Gaussian Image Noise (pixels)
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Translation Error (%)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Mean Translation Error
LHM EPnP+GN OPnP DLS/gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC) Newton (OC) LM
Figure 2.6.: Accuracy for the central PnP problem. Top: error of rotation and translation as a function of the number of input correspondences (pixel noise is set toσ=1.0). Bottom: error with respect to pixel noise using 10 input correspondences.
the PnP problem - as being a degenerate case in the scaled Euclidean registration context - is handled properly by the SVD-based pseudo-inverse computation from Sec. 2.3.2, as all algorithms (UPnP, DLS/gDLS and our solvers) are supplied with the homogeneous matrixMh from Eq. 2.9. We note again that linear parameter elimination inside the original gDLS algorithm [SFHT14] cannot be used here, because it will attempt to invert a singular matrix to obtain the pseudo-inverse ˜A†. Likewise, the linear parameter elimination technique from the original DLS algorithm [HR11] is not applicable whenever the scale needs to be estimated in general geometric configurations.
2.5.1.4. The Degenerate Case of Parallel Lines and Planes
We evaluated another degenerate case, where all planes and lines are parallel in one common directionv0. Then, the translation is ambiguous up to this common vector, so anyt=t0+γv0represents a valid solution for arbitrary γ. Again, the linear parameter elimination techniques inside the original algorithms [HR11,SFHT14,KLS14]
will fail, because they attempt to invert a singular matrixA˜TA˜ (see Sec. 2.3.3). By contrast, our SVD-based computation of the pseudo-inverse will handle this degeneracy correctly.
The evaluation was similar to the general case (Sec.2.5.1.1) except that the lines and planes were constrained to be parallel to some random directionv0. For the translation error we projected the error vector between the translation offsetst0 returned by the algorithms and the ground-truth translationtGT onto the orthogonal complement ofv0, i.e.errt=k(I−v0vT0)(tGT−t0)k.
2.5. Evaluation
number of correspondences
3 4 10 30 100 300 1000 3000
rotation error (degrees)
10-3 10-2 10-1
100 rotation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
3 4 10 30 100 300 1000 3000
absolute translation error
10-4 10-3 10-2
translation using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
3 4 10 30 100 300 1000 3000
relative scale error
10-4
10-3 scale using lines gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
67 20 60 200 600 2000 6000
rotation error (degrees)
10-3 10-2 10-1
100 rotation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
67 20 60 200 600 2000 6000
absolute translation error
10-4 10-3 10-2
translation using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
number of correspondences
67 20 60 200 600 2000 6000
relative scale error
10-4
10-3 scale using planes gDLS (OC) UPnP (OC) hGAPS (OC) GAPS (OC)
Figure 2.7.: Mean errors of the estimated rotation, translation, and scale for scaled Euclidean registration with the lines and planes being parallel to one common direction.
In Fig. 2.7the results are depicted. It can be seen that all algorithms succeed in estimating correct results with similar accuracy. We also note that this is possible even for three point-to-line and six point-to-plane correspondences, despite the fact that we also solve for the scale. This is because once again only six DoF in total need to be estimated.