4. Edge-Based Tracking Methods 27
4.11. Evaluation of the Line Tracking Methods
Image sequences of several scenes were recorded and used as input for dierent tracking methods. The camera pose was initialized either manually or by detecting markers. In the rst experiment, an object in a static camera sequence of 949 frames was tracked with dierent approaches: single hypothesis (Ma), multiple hypotheses (Mb), single hypothesis with the adaptive method (Mc), and nally multiple hypotheses with the adaptive method (Md) as described in the algorithm of Section 4.9.
If method Ma is used, many sample points are assigned to false edges. False correspon-dences lead to an inexact pose estimate and eventually to the loss of the track. Using method Mb signicantly improved the tracking results, as the line model sticks much better on the object in the image sequences. However, when fast motion occurs, the estimated pose is always several frames behind the real camera pose. It also sometimes happens that the minimization process is stuck in a local minimum, thus not leading to the desired result. Method Mc, that uses only the most likely hypotheses with regard to the previous measurements, produces better results during large movements, but still has problems if other edges are near the line to be tracked. False correspondences lead to a bad adaptation of the edge properties and quickly cause the edge lter to learn wrong properties of the edge. If method Md is applied, the problems described above are avoided. The camera pose is estimated correctly during the whole sequence. Taking into account multiple hypotheses makes the edge lter much more likely to be updated with the properties of the correct edge. Therefore, the adapted state is more accurate and leads to a better result in nding the most probable gradient maximum. Figure 4.9 illustrates the results of the four tracking methods described above.
Since the computational complexity is higher for Md, it is slightly slower than the other ones. On a 2.8 GHz Pentium IV, it needs about 60 milliseconds for an iteration step with an image size of 640x480 pixels, whereas Mb takes about 50 milliseconds on average for the image sequence.
4.11. Evaluation of the Line Tracking Methods
(a) (b)
(c) (d)
Figure 4.9.: Results of the dierent tracking methods: (a) the single hypothesis method loses the track at frame 451, (b) the multiple hypotheses method fails at frame 428, (c) the adaptive method fails at frame 462, (d) the adaptive approach with multiple hypotheses stays on track during the whole sequence.
In another experiment, the indoor environment of an oce building was tracked with both Mc and Md (see Figure 4.10). Again, it can be observed that fast movements are handled much better by the adaptive algorithm. With the non-adaptive approach, tracking sometimes fails, because too many wrong 2D/3D-correspondences are used for the pose estimation.
Figure 4.13 shows the result of our adaptive line tracking algorithm in an industrial scenario. The camera path can be tracked successfully throughout the sequence as long as enough parts of the object are visible in the camera image, where lines in the manually created line model are available. Only when the camera turns away from the scene, the tracking fails.
To measure the run-time as a function of the number of sample points, another object was tracked from dierent distances. As seen in Section 4.10, the number of sample points depends on the length of the projected lines, and therefore on the distance between the camera and the tracked object. This means that an iteration step of the tracker needs more computational time when the object is close to the camera and less time when it is further away. Figure 4.11 shows an object tracked from dierent distances and a scatter
4. Edge-Based Tracking Methods
(a) (b)
Figure 4.10.: Tracking the oce sequence with the non-adaptive (a) and the adaptive method (b).
plot illustrating the run-time of an iteration step versus the number of used sample points.
The sample point density was chosen so that the distance between two adjacent search lines was at least 10 pixels. To increase the frame rate of the tracking, the sample point density can be decreased. However, if the sample point density is too small, the estimated 6 degrees of freedom of the camera start jittering. The tracking algorithm needs about 50 milliseconds on average for an iteration step, which means that it can run at a frame rate of about 20Hz.
If some parts of an object are occluded, it is still possible to estimate the camera pose (see Figure 4.12). Due to the robustness towards outliers of the Tukey estimator function, the estimated pose is correct even with a certain amount of outliers. If a higher occlusion proportion is expected, the Tukey constant c of equation (2.33) can be set to a lower value. The Tukey constant shall not be too small though, otherwise tracking results become unstable and start jittering. If there is very little or no occlusion, the Tukey constant can be set to a higher value.
The adaptive approach, which tries to maintain the visual appearance of a line control point, improved the tracking robustness by using only the most probable edge for every sample point during the rst minimization step, so that the pose estimation does not get stuck in local minima. After the rst minimization, the resulting6degrees of freedom lie around the desired minimum of the error function. The way through many local minima caused by multiple hypotheses is thereby avoided. A uniform distribution of sample points leads to a balanced set of 2D/3D correspondences and keeps the computation costs low.
Finally, using multiple hypotheses helps to nd the correct edge out of many possible gradient maxima in the image and leads to more accurate measurements that are used for learning the visual properties of an edge.
4.11. Evaluation of the Line Tracking Methods
0 20 40 60 80 100 120
0 100 200 300 400 500 600
run-time in milliseconds
number of sample points
Figure 4.11.: Scatter plot of the run-time versus the number of sample points used for tracking.
Figure 4.12.: Tracking an object with occlusion. The markers are only used for initialization.
4. Edge-Based Tracking Methods
Figure 4.13.: Tracking the engine hood of a car with the adaptive line tracking method.
The line model is manually created.