5 Evaluation
(a)
5-hole-bar 3-hole-bar
screw nut
cube screw ring
screw 3-hole-bar 3-hole-bar 5-hole-bar
screw cube cube screw wheel rim
screw wheel rim
(b)
Figure 5.8: The setup of experimental investigation 3. a) A 20-part toy airplane in its reference pose. b) A sketch of the airplane model’s kinematic tree. Only the pose parameters of the parts in light blue are recovered
5.3.1 Methology and Data Sets
The setup of the third experimental investigation is illustrated in Fig. 5.8. It shows the assembly that is localized in the following, namely a 20-part toy airplane, together with a sketch of its kinematic tree. The assembly is mounted to a fixture consisting of a 5-hole-bar, a red cube and a yellow screw that are not part of the model. To manually record the ground truth data of each individual part would have been too difficult with the available vernier calipers and goniometer. Thus, only the part poses of the five best accessible parts are considered in the following. The respective parts are visualized in Fig. 5.9(a). As ground truth, the rotation parameter of parts 1, 2, 3, and 5 was recorded that was least constrained by the assembly structure. For example, the least constrained rotation parameter of the nut part 1 was the rotation around the screw thread to which the nut was attached. Furthermore, the least constrained translation parameter of part 1 and 4 was recorded, which was a translation along the individual screw thread to which the respective parts were attached. As in experimental investigation 1, the accuracy of the manual measurements is expected to be better than1◦ for rotation parameters and better than 1mm for translation parameters.
The experimental investigation was conducted in the following way. First, a camera was statically placed at a distance of 80cm from the assembly. The camera was then zoomed to capture images with a scale of 0.3mm per pixel and calibrated to the fixture to which the airplane was mounted. Afterwards, 50 images of the assembly were captured in which the part poses vary systematically. The extreme points of variation are illustrated in Fig. 5.9. For each image measurement, the associated ground truth was recorded as indicated above. Then, the EKPF was used to estimate the pose parameters of parts 1 to 5, together with the pose parameters of their respective parents within the kinematic
5.3 Experimental Investigation 3
1 2
3 4
5
(a) (b)
Figure 5.9: The range of recorded toy airplane poses. a) Pose parameters of the numbered parts are manually measured and recorded as ground truth. b) Throughout the experimental investigation, the assembly pose was varied in between the pose depicted in the left image and this pose
tree. Note that the root node part, i.e. the 5-hole-bar, was held by the fixture and its pose parameters known from the camera calibration. The complete set of seven localized parts is marked in light blue within the kinematic tree sketch in Fig. 5.8(b).
Taken together, the localized parts exhibit 42 DOF. However, 14 DOF are strongly con-strained by the assembly structure. The structurally concon-strained parameters are part of the particle filtering state space but their values are nearly constant and known in ad-vance from the assembly reference pose. Consequently, the number of DOF that were effectively recovered in this experimental investigation is 28. The pose parameters of the unlocalized parts were taken to be the values of the reference pose represented by the kinematic tree, though this constraint wasn’t imposed on the real assembly when cap-turing the images. The scene illumination was again provided by two 110W cold light lamps that were placed to the left and to the right of the camera. Furthermore, the scene illumination was influenced by strongly varying amounts of daylight.
5.3.2 Results
The mean error and standard deviation of the recovered pose parameters of parts 1 to 5 w.r.t. the ground truth is given in Tab. 5.4. For the rotation parameters, the mean error is−1.4◦ in the worst case and−0.3◦in the best case, while the standard deviations range from0.6◦to3.1◦. More specifically, the highest two standard deviations and mean errors are associated with the most challenging parts 1 and 3. The former experiences inter-part occlusion of up to 80% of its contour length while the latter exhibits strong
5 Evaluation
Table 5.4: The mean error (µ) and standard deviation (σ) of rotation and translation parameter measurements w.r.t. the ground truth
Rotation Translation
Part 1 Part 2 Part 3 Part 5 Part 1 Part 4 µ -0.4◦ -0.4◦ -1.4◦ -0.3◦ 0.2mm 0.3mm σ 3.1◦ 0.6◦ 3.1◦ 2.1◦ 0.5mm 0.6mm
(a) part 1, rotation (b) part 1, translation
(c) part 2, rotation (d) part 3, rotation
(e) part 4, translation (f) part 5, rotation
Figure 5.10: Histograms of the pose parameter deviations from the ground truth. Solid vertical lines denote the mean error. Dashed lines visualize the standard deviation
5.3 Experimental Investigation 3
(a) (b)
(c) (d)
(e) (f)
Figure 5.11: Results of the airplane localization. The visible contour edges of the localized air-plane parts have been backprojected to the image air-plane under the recovered part poses. a,c,e) Best three results, ranked by the sum of squared deviations from the ground truth. The backprojected contour edges fit quite well to the physical objects.
b,d,f) Worst three results
5 Evaluation
rotational shape symmetries w.r.t. the axis of the recovered rotation parameter. In contrast to this, the best localization performance is achieved for part 2 which provides favorable edge information due to its very elongated shape. For this part, the achieved results are competitive to the results of the system in [HOW96] which localized a desklamp that was of similarly elongated shape. Concerning the recovered translation parameters, our system again performs competitively well in comparison to all systems presented in Chap. 2.6, i.e. with a mean error and standard deviation of less than 1mm.
The results in Tab. 5.4 were achieved by executing the EKPF with 500 Particles and 5 iterations of mean shift. Recovering the 28 DOF takes 56s on a 2GHz Pentium IV. The memory consumption of the airplane assembly model is 41MB. Figure 5.11 provides a visual impression of the localization accuracy of the system, by means of the best and worst three results w.r.t. the sum of squared pose parameter deviations from the ground truth. Furthermore, Fig. 5.10 shows the resulting histograms of the pose parameter devi-ations from the ground truth.