Environment Clearance

For a part with complex shape, Environment Clearance (BERENSON et al., 2007) can be used instead of Pointwise Shortest Distance. For example, cavities and holes on the surfaces of the part may be feasible assembly grasp locations even if the surfaces of these pockets are located very close to the environment, because an assembly operator can access these pockets as long as their openings are not blocked. To color the part surfaces with method Environment Clearance, the clearance along the normal vector at each vertex has to be calculated. Consequently, Environment Clearance is much more computationally expensive, but it is also much better at isolating/highlighting potential grasp locations than Pointwise Shortest Distance.

Before Environment Clearance is computed, algorithm Visual Shell is applied to find a consistent normal orientation over the surfaces of the part. Visual Shell is similar to the algorithm presented in BORODIN et al. (2004). However, Visual Shell has been implemented on GPU and hence it runs much faster. To speed up the computation even further, Visual Shell ignores all triangles that are invisible from the outside of the part.

Environment Clearance prefers triangle vertices with high clearance in the directions of the corresponding vertex normal vectors. From each vertex 𝑝𝑝 on the part surfaces,

rays are cast inside a (right circular) cone with aperture 𝜃𝜃, its apex located at 𝑝𝑝, and its alignment defined by the vertex normal vector 𝑛𝑛, as shown in Figure 2.2. The Environment Clearance at 𝑝𝑝 is then defined as:

𝐸𝐸𝐸𝐸(𝑝𝑝,𝜃𝜃) = min𝑟𝑟:𝑟𝑟∙𝑛𝑛≤cos (𝜃𝜃/2)𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷(𝑝𝑝,𝑟𝑟) (2.1) where 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷(𝑝𝑝,𝑟𝑟) is the shortest distance between 𝑝𝑝 and the nearest point ray originated from 𝑝𝑝 with direction 𝑟𝑟 hits either the part itself (although not at 𝑝𝑝) or the environment. Our implementation allows the user to choose between (i) ray-to-part collision detection, (ii) environment collision detection, and (iii) ray-to-environment-and-part collision detection, as shown in Figure 2.2. The third collision detection mode is the default one. However, the first two modes are faster to compute and are more appropriate for certain types of analysis. For example, to visualize clearance of cylinder walls of a cylinder/engine block, there is no need to compute the distances between the cylinder walls and the environment around the engine and hence the first mode is the best choice. Moreover, both the distance range and the color range can be adjusted by the user. Furthermore, the user can vary the cone aperture in order to cover a smaller or bigger region of the environment. In BERENSON et al. (2007), the default value for angle 𝜃𝜃 is 𝜋𝜋⁄12. Finally, the user can choose the number of rays to be casted inside each cone. Given the number of rays, the positions of the intersection points between the rays and the cone’s flat base can be obtained from SPECHT (2017). These positions are the centres of the best known packings of equal circles in a circle, where the number of rays is equal to the number of circles to be packed. The default number of rays is set to 9 (PIRL, 1969).

Fig. 2.2 In subfigure (a), a (right circular) cone with apex 𝒑𝒑, aperture 𝜽𝜽, and axis 𝒏𝒏, where 𝒏𝒏 is the surface normal vector at triangle vertex 𝒑𝒑 on the surface of the spherical part. In the other three subfigures, the cylinder in the middle and the transparent cylinder represent the part to be assembled and the environment, respectively, and distance in a range from 0 mm to 200 mm is mapped to color in a range from red to blue. Subfigures (b), (c), and (d) are rendered using (i) ray-to-part collision detection, (ii) ray-to-environment collision detection, and (iii) ray-to-environment-and-part collision detection, respectively. The order of the subfigures is left-right.

3 Results

In order to demonstrate the applicability of the two methods we presented in this paper (i.e., Pointwise Shortest Distance and Environment Clearance), two relevant test cases from the automotive industry are presented in this section. As mentioned before, we have yet to conduct a study with assembly simulation experts to validate the degree of usability of these two methods for them; however, our validation proposal is listed in Section 4. The results of the validation study will be presented in future work.

Firstly, a CEM box needs to be installed in a dashboard. In Figure 3.1, the CEM box colored in Accumulative mode had to follow a precomputed path in order to be installed. Clearly, the surface patches covered by green and blue are potential feasible grasp locations, whereas the surface patches covered by red correspond to the parts of the CEM box that came too close to the dashboard. Efficient distance computation in IPS enables us to compute shortest distances from a part modeled by a mesh containing a large number of triangles and render it interactively. For example, a Center Console model (as shown in Figure 3.2) that is frequently used at FCC for testing contains 155646 vertices and 233489 triangles.

Fig. 3.1 The CEM box located at its start configuration is shown in Accumulative mode. To color the vertices on the CEM box surfaces, the CEM box traced the path from its start configuration to its goal configuration.

Secondly, as shown in Figure 3.2, a gear stick and a handbrake are covered by a Center Console, which is the part that has to be removed. In addition to the gear box and the handbrake, the environment also contains a dashboard and a roof. Both Pointwise Shortest Distance and Environment Clearance are used to color the surfaces of the Center Console and the results are shown in Figure 3.2. Environment Clearance is more computational expensive than Pointwise Shortest Distance, but it does provide additional information about the surfaces of the part beyond the shortest distance between each vertex on the surfaces of the part and the environment. For example, as shown in subfigure (a) of Figure 3.2, several surface patches on the left side of the Center Console are colored red using Pointwise Shortest Distance (in Instant mode), because triangle vertices on these patches are located very close to either the gear stick or the handbrake, even though these

patches can be easily accessed by a manikin hand tracing a high clearance path.

However, as shown in subfigure (b), these surface patches are rendered in blue.

Furthermore, the edges of the Center Console’s storage boxes are rendered in green in the right subfigure. Clearly, Environment Clearance is better at distinguishing potential feasible grasp locations from locations that cannot be easily accessed.

Finally, in subfigure (c), a manikin grasps the edges, which are classified as feasible grasp locations by Environment Clearance, of the Center Console’s storage boxes with both hands.

Fig. 3.2 The Center Console is colored using Pointwise Shortest Distance (in Instant mode) in subfigure (a), whereas Environment Clearance (also in Instant mode) is used to color the surfaces of the Center Console in subfigure (b). To synthesize the figure in subfigure (b), ray-to-environment-and-part collision detection was chosen over environment collision detection and ray-to-part collision detection. In subfigure (a), distance in a range from 0 m to 2.6 m is mapped to color in a range from red to blue, whereas the maximum distance is 2 m in subfigure (b). In Subfigure (c), a manikin grasps the center console with both hands. The roof is rendered as a semi-transparent object in the subfigure. The order of the subfigures is left-right.

4 Discussion

Methods Pointwise Shortest Distance and Environment Clearance enable a DHM tool user to color part surfaces by taking environmental constraints into account so that he/she can easily identify potential feasible grasp locations for a manikin.

However, these two methods do not identify the best grasp location nor the correponsindg grasp points for a manikin hand. In order to identify these grasp points, additional part parameters such as weight and material properties must be taken into account by a grasp planner. Furthermore, as the last step, the grasp planner is required to bring the fingers of the hand to the grasp points. Such a grasp planner is not presented in this paper.

In BERENSON et al. (2007), Environment Clearance enables an automatic grasp planner to find stable and collision-free grasps faster in cluttered environments. In the future, we intend to integrate these two methods with a grasp planner to enable automatic fast planning of grasp actions in IMMA.

Another direction for future work is to enable two-arm cooperation (i.e., exchanging and regrasping assembly parts using manikin’s two arms). By divining a precomputed assembly path for a part into multiple subpaths, either manually or automatically, Pointwise Shortest Distance and Environment Clearance can be applied to color the part surfaces so that feasible grasp locations for each subpath can be easily identified. By taking the feasible grasp locations for all subpaths as input, a motion planner will be able to return a plan that coordinates the motions of manikin’s two arms.

Moreover, we apply at moment both Pointwise Shortest Distance and Environment Clearance to meshes we have obtained from our customers without a resampling step. Since the running time of the two methods is proportional to the number of mesh vertices and the shapes of mesh triangles affect the shapes of the potential grasp locations rendered by IPS on mesh surfaces, we are considering to add a resampling step in order to generate a new discretization of the original geometry with a mesh that exhibits the uniform sampling property. For an overview of recent advances in remeshing of surfaces, we refer to ALLIEZ et al. (2008).

Furthermore, we are planning to conduct an industrial user study to demonstrate that the proposed methods indeed enhance the flexibility and hence the usability for the DHM tool user who wants to specify the manikin’s hand grips. As for assembly simulation experts, we intend to invite at least 8 engineers from four Swedish vehicle OEMs (2 from each company). All study participants are then randomly divided into two groups. We will also prepare 10 test cases. The first group will be assigned the first 5 test cases where each participant has to specify a grasp for each case with IMMA without access to neither Pointwise Shortest Distance nor Environment Clearance, whereas the second group will be assigned to the same test cases where each participant has to use both Pointwise Shortest Distance and Environment Clearance before specifying a grasp for each case with IMMA. Next, the two groups are assigned the remaining 5 test cases. But this time, the first group has to use both Pointwise Shortest Distance and Environment Clearance, whereas the second group is not allowed to use them. Moreover, the study participants will be asked to respond to a questionnaire to find out whether they think that Pointwise Shortest Distance and Environment Clearance have enhanced the flexibility and hence the usability for them. Furthermore, the study participants will be asked to score each grasp he/she specified with IMMA. Specifically, we want to find out whether Pointwise Shortest Distance and Environment Clearance enable the study participants to find better grasps. Finally, to find out whether Pointwise Shortest Distance and Environment Clearance enable the study participants to find good grasps faster, all test runs will be timed.

5 Conclusions

We have presented two methods called Pointwise Shortest Distance and Environment Clearance for coloring part surfaces by taking environmental constraints into account. The implementation of these two methods are robust enough to handle triangle meshes with common geometric flaws such as cracks, gaps, and even inconsistently oriented normal vectors. The goal of the proposed methods is to enhance the flexibility and hence the usability for a DHM tool user who wants to specify manikin’s hand grips on a part by highlight areas on the surfaces of the part that are suitable for hand grips. These two methods constitute the first step toward automatic grasp planning in IPS IMMA DHM tool. Currently, we are planning to conduct an industrial study in order to demonstrate the benefits of these two methods.

List of references

Alliez, P.; Ucelli, G.; Gotsman, C.; Attene, M.: Recent Advances in Remeshing of Surfaces, Recent advances in remeshing of surfaces, Shape Analysis and Structuring 2008, pp. 53-82.

Berenson, D.; Diankov, R.; Nishiwaki, K.; Kagami, S.; Kuffner, J.: Grasp planning in complex scenes, Proceedings of the 7th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2017), pp. 42-48.

Borodin, P.; Zachmann, G.; Klein, R.: Consistent normal orientation for polygonal meshes, Proceedings of the Computer Graphics International (CGI 2004), pp. 18-25.

Högberg, D.; Hanson, L.; Bohlin. R.; Carlson, J. S.: Creating and shaping the DHM tool IMMA for ergonomic product and production design, International Journal of the Digital Human 2016, Vol. 1, No. 2, pp. 132-152.

Li, Y.; Saut, J. P.; Pettré, J.; Sahbani, A.; Multon, F.: Fast Grasp Planning Using Cord Geometry, IEEE Transactions on Robotics 2015, Vol. 31, No. 6, pp. 1393-1403.

Miller, A.; Allen, P. K.: Graspit!: A Versatile Simulator for Robotic Grasping, IEEE Robotics and Automation Magazine 2004, Vol. 11, No.4, pp. 110-122.

Pirl, U.: Der Mindestabstand von n in der Einheitskreisscheibe gelegenen Punkten, Mathematische Nachrichten 1969, Vol. 40, pp. 111-124.

Specht, E.: The best known packings of equal circles in a circle (complete up to N = 2600) [Online], Available: http://hydra.nat.uni-magdeburg.de/packing/cci/. [Accessed:

15-Mar-2017].

Miyajima et al.:

Optimal Arrangement of Inertial Sensors on a Motion