Adapting the similarity calculation is a core user-adaption in the visual analytics pipeline and has a direct impact on the algorithm and model performance. As we have already seen in Section 4.5, the projection-based similarity function allows us to interpret its results visually (c.f. Section 3.6.1) and gives rise to interpretable interactions for a similarity definition and adoption.
5.6.1 | User-guided Matrix Comparison in the Matrix Projection Explorer Framework
Our projection-based distance calculation technique, as presented in Section 4.5, is proposed as a basis for a range of applications. Specifically, we see two important domains:
(1) Applications using the projection-based approach for the distance calculation, and (2) Applications that enable domain experts to draw conclusions from the projection view as a complementary view for visual matrix analysis. We developed a prototype that implements the proposed distance computation and uses it to support visual exploration
Figure 5.11Matrix Projection Explorer is used to visualize matrices and their projections. The overview (2) shows a distance meta-matrix of all pairwise matrix distances for the VAST Challenge 2013 dataset with 120 matrices. Patterns, like closely related (dark groups) and outlying (light rows) matrices, stand out. The projection view (3) lets the user explore the selected matrices’ structural similarities expressed in the projection space.
in sets of matrices (Matrix Projection Explorer). We now describe the system and proposed analytic workflows.
Our system consists of two parts: (1) A NoSQL database to store matrices and cache projections. (2) A visual front end, consisting of four components (see Figure 5.6):
1. Data Explorer View: Allows the selection of a set of matrices to be examined and visualized.
2. Matrix Visualization View: Shows matrix visualizations in a heat-map-style display (for individual matrices) or an overview of a set of matrices (sorted by distance). A semantic zoom function allows the transition between projection viewports, the matrix view and the meta-matrix overview (see, e.g., Figure 5.12).
3. Projection View: The main interactive component, which can be found at the low-est semantic zoom level for every matrix pair, depicts the chosen matrices’ two-dimensional projections. The distance calculation is represented by means of the graph matching. The user can interact visually with the algorithm by excluding or including vertices and edges from the matching or adapt the penalty calculation to the task at hand.
4. Similarity Table: Shows the distance scores for a set of two or more matrices selected in the Data Explorer View.
In addition, a legend allows the highlighting of matrices, a color map shows colors applied in the matrices and a retrieval view ranks the most similar matrices to the current selection in ascending order. Finally, Views for experiments allow the user to start and keep track of experiments. As an alternative to the Matrix Visualization View, an interactive matrix visualization can be added, which would allow the user to highlight row/column selections in the projection plane and vice versa.
5.6.2 | Workflow and Interaction
Figure 5.12Excluding vertices from the calculation helps to filter out aspects of low importance.
In this soccer analysis task, it makes sense to exclude goalkeepers (numbers 0 and 14, middle) to find semantically similar game situations with a low goalkeepers influence.
From the left image we can see that two patterns occur: (1) The pattern [a] depicts a corner-kick (2) the diffusion-like pattern [b] refers to a goal-kick. After the similarity steering interaction (right image), the pattern [b] becomes even more obvious, and the area [c] gets apparent. This aspect denotes, in turn, situations, where the goalkeeper had an influence on the entire game situation.
When the user selects a set of matrices, their high- and low-dimensional represen-tations are rendered on the screen as matrix views and projection views. The two-dimensional points are rendered at the positions determined by the selected projection technique in the projection view. The matrix representation is rendered following the best practices presented in [Fek04]. All pairwise distances between the matrices are also calcu-lated, and the user can switch to ameta-matrixrepresentation of these. We experimented with a MDS projection of the distance values. While it proved beneficial in perceiving similarities (i.e., groupings) we rejected the idea because it was hard to reflect interactive changes to the distance calculation in a visually traceable manner.
For inspection purposes, the user can select a calculation to show the best possible bipartite graph matching allocation, as described in Section 4.5.1. The matching is
visual-ized by adding edges to the projection view (see Figure 5.6 (3)), connecting the matched vertex pairs. The user then has the means to interpret the matching and reason on its performance visually. In addition to this static procedure, the user caninteractwith the similarity algorithm in a feedback loop by excluding or including vertices and recomputing the bipartite graph matching accordingly. The result is then added to the similarity view, building a possible feedback loop.
5.6.3 | User-Guided Distance Calculation
Figure 5.13User-steerable distance score modification for projection-based distances: Users can apply a strong penalty to formulate restrictive similarity queries, emphasizing the structural and topological similarity. On the other hand, no penalty or even the exclusion of long/short edges introduces fuzziness in the process and allows matching structurally similar matrices while ignoring some differences.
Our approach explicitly supports the interactive guidance of the distance calculation by the user. As for example, Figure 3.7 or Figure 5.12 illustrate the user can zoom from an overview distance matrix into a specific area of interest and investigate the impact of the dimensionality differences to the distance score. By zooming from the overview meta-matrix into an area of interest, the user can investigate each pairwise meta-matrix comparison in the projection view. In this view three different user interactions are possible:
1. Selecting and deactivating projection points allows the exclusion outliers or other-wise irrelevant rows/columns;
2. Selecting edges with a lasso or by clicking on the respective distance histogram bin, allows the exclusion ranges of edge lengths from the distance;
3. Adapting the penalty function (clicking on the distance histogram penalty bin or via a specific penalty dialogue) allows modification of the effect of the matrices’ size differences on the distance.
Figure 5.14Changing the penalty function has a large impact on the appearance of the dis-tance meta-matrix showing all pairwise matrix comparisons. From left to right, the ZeroPenaltyandMaxDistSquarepenalty functions are rendered in the upper diago-nal part of the matrix. The lower part shows theMaxDist, for reference purposes.
With these interactions, a large part of the fuzziness spectrum regarding the graph matching problems can be covered. Figure 5.13 describes the relationship between the penalty on the one hand and the exclusion of edges on the contrary. If no penalty is artificially added (Figure 5.13 (middle)) all projection points that can be matched will be matched and all dimensionality differences are ignored. Thus, structurally similar matrices are retrieved. If the user wants to emphasize the dimensionality differences than a strong penalty function can be applied (Figure 5.13 (left)). In contrast, the user may decide to ignore the dimensionality and even ignore some level of structural modifications by excluding long edge-ranges in the projection panel (Figure 5.13 (right)). We depict the usefulness of our interaction mechanisms in the use cases in Sections 5.8.2 and 5.8.2.
Figure 5.14 visually depicts the impact of changing the penalty function for the VAST 2013 Challenge Dataset. From left to right, theZeroPenaltyandMaxDistSquarepenalty functions are plotted in the upper diagonal part of the matrix. The lower diagonal part shows theMaxDist, for reference purposes.
The choice of penalty function is closely related to the graph matching tasks at hand.
If the size differences are important, a “strong” penalty must be applied. The lower the penalty, thefuzzierthe distance calculation becomes. In other words, a “weaker”
penalty function, e.g.ZeroPenalty, shifts the focus to matrix comparisons that ignore size differences and try to derive a statement from the available information. Other penalty functions are possible, ranging from a static penalty score to a penalty that reflects the situation in high-dimensional space.