Analyses of the Used Features

In order to ensure that the used feature algorithms are suitable for image retrieval tasks the feature vectors are analysed. For base-level analyses, automatic approaches are appropri-ate since they are economical and reproducable. Classification tasks resembling cappropri-ategory searches are most suitable in this case. The three domains introduced above (artexplosion, myMondrian and shark webcam) are merged to one data set to get a broad database.

Beginning with a qualitative inspection the visualisations of the feature distributions in the feature spaces are examined. For visualisation purpose a PCA is performed and the projection on the two directions of the largest variances are presented in figures 3.8, 3.10 and 3.11. Some assumptions regarding the separability of meaningful subsets in certain feature spaces emerge:

- The structure features are suitable for detecting the four camera positions of the shark webcam. Clustering this data set based on the structure features therewith is suitable.

- All feature types can distinguish between different sequences of the myMondrian set.

The combined structure feature (intensity, hue and saturation) seems to be more suitable than the single structure feature. This can be recorded as an example for the advantages of feature combinations.

- In the colour feature spaces the myMondrian set varies in a quite orthogonal direction to the orientation of the other sets. This is a hint, that the most suitable feature combination depends on the used domain.

- The (semantic) categories of the artexplosion collection cannot be clustered easily in the used low-level feature spaces. Low-level features are not sufficient to describe the image content (semantic gap).

-0.4 -0.3

0 0.25 0.5 0.75 1

hue value histogram

-0.4 -0.3 -0.2

0 0.2 0.4 0.6 0.8 1

lightness value histogram

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

saturation value histogram

-0.4 -0.2 0 0.2 0.4

-0.4 -0.2 0 0.2 0.4

quantisation of HLS

-4 -2 0 2 4

-4 -2 0 2 4

IHS value of the structur

-1 0 1 2 3 4

0 1 2 3 4 5

intensity values of the structure Figure 3.10: Distribution of the three domains along the eigenvectors with the two largest eigenvalues. The eigenvectors are computed in the combined data set based on different image features.

Green stands for artexplosion photos, red marks myMondrian images and blue represents pictures of the shark webcam.

artexplosion −1.04 0.74 2.06 0.54 2.06 0.61 −1.18 0.73 −0.52 0.08 1.17 0.6 0.66 0.63 −1.5 0.87 1.11 0.47 0.69 0.23 -underthesea −0.9 0.85 1.82 0.54 1.95 0.5 −1.29 0.71 −0.37 0.08 1.13 0.57 0.79 0.53 −1.33 0.8 1.11 0.32 0.51 0.25 -animals −0.8 0.72 1.81 0.54 1.6 0.63 −1.01 0.84 −0.48 0.07 0.87 0.56 0.95 0.57 −1.29 0.54 1.43 0.47 0.81 0.11 -doorswindows −1.01 0.55 2.21 0.50 2.33 0.66 −1.07 0.56 −0.64 0.07 1.02 0.57 1.0 0.62 −1.62 0.56 1.15 0.46 1.00 0.1 -teddybears −0.89 0.65 2.11 0.50 2.06 0.43 −0.87 0.70 −0.55 0.05

0.91 0.44 0.85 0.47 −1.59 0.63 1.3 0.35 0.88 0.07 -sunrisesunset −1.52 0.74 2.21 0.5 2.19 0.54 −1.37 0.7 −0.58 0.08 1.65 0.53 0.04 0.28 −2.14 0.94 0.58 0.36 0.21 0.18 -venezuela −0.97 0.5 2.23 0.34 2.19 0.42 −0.91 0.53 −0.67 0.1

1.01 0.37 0.61 0.63 −1.27 0.66 1.16 0.31 1.09 0.06 -iceland −1.11 0.4 2.38 0.4 2.4 0.26 −1.68 0.61 −0.46 0.04 1.54 0.19 0.15 0.27 −0.43 0.52 1.31 0.33 0.74 0.05 myMondrian −2.13 0.43 3.53 0.49 1.19 1.03 −0.41 0.35 −3.74 0.27

−1.03 1.05 1.96 0.71 −1.02 0.55 0.28 0.78 0.48 0.09 shark webcam 0.001 2.83 1.44 1.17 2.45 0.17 −0.83 1.55 −0.54 0.001

0.9 0.82 1.2 0.41 −0.66 0.27 1.39 0.09 0.32 0.01 Table 3.3: Mean (left value) and variance (right value) along the first (first row) and second (second row) principle components of the combined data set. The bold values select categories and features where the variance along the second eigenvector exceeds that one along the first eigenvector. This indicates, that single categories have their largest variety in another direction than the entire data set.

These observations have to be supported by further quantitative analyses. An obvi-ous measure to analyse the distribution of the data in the feature space is the variance.

Therefore the eigenvalues of the combined image set are computed and the five largest ones are listed in figure 3.9 for each feature.

The results show, that the structure features and the texture feature detect better the variablity within the set. Thus they are more suitable to detect interesting subsets.

This coincides with the qualitative observation in figures 3.8, 3.10 and 3.11. Regarding colour, the colour histogram seems to be the most suitable for detecting differences between images.

Since structure and texture seem to be appropriate to divide image sets into subsets, they are analysed in detail regarding single domains (see table 3.3).

One observation is that in some feature spaces and for some domains or categories the variance along the second eigenvector exceeds the variance along the first eigenvector of the combined data set. Consequently the main extension of the data within a feature space depends on the domain. Coinstantaneously the efficiency of the image features also depends on the image domain. The development and selection of domain dependent feature detection algorithms may be a consequence.

Similar to that is the task dependent feature weighting used in [Deselaers et al., 2004a].

The retrival performance based on the error rate in a classification approach is analysed.

They observed that colour histograms are a good choice to describe arbitrary photographs

– doorswindows 0.002 1.005 0.773 0.781 0.747 0.125

0.001 0.897 0.657 0.730 0.513 0.051

– teddybears 0.003 1.215 0.739 0.933 0.896 0.087

0.002 0.982 0.441 0.520 0.393 0.041

– sunrisesunset 0.008 1.452 0.765 1.016 0.898 0.195

0.005 1.105 0.420 0.637 0.500 0.075

– venezuela 0.001 1.406 1.042 0.844 0.944 0.115

0.001 1.035 0.439 0.665 0.447 0.059

– iceland 0.001 1.263 0.737 0.689 0.982 0.059

0.001 0.860 0.453 0.378 0.495 0.044

myMondrian 0.071 2.502 1.072 1.193 0.982 0.905

0.007 1.446 0.641 0.648 0.545 0.094

shark webcam 0.002 3.898 1.665 0.649 1.729 0.014

0.001 1.250 0.481 0.268 0.587 0.003

Table 3.4: The two largest eigenvalues of the different feature spaces. PCA is computed on each subset separately.

whereas the pixel values combined with a suitable distance measure are better for medical radiographs. Therefore they confirm the demand to select image features task and domain dependently.

In section 5.2.3 the distribution of a single subset in relation to the image domain is used to evaluate the impact of a data space transformation. The developed measure compares the distances within a relevant subset with the distances to the remaining data.

Independently from the transformation approach it can be observed that again the sepa-rability of different subsets depends on the used feature.

As has been shown in table 3.3 the different categories and domains show larger vari-ances along the second eigenvector in different feature spaces. This motivates the assump-tion that for different data sets different features are more suitable to describe these sets.

Therefore the eigenvalues in the feature spaces according to single subsets are computed and listed in table 3.4.

The category dependent eigenvalues exceed the eigenvalues computed in the entire data set (see table 3.3). Domain dependent features may be advantageous. Just the second eigenvalues in the texture feature space are smaller. This is a hint that the entire data set in the texture space constitutes a mixed and compact cluster without explicite directions for the individual domains. This is confirmed by the fact that in the PCA-space of the combined data the second eigenvalue is larger for almost all regarded subsets (see table 3.3). The visualisation of the distributions in figures 3.8, 3.10 and 3.11 indicates this. The texture feature may be adequat to describe the textures in all domains but unsuitable to perform a categorisation into the three domains.

-5 -4 -3 -2 -1 0

-1 0 1 2 3 4 5

hue value of the structure

-1 0 1 2 3

-4 -3 -2 -1 0 1

saturation value of the structure Figure 3.11: Distribution of the three domains along the eigenvectors with the two largest eigenvalues. The eigenvectors are computed in the combined data set based on different image features. Green stands for artexplosion photos, red marks myMondrian images and blue represents pictures of the shark webcam.