Source space

Neuromagnetic responses in the signal space can yield insights in the time course of the underlying cortical processes. However one should be very cautious in interpreting these in terms of their un-derlying cortical sources as any distribution of neuromagnetic fields (measured in two-dimensions) can originate from an infinite number of source configurations in the 3-dimensional source space.

In addition, recording MEG signal waveforms depends of the subjects head position within the ar-ray of sensors, making it difficult to interpret averaged magnetic fields as a given sensor might col-lect data originating from different brain regions across subjects.

These disadvantages of the MEG, can both be avoided by transforming the data from the signal to the source space by the Minimum Norm Estimate. As described previously, this procedure is based on the same set of dipoles for each subjects and allows interference of activity to underlying brain structures.

Prior to transforming the data to the MNE, the individual grand average across all concepts was subtracted from the averaged neuromagnetic activity of each concept (or category) to eliminate ac-tivity related to the processing of all objects.

A.3.2.2.1.1 Difference waveforms

Figure A-12 presents the topographical distribution of the difference waveforms for each category.

During ‘early’ time windows (between 90 and 290 ms after picture onset) evidence for larger activi-ty in ventral regions was found, which was partially modulated by object category (DEPTH:

36

A.3.2 : Results - Evoked magnetic fields

90-130 ms: F(2,18)=4.16, p=.044; 130-170 ms: F(2,18)=3.12, p=.073 ; 250-290: F(2,18)=2.96, p=.095; CATEGORY x DEPTH: 130-170 ms: F(6,54)=2.32, p=.095; 170-210 ms: F(6,54)=3.23, p=.048; 250-290 ms: F(6,54)=2.32, p=.085). Post-hoc contrasts for the interaction CATEGORY x DEPTH analyzed the factor DEPTH separately for each CATEGORY.

In the time window starting 130 ms after stimulus onset, the categories animals, flowers and clothes (but not furniture), showed more activation in the ventral area (compared to the other ones). In the following time window (170-210 ms) the same topographical pattern was revealed for ‘Flowers’, while DEPTH did not differ for the three other categories in post-hoc tests. Finally, the statistical tendency CATEGORY x DEPTH for the time interval from 250 to 290 ms was due to more activity in the ventral stream (compared to the two other ‘streams’) for animals and flowers, while for clothes and furniture the levels of depth did not differ from each other.

Moreover, for almost all time intervals higher dipole moments were found for the posterior regions (GRADIENT: 90-130 ms: F(1,9)=3.76, p<.09; 130-170 ms: F(1,9)=4.89, p=.05; 170-210 ms:

F(1,9)=5.70, p<.05; 210-250 ms: F(1,9)=8.59, p<.02; 250-290ms: F(1,9)=4.31, p<.07; 370-410 ms:

F(1,9)=4.98, p<.06 ; 410-450 ms: F(1,9)=10.24, p<.02).

Figure A-12: Minimum norm estimate for each category after subtracting the average across all concepts. The left panel shows a top view of the head (nose up) and the right panel shows a back view of the same data. For illustration four time windows (90-130 ms, 170-210 ms, 250-290 ms, 330-370 ms) were selected.

A.3.2 : Results - Evoked magnetic fields

In addition, a statistical tendency for more right- than left hemispheric activity (F(1,9)=3.69, p<.09) during the first time window (90-130 ms) was found. The interaction CATEGORY x

HEMISPHERE (F(3,27)=4.36, p<.0.03) for the time window 250-290 ms indicated that categories tended to differ in the right, but not the left hemisphere (CATEGORY: left: F(3,27)=0.71, n.s.;

right: F(3,27)=2.85; p<.09). In the right hemisphere, clothes elicited significantly lower source activity than flowers and furniture.

No other main effects or interactions involving the factor category were found.

In sum, source strength was highest over posterior regions across all time windows. During early time windows some categories showed higher activity over ventral compared to dorsal regions.

A.3.3 Spatio-temporal correlations

Figure A-13 illustrates the within- and between category correlations for one area (corresponding to the right temporal cortex) in the time window from 170 to 210 ms after onset of the picture. For each of the four categories, the highest correlation is between the objects belonging to the same cat-egory.

38

Figure A-13: Within- and between-category contrast scores for the right temporal region (area 12). Furn: furniture, cloth: clothes, anim: animals, flow: flowers.

A.3.3 : Results - Spatio-temporal correlations

The correlation coefficients were further reduced by computing the sum of the difference of the within and between category Z’-scores, resulting in one single contrast score for each of the 17 ar-eas for each time window. Larger values indicate that objects of the same category correlate higher than objects of different supra-ordinate categories. To visualize the topographical distributions, the contrast scores were projected on the two-dimensional head schemes and are displayed in Figure A-14

Statistical analysis focused on the central and posterior areas. In a first step, the mean contrast z-score across all the 13 areas was computed. The mean contrast z-score was significantly different from zero for all time windows (90-130 ms: t(9)=4.50, p<.002; 130-170 ms: t(9)=5.38, p<.001;

170-210 ms: t(9)=4.86, p<001.; 210-250 ms: t(9)=5.02, p<.001; 250-290 ms: t(9)=6.08, p<.001;

290-330 ms: t(9)=5.35, p<001.; 330-370 ms: t(9)=5.52, p<.001; 370-410 ms: t(9)=5.71, p<.001;

410-450 ms: t(9)=4.86, p<.001). This indicates higher correlations within categories then across cat-egories.

ANOVAs were performed to further analyze any category-specific localization and time course of these differences. Separate contrast scores for each category and area were entered into the analysis.

A main effect CATEGORY was observed during later time windows (330-370 ms: F(3,27)=5.90,

Figure A-14: Topographical distribution of the contrast scores for each consecutive time window.

A.3.3 : Results - Spatio-temporal correlations

p<.01; 370-410 ms: F(3,27)=3.03, p<.07; 410-450 ms: F(3,27)=3.39, p<.05; see Figure A-15). Sta-tistical contrasts between categories revealed higher contrast scores for animals compared to flow-ers (between 300 ms and 410 ms) and to clothes (330 to 450 ms after stimulus onset). Contrast scores of furniture were higher than contrast scores of clothes (330 to 450 ms).

The factor CATEGORY tended to interacted with DEPTH only during one time window (210-250 ms; CATEGORY x DEPTH: F(6,54)=2.70, p>.06). In the ventral stream, animals had higher con-trast scores than all other categories.

Independent of the category, contrast scores had a distinct topographical distribution as indicated by the following statistical effects. There was a tendency for higher contrast scores over posterior com-pared to central regions (GRADIENT: 130-170 ms: F(1,9)=4.02, p<.08; 210-250 ms: F(1,9)=3.49, p<.1; 250-290 ms: F(1,9)=4.85, p<.06: see Figure A-14). Additionally, an interaction HEMI-SPHERE x GRADIENT (170-210 ms: F(1,9)=9.05, p<.02; 330-370 ms: F(1,9)=4.17, p<.08) was found. For the time window from 170 to 210 ms, post-hoc tests showed that contrast scores were

40

Figure A-15: Illustration of the main effect CATEGORY for the time windows deom 330 to 450 ms.

Contrast scores by category

0 0.2 0.4 0.6 0.8 1 1.2

330-370 370-410 410-450 ms

contrast score Animals

Flowers Clothes Furniture

A.3.3 : Results - Spatio-temporal correlations

lower over the left compared to the right hemisphere and larger over posterior compared to central areas. For the time window from 330 to 370 ms, subtests did not yield any significant differences between regions.

Moreover, contrast scores differed in their topographical distribution in dorso-ventral direction (DEPTH: 170-210 ms: F(2,18)=3.55, p<.09; 210-250 ms: F(2,18)=3.22, p<.1; 250-290 ms:

F(2,18)=8.57, p<.01), which partially interacted with the factor hemisphere (HEMISPHERE x DEPTH: 90-130 ms: F(2,18)=3.88, p<.07; 130-170 ms: F(2,18)=6.33, p<.02; 290-330 ms:

F(2,18)=3.68, p<.06). Post-hoc testing was not significant for the time interval covering the time pe-riod 90-130 ms after picture onset. For all other significant time windows the most dorsal regions showed lower scores than the ventral regions (all F(2,18)>3.6, p<.08). In the time windows with a significant HEMISPHERE x DEPTH interaction, this effect of higher contrast scores in ventral compared to dorsal areas was only prominent in the left hemisphere (RH: DEPTH: all

F(2,18)>4.29, all p<.04).

Overall, spatio-temporal correlations indicate higher association between objects belonging to the same compared to objects belonging to different superordinate categories. This effect was most pro-nounced over posterior and ventral regions.

A.3.4 Hierarchical clustering

Hierarchical clustering was performed for each hemisphere. In addition to the time windows with a width of 40 ms, two time intervals were examined: one covering the time period from 120 to 210 ms, and one from 210 to 450 ms after onset of the picture. Figure A-16 illustrates the clustering pro-cess for the left and right hemisphere in the time window from 210 to 450 ms. As can be seen, hier-archical clustering for the left hemisphere resulted in combining the a priori semantic categories.

After 12 passes (see column 13 in the left panel of Figure A-16), the four super-ordinate categories

A.3.4 : Results - Hierarchical clustering

were reconstructed by the clustering procedure. For the right hemisphere, occasional misclassifica-tion appeared and no original category became fully evident.

42

Figure A-16: Dynamic process of unsupervised hierarchical clustering of the data averaged between 210-450 ms over the left and right hemisphere. For illustration, each base-level concept was assigned a sequential number from 1 to 16. Each column matrix represents the ad-hoc clustering at each pass. This process starts with 16 different vectors and ends with one collapsed cluster. Shaded cells highlight the online clustering of base-level concepts within one super-ordinate category.

Figure A-17: Dynamic process of unsupervised hierarchical clustering of the data averaged between 120-210 ms over the left and right hemisphere. For illustration, each base-level concept was assigned a sequential number from 1 to 16. Each column matrix represents the ad-hoc clustering at each pass. This process starts with 16 different vectors and ends with one collapsed cluster. Shaded cells highlight the online clustering of base-level concepts within one super-ordinate category.

A.3.4 : Results - Hierarchical clustering

For the earlier time window (see Figure A-17) classification was less good. For both hemispheres, the four resulting clusters (after 12 passes) were all combinations of objects from different cate-gories. On a pure descriptive level, a tendency for slightly better classification over the left hemi-sphere became evident: all animals clustered together (but also other categories fell in this cluster) and the category of ‘clothes’ was almost completely reconstructed. On the other hand, for the right hemisphere, there was no cluster, which contained more then two members of a specific category.

The hierarchical clustering method also allows following the dynamic process of clustering: more similar concepts cluster together earlier than less similar concepts. For the later time window (210-450 ms), it is notable that over the right hemisphere the initial six passes all yielded correct classifications.

For the earlier time window (120-210 ms), over both hemispheres only the first three passes result-ed in combinations of objects belonging to the same category. Comparing the time course of clus-tering for each category, there is no indication for a faster clusclus-tering of any category.

The output of the hierarchical clustering algorithm for each time window was further reduced by computing Symmetrical Uncertainty Coefficients (see Appendix C) for the clustering solution that resulted after running the algorithm 12 times. This was performed for different time windows and both hemispheres. Figure A-18 illustrates the outcome of this procedure for each 40 ms time win-dow and each hemisphere.

A.3.4 : Results - Hierarchical clustering

A slightly better categorization over the left hemisphere for later time windows is reflected in the higher uncertainty coefficients. The time course of the coefficients shows 'best' categorization at 370 ms over the left hemisphere.