Neuromagnetic correlates of categorization in normal controls and schizophrenic patients

(1)

Neuromagnetic correlates of

categorization in normal controls and schizophrenic patients

Dissertation

zur Erlangung des akademischen Grades des Doktors der Naturwissenschaften

an der Universität Konstanz,

Mathematisch-Naturwissenschaftliche Sektion, Fachbereich Psychologie

vorgelegt von Andreas Löw

Tag der mündlichen Prüfung: 08.11.2007

1. Referentin: Prof. Dr. Brigitte Rockstroh

2. Referent: Prof. Dr. Thomas Elbert

(2)

Danke

Die Tatsache, dass dies die letzten Zeilen sind, die ich dieser Arbeit hinzufüge, wird sicher alle die mich kennen, viel mehr erfreuen als umfangreiche Dankesworte.

Dennoch geht zuallererst und allermeist mein besonderer Dank an Brigitte Rockstroh und Thomas Elbert, für die stets hilfreiche und geduldige Betreuung dieser Arbeit, sowie die jahrelange

Förderung. An dieser Stelle auch ein dickes Dankeschön für die einzigartigen Rahmenbedingungen in Konstanz, ohne die diese Arbeit nicht möglich gewesen wäre.

Für Fragen zur Datenanalyse und Elektrophysiologie war Patrick Berg stets ein geduldiger und kompetenter Ansprechpartner – Danke Patrick!

Herzlichen Dank auch an alle Pfleger und Therapeuten der Station 33 im Zentrum für Psychiatrie Reichenau für die kontinuierliche Unterstützung und Erhebung der psychopathologischen Daten.

Vielen Dank auch an Annette Gomolla, die weit mehr als eine 'Hilfs'kraft bei der Datenerhebung und -reduktion war. Grossen Dank schulde ich allen Patienten und Kontrollprobanden, die mir durch ihre Teilnahme einen grossen Dienst erwiesen haben.

Darüber hinaus bin ich einer Vielzahl lieber Menschen für hilfreiche Diskussionen, Kommentare und vielem mehr sehr dankbar– hier sind insbesondere Shlomo Bentin, Yaron Silberman, Christian Wienbruch, Andreas Keil, Peter Lang und Margaret Bradley zu nennen.

Bei meinen Freunden und Kollegen in Konstanz und Gainesville möchte ich mich für meine oftmalige mentale und physische Abwesenheit entschuldigen. Auch wenn ich Euch zuweilen 'vergessen' zu haben schien, so wart ihr stets für mich da.

Voller Dankbarkeit bin ich meiner Familie für alle Unterstützung.

Andreas Löw

(3)

A Object categorization in healthy controls

A.1 Introduction

A fundamental human ability is to group objects (e.g. a sparrow and a nightingale) together into meaningful sets (e.g. birds). This treatment of physically non-identical stimuli as equivalent is a basic cognitive function and is referred to as categorization, while objects considered to be equivalent are grouped into categories. Any object can be categorized at different levels of abstraction or specificity (Jolicoeur et al., 1984; Murphy & Brownell, 1985). For example, a dog can be identified as a 'dog', but more generally it might be identified as an 'animal'. Or more specifically one might say it is a 'poodle'. Even if all objects can be categorized at multiple levels of abstraction, there seems to be one level of categorization with a privileged status, because (a) a single mental image can be formed, (b) category members share a similar shape, (c) similar motor action are used to in- teract with the category member and (d) subjects categorize faster at the basic level than at the super- or subordinate level in naming and category verification tasks (Rosch et al., 1976; Jolicoeur et al., 1984). This level is defined as basic level (e.g. dog), in contrast to the more specific subordinate (e.g. poodle) and the more general superordinate level (e.g. animal).

The infinite number of objects in the world, which are continuously processed by the visual and au- ditory systems out of the stream of stimuli, directs the attention to understand the underlying brain functions of categorization.

This thesis investigates human object categorization within the framework of 'cognitive neuro- science', focusing on empirically determined measures of brain activity associated with object categorization. As a background, basic mechanisms and anatomy of the visual system are briefly re- viewed.

(6)

A.1.1 : Introduction - The visual system

A.1.1 The visual system

The well-known pathways of the visual system are schematically presented in Figure A-1.

Light energy is converted into neural activity by specialized photoreceptors in the retina at the back of the eye. Most of the retinal ganglion cells project to the lateral geniculate nucleus (lgn) located in the dorsal thalamus, and from there to V1, the primary visual cortex in the occipital lobe (Brodman Area 17). V1 consists of a grid (1 mm by 1 mm) of retinotopically organized functional units called hypercolumns. Each of the hypercolumns analyzes information from one small region of the retina, where adjacent hypercolumns process information from adjacent areas of the retina. The fovea is over represented - there is about the same number of columns devoted to the fovea as to the periph- ery.

Beyond V1, two large-scale cortical processing streams have been identified (Ungerleider &

Mishkin, 1982; Ungerleider & Pasternak, 2004), which are illustrated in Figure A-2.

2

Figure A-1: Schematic view of the visual pathway from the eye to primary visual cortex (V1). Adapted from Polyak (1957).

(7)

Based on electrophysiological studies in monkeys, Ungerleider and Mishkin (1982) suggested that a) one path extends dorsally toward the parietal lobe serving the analysis of spatial information and localization of objects ('where'-system, dorsal stream) and b) one path projecting ventrally to the temporal lobe concerned with the identification of objects ('what'-system, ventral stream).

A more recent view by Goodale et al. (Goodale & Milner, 1992; Goodale & Humphrey, 1998;

Goodale & Westwood, 2004; Milner & Goodale, 1995) has offered a reinterpretation of the processing carried out by the two streams. This view does not distinguish between 'what' and 'where' and instead defines functional aspects of the two paths in terms of the transformations performed on the visual input: the ventral stream plays a major role in the perceptual representation of the world and the objects within, while the purpose of the dorsal stream is the control of goal-directed actions directed at those objects. In this approach information about objects like size or spatial location is processed by both streams in a unified fashion, differing only in the nature of the processing that each stream performs on the input.

These views are not mutually inconsistent and both assign an essential role to the ventral visual stream for the processing of objects. Along this path (from V1, through V2 and V4 to the posterior IT (also named area TEO in the monkey, see Figure A-3) the retinotopic organization becomes

Figure A-2: Schematic view of the dorsal and ventral visual processing streams.

(8)

coarser, and in the anterior IT (also known as area TE) no retinotopic organization is preserved (Tanaka, 1996, 1997).

A.1.2 Neuroanatomy of object processing and categorization: empirical findings

A.1.2.1 Findings in animals

The field of animal research on object recognition and categorization was opened by the pioneering work of Herrnstein & Loveland in pigeons (1964). Pigeons could not only discriminate between pictures containing a human being or not, they were also able to transfer this categorization ability to new sets of pictures.

In the 70s, animal research started to investigate object recognition and categorization in Macaques, a nonhuman primate with a high developed temporal cortex compared to nonprimates (Rolls, 2000).

The anterior inferior temporal cortex (area TE) in the macaque brain plays an important role in visual object recognition and research primarily focused on the registration of neural activity within this region. Monkeys with a bilateral ablation of the area TE showed deficits in tasks that required the visual recognition of objects (Dean, 1976).

In their pioneering work, Gross and Desimone (Desimone & Gross, 1979; Gross et al., 1972) 4

Figure A-3: Schematic view of the ventral visual processing stream.

(9)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings recorded activity from single cells in the visual areas of the temporal lobe and found neurons that responded best to complex visual stimuli, especially to faces. In the following, one of the obstacles in studying the processing of objects on a neuronal level was to determine the fine-grained stimulus selectivity of single neurons.

Based on previous work by Desimone et al. (1984), Tanaka et al. (Kobatake & Tanaka, 1994:

Tanaka et al., 1991) developed a reduction method to determine critical features for the activation of cells in area TE. In a first step, a large number of complex stimuli were used and single cells responding best to a given object were identified. Next, the most effective image for each cell was simplified step by step whilst maintaining maximal activation of the monitored cell. The simplest image still producing maximal activation was defined as the critical feature for a cell. These features for neurons in the area TE are more complex than features like orientation, size, color or sim- ple texture, which are represented by cells in V1 (Callaway, 1998). On the other hand these feature did not show enough specificity to represent natural objects on the level of single cells ('grand mother cell' hypothesis) as was suggested earlier by Barlow (1972).

Further evidence for the important role of moderately complex features in activating cells in area TE comes from a different method employed by Wang et al. (1996). Wang and coworkers also first determined the critical features of cells by recording from single neurons and then used optical imaging, a method which utilizes the relationship between the amount of reflected light and neuronal activity (see Gratton et al., 2003 for details). Compared to single cell recordings, optical imaging measures average neuronal activity from a rather wide cortical region. The activity spots (with a diameter of about 0.5 mm) in response to the presentation of a critical feature corresponded to the position of the previously recorded single cells. This topographical activity pattern suggests that cells responding to the same feature are clustered locally. Such a clustering of cells with similar feature selectivity in a columnar organization is also supported by results of Fujita et al. (1992). Fujita

(10)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings and coworkers vertically penetrated the cortex and for the whole thickness of the gray matter cells responding to similar features were found; in contrast, if nerve tissue was penetrated in oblique direction, cells with a similar selectivity were limited to short spans of distance of about 300 to 400 microns.

Though the majority of cells in the IT respond preferentially to the presentation of moderately complex feature, there is a portion of cells with maximal activity to facial stimuli (Desimone, 1991; Per- ret et al., 1982). These 'face cells' were used to examine the view-invariance of representations in the IT. While some of these 'face cells' were found to be relatively invariant in terms of size, contrast and spatial frequency of the stimuli (Rolls, 1992; Tovee et al., 1994), within the same region a larger number of cells with view-dependent responses have been found. For example, Perret et al.

(1985, 1992) reported neurons responding to the profile of a monkey face, but not to the frontal view of the same face. Booth and Rolls (1998) examined view-invariance using non-facial stimuli.

Plastic objects were placed in the cages of monkeys and after the animals were able to explore the objects, activity of neurons for different views of the same object was recorded. The majority of neurons responded only to some views of a particular object. However a subset of neurons fired exclusively to a particular object independent of its view.

In the aforementioned studies objects have been presented instantaneously and isolation. However, in real life the visual system must locate and identify objects in complex surroundings. Sheinberg &

Logothetis (2001) tried to account for this with an elegant experimental design. Target objects were embedded in natural scenes, while single cell activities as well as eye movements were recorded.

This allowed monitoring visually guided search processes while simultaneously recording the neural activity. In a separate condition the target objects were presented in isolated views without any context. The neural activity associated with the processing of the target objects in isolated or embedded views was found to be highly similar, corroborating the validity of previous studies that em-

6

(11)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings ployed only isolated objects. In the 'embedded' condition activity related to the target object was found shortly before the manual response of the monkeys. Detailed inspection of the time course revealed that this identity-related activity was sometimes observable right before target fixation, indicating that the information 'also was used to guide behavior'.

Most animal researchers have concentrated on the possible mechanisms of object representation, but little is known about the process of categorization of objects. While there is evidence that monkeys show categorization abilities in the wild (e.g. Seyfarth et al., 1980) and in the laboratory (e.g.

Delorme et al., 2000), only a few studies attempted to record neuronal activity during categorization tasks in the monkey.

Category-selective neurons have been found in the IT by Vogels (1999). In this study, monkeys were trained to categorize tree- and non-tree images and a set of neurons responded almost exclusively to trees during task performance. On the other hand, this category-specifity was limited to subsets of the category, i.e. no neurons were found that responded to all exemplars of trees, which makes it unlikely that categories are coded on the basis of single neurons.

Such category-selective activity in the anterior IT is shaped by visual features of an object as was shown by Sigala & Logothetis (2002). The authors trained monkeys to categorize line drawings of faces or fishes into two groups. The stimuli differed on four visual features (e.g. eye height, nose length), however only two of them were relevant for the categorization task. Neurons in the anterior IT showed enhanced activity for the relevant features compared to the non-relevant ones.

Tsao et al. (2003) used fMRI to detect category-related activity in the macaque brain evoked by pictures of hands, bodies, faces, fruits and man-made objects. Specialized patches of brain activity (extending from V4 to TE) were only found for faces and bodies, but not for any of the other categories. Recently, Gil-da-Costa and coworkers (2004) used PET to measure brain activity

evoked by the perception of species-specific vocalizations. Interestingly, early (V2, V3, V4) as well

(12)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings as higher-order (TEO and TE) areas along the ventral visual-processing stream showed activation for species-specific calls compared to non-biological sounds.

Freedman and coworkers (2001) used stimuli from the categories 'dogs' and 'cats' and blended these prototypes with a digital morphing system. The resulting stimuli varied in the relative amount of 'dog' or 'cat'. Monkeys performed a dog/cat-categorization task and activity in the prefrontal cortex (PFC) was registered. About one-third of the responding neurons in the lateral prefrontal cortices reflected the sample's category. Remarkably, most of these neurons were not influenced by the morphing level. For instance, neurons showed a similar response to images of dog prototypes (100 % dog-like) and cat-like dogs (60% dog, 40% cat), indicating a sharp “boundary” between categories.

However, the exact role of prefrontal structures in categorization tasks needs further exploration.

Thorpe & Fabre-Thorpe (2001) suggested that the IT and the PFC subserve different roles. Whereas the IT might deliver highly processed visual information, the PFC might be involved in decision making about category membership of an object.

In addition to the functional organization of object recognition and categorization, the time course of activity along the ventral visual pathway is another point of interest. Given that the visual processing of an object requires a sequence of several stages from the retina to the anterior temporal cortex, it is impressive that neuronal object-related activity in the anterior temporal cortex as the fi- nal purely visual processing stage was consistently found around 100 ms after stimulus onset across studies (e.g. Oram & Perrett, 1992; Sugase et al, 1999). This leaves just a few milliseconds for each processing stage along the visual processing stream and makes it very likely that the signal is processed in a feed-forward manner without any recurrent or feedback processing stages (Thorpe &

Fabre-Thorpe, 2001). Keysers et al. (2001) additionally have shown that face-selectivity of neurons in the temporal cortex is still preserved even at ultra-rapid presentation rates of 14 ms/image.

In sum, animal studies have shown that most neurons in the IT respond best to stimuli below the 8

(13)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings level of objects and this activity is mostly view-variant. There is no compelling evidence for the representation of object categories on the level of single neurons.

Moreover, studies have started to disentangle how objects might be represented in the brain. The representation of objects in the IT might be built up by distributed feature columns, each responding to a specific visual feature of the object (Wallis & Rolls, 1997; Wang et al.1996; Tsunoda et al., 2001).

However one should be cautious in transferring findings directly from animals to humans (Crick &

Jones, 1993). In particular, the capability for language in humans might be reflected in differences in the neuroanatomy of the temporal cortex as well as in the way in which objects are represented.

A.1.2.2 Findings in neuropsychological patients

While animal studies provide information about object-specific processing mechanisms on the neuronal level, the study of human subjects with deficits resulting from brain damage or disease (e.g.

herpes simplex encephalitis) adds insights into mechanisms of object processing and categorization in humans. Neuropsychological studies do not directly address the process of object categorization and rather attempt to explain the functional neuroanatomy of the semantic system and how conceptual knowledge is organized in the human brain.

Deficits in distinguishing semantic categories/domains have first been reported in detail by War- rington and Shallice (1984). The authors described four patients with impairment in the recognition of living compared to non-living objects following temporal lobe damage.

Such domain-specific semantic deficits for living objects have been described for numerous other cases including tasks like picture naming, word-picture matching or generation of definitions (e.g.

Basso et al. 1988; De Renzi & Lucchelli, 1994; Sartori & Job, 1988; Sheridan & Humphreys 1993;

for reviews see: Caramazza, 1998; Forde & Humphreys, 1999). The opposite pattern, a deficit in the

(14)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings processing of non-living compared to living stimuli, has been observed less frequently (e.g. Cappa et al., 1998a; Sacchet & Humphreys, 1992).

One of the most influential models to account for this domain-specificity is the ‘sensory/functional account’ first proposed by Warrington and Shallice (Warrington & Shallice, 1984; Warrington &

McCarthy, 1987). This model assumes that living and non-living objects differ in their sensory (primarily visual) and functional (non-sensory) properties and that these properties are represented in anatomically separable 'stores'. Visual properties of objects are more important for distinguishing between members of the living domain such as animals or fruits, while artifacts such as vehicles and tools are distinguished primarily by their functional properties. Consequently, damage to the sensory or functional subsystem will disproportionately affect the living and non-living domain.

However, detailed analyses of several patients showed that the boundary between the living and non-living domain is not so clear and patterns predicted by the sensory/functional theory could not always be found (Lambon Ralph et al, 1998; Moss et al., 1998; Moss & Tyler, 2000). For instance, Moss & Tyler (2000) described a patient with a domain-specific deficit for artifacts, but there was no evidence for any loss of functional compared to visual capacities as proposed by the

sensory/functional account.

Moreover, it was argued that category-specific deficits are just a by-product of uncontrolled stimulus factors as several studies did not account for frequency or familiarity differences between living and non-living things. Funnell and Sheirdan (1992) pointed out that some earlier studies were con- founded with artifacts of incomplete stimulus matching across domains. Other studies have selected stimuli more carefully and succeeded in replicating previous domain-specific findings (Kurbat, 1997). The shortcomings of the sensory/functional approach stimulated the development of alternative explanations for category-specific deficits. Caramazza and coworkers (Caramazza & Shelton, 1998; Caramazza & Mahon, 2003) have suggested an alternative theory. According to their Orga-

10

(15)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings nized Unitary Conceptual Hypothesis (OUCH), category- or domain-specific differences are not the product of sensual/functional object properties. Instead, they claim two fundamentals of objects: (a) the characteristic features of an object are highly correlated and (b) members of a superordinate category have many features in common. Because intercorrelated features are not distributed homoge- neously in the multidimensional space, there are denser regions with highly correlated features and these bundles of intercorrelated properties result in a categorical organization of object representations. Evolutionary pressures created a tripartite system for objects: animals as predators (but also source of food), plants (fruits and vegetables as food and medicine) and artifacts.

Support for this hypothesis comes from neuropsychological case studies of patients with selective sparing of the category of animals (Hillis & Caramazza, 1991) or selective damage of the categories of fruits and vegetables (Farah & Wallace, 1992). Thus, the OUCH model is able to explain some of the results that have challenged the sensory/functional theory.

Other neuropsychological studies reported specific deficits for body parts (Suzuki et al., 1997) or cows (Assal et al, 1984) and none of the above models can account for all neuropsychological data.

Taken together, neuropsychological lesion studies attempted to fractionate the semantic system with the most common distinction between living and non-living domain. However, case studies demon- strating deficits across these borders suggest a less coarse division.

A.1.2.3 Neuroimaging

Functional magnetic resonance imaging (fMRI) or positron emission tomography (PET) allow for the examination of the human brain with a high spatial resolution. These methods register brain activity by measuring cerebral blood flow or metabolism, and due to the rather slow changes of these indices their temporal resolution is low compared to EEG and MEG.

Numerous studies have been undertaken to explore possible neurophysiological correlates of the

(16)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings processing of different categories of objects and a synoptical table can be found in Appendix A.

A.1.2.3.1 Studies focusing on face processing

One subgroup of imaging studies focused on the neuroanatomical substrates of face recognition, which represents a particular class of objects, because a) single cells in animals have been found probably tuned to the processing of faces (Gross et al., 1972; Perrett et al., 1992), b) face processing was suggested to be innate (Goren et al., 1975; Valenza et al., 1996) and c) prosopagnostic patients have been described in the literature (Barton, 2003; Sergent & Signoret, 1992). A region in the (predominantly right) fusiform gyrus, the FFA (fusiform face area) is consistently activated by the processing of individual faces (e.g. Chao et al., 1999a; Gorno-Tempini & Price, 2001; Grill-Spector et al., 2004). While this was interpreted as clear and compelling evidence for a separable and distinct 'face module' in the brain (Kanwisher et al., 1999; Kanwisher, 2001), the face-specific responding of this area was criticized by Gauthier et al. and others (e.g. Gauthier et al., 1999; Tarr &

Gauthier, 2000). Subjects were trained to categorize novel stimuli (3-dimensional figures, so-called 'greebles') and functional brain imaging showed that the FFA becomes more active with increasing expertise in the categorization task (Gauthier et al., 1999). Expertise-related modulation of the FFA and an additional face-selective region in the right occipital lobe ('OFA') was further demonstrated in another study by Gauthier et al. (2000a). Bird and car experts performed a one-back-repetition judgment task including pictures of faces, birds, cars and common objects. The level of expertise was manifested in an interaction between FFA activity and the level of expertise. Bird experts showed increased activity in the FFA when they processed bird stimuli, but not car stimuli, while car experts showed the reversed pattern of results. This indicates that activation of the FFA is not limited to the processing of facial stimuli.

Furthermore, not only the level of expertise, but also the level of categorization modulated activity in the FFA (Gauthier et al., 1997, 2000b). Subjects categorized stimuli at the subordinate level (e.g.

12

(17)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings sparrow) as well as at the basic level (e.g. bird). Activity in the FFA was higher for categorization at the subordinate level.

A.1.2.3.2 Studies focusing on the processing of different object categories

While the aforementioned studies examined if the processing of faces is localized in a specialized area, other neuroimaging studies attempted to find category-specific brain regions for different object categories. Using PET, Martin et al. (1996) presented pictures of animals and tools and revealed category-specific activation. Naming of animals led to activation of the left medial occipital lobe, which was not activated by tools. On the other hand, naming of tools elicited significantly greater activity in the left premotor cortex as well as a region in the left middle temporal gyrus. In a similar study, Moore & Price (1999) used two categories of man-made objects (vehicles, tools) as well as two categories of natural objects (animals, fruits). Compared to natural objects, man-made objects evoked increased activity in the left medial extrastriate cortex. Natural objects elicited activation in the anterior temporal cortex (bilateral) and the right posterior middle temporal cortex. These results were found across two different tasks (naming; word-picture matching).

In another fMRI study, Chao et al. (1999b) presented pictures of animals and tools and got compa- rable results across several tasks (naming, delayed match-to-sample, passive viewing). Animals (relative to tools) activated the lateral region of the fusiform gyrus (including the occipitotemporal sulcus, whereas tools (relative to animals) led to bilateral activation in the medial fusiform gyrus (including the collateral sulcus). In addition, pictures of faces and houses were also included in the study. Facial stimuli clustered in the same region as animals and houses elicited a response in the 'tools region'.

A study by Aguirre et al. (1998) focused on 'buildings' and found an area sensitive for this category close to the 'houses area' of the study by Chao et al. (1999b). This specific activity was limited to the right hemisphere in the lingual sulcus. A similar region for famous and non-famous buildings

(18)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings (relative to faces) was also reported by Gorno-Tempini & Price (2001) in the parahippocampal/lingual cortex.

Ishai et al. (1999) found evidence for regions along the ventral visual pathway responding to different stimulus categories (faces: lateral fusiform gyrus, houses: medial fusiform gyrus, chairs: inferior temporal sulcus). However, these areas were not activated exclusively by the categories and there was considerable overlapping of areas activated by each category. In a subsequent paper (Ishai et al., 2000), the same distributed differential activation pattern was also found in the ventral occipital cortex, whereas no such consistent topological arrangement could be discerned in the dorsal part of the occipital cortex. Category-specific response patterns have also been examined in a recent study by Haxby et al. (2001). Pictures of faces, cats, and five different man-made objects were presented within a one-back repetition paradigm. Each object-category was found to be represented by a highly distinct activation pattern in the ventral temporal lobe, indexed by within-category correlation between the response patterns of odd and even runs of each object. This within-category similarity could even still be demonstrated after removal of those regions that responded maximally to a given object.

Given that there is considerable variance between the studies with regard to methods (PET, fMRI), tasks (passive viewing, naming, n-back, etc), stimulus materials (line drawings, black and white photographs, color photographs, words), aim of the study (differentiate between man-made objects and artifacts; find representations of single object categories) and differences in statistical analysis (Huettel et al., 2004) it is not surprising that results are rather inconsistent across studies. Although each study succeeded in identifying category-related areas, many of these ‘category-specific’ regions are not activated consistently across studies for a given object category.

However, some tendencies that a region is more likely activated by a specific category are evident across studies. Kanwisher and colleagues (Downing et al., 2001; Epstein & Kanwisher, 1998;

14

(19)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings Kanwisher, 2000) have proposed a limited number of specialized processing modules for the recognition of faces (fusiform face area), places (parahippocampal place area) and body parts (extrastriate body area) - other object categories are proposed to be processed by the remaining ventral temporal cortex. Rather than being localized in category-specific areas or modules, it was suggested by Haxby et al. (2001) that object categories are organized in distinct response patterns all across the occipito-temporal cortex (‘ object form topography’). Each object category has its unique distributed “signature” across the whole occipito-temporal cortex, which is replicable if the specific exemplar or the view-point are changed (Spiridon & Kanwisher, 2002).

In sum, neuroimaging studies revealed that the categorical structure of the semantic system is related to the ventral occipito-temporal cortex. Objects might be represented by a discrete functional topography of cortical regions along the ventro-temporal cortex. The variance between studies that attempted to localize category-specific areas is considerable.

A.1.2.4 EEG and MEG studies

The high temporal resolution of EEG and MEG in the range of milliseconds makes them an ideal tool for 'on-line'-monitoring of cognitive processes. This allows one to uncover processes underlying task performance and to track their time course.

The N400, a negative component in the human ERP peaking around 400 ms after presentation of a stimulus, was used frequently as an electrophysiological index of semantic processing. The N400 can be elicited by semantically inappropriate words in sentences (Kutas & Hillyard, 1980). More- over, the scalp distribution of the N400 differs between concrete and abstract words (Kounios &

Holcomb 1994), however no study related the N400 to the categorization or representation of objects.

Early ERPs differences between words, faces and line drawings of objects were found around 150

(20)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings ms after stimulus onset, with words and faces being more positive than objects (Schendan et al.

1998). Different word categories (animal names, proper names, verbs and numerals) were examined by Dehaene (1995) in a word classification task. Around 250-280 ms after stimulus onset, animal names and verbs elicited a left temporo-parietal negativity, proper names elicited a left inferior temporal negativity and for numerals a broadly distributed positivity over both hemispheres was found.

In a similar time window different classes of words showed differences in the ERP, e.g. nouns and verbs (Pulvermüller et al., 1999) or different kinds of action verbs (Hauk & Pulvermüller, 2004).

Categorization of objects at different levels of abstraction was examined by Tanaka et al. (1999).

Subordinate level (compared to basic level) categorization was reflected in an enhanced negativity over left posterior regions around 140 ms after stimulus onset. This enhanced negative deflection for subordinate level categorizations might reflect additional perceptual processing as suggested by cognitive studies by Jolicoeur et al. (1984). In contrast, starting about 350 ms after stimulus onset a negative deflection at frontal EEG channels for superordinate categorization was observed. Consis- tent with an assumption by Jolicoeur et al. (1984), this might reflect additional semantic processing necessary for superordinate categorization. Using a visual oddball design with subordinate, basic or superordinate target objects, Large et al. (2004) found differences between superordinate and basic level categorizations in a similar time window (320-420 ms) in the visual processing stream. How- ever, subordinate categorizations differed significantly from basic level categorizations only during later stages (450-550 ms). Kiefer (2001) compared pictures and words representing natural and arti- factual objects using ERPs to assess the contribution of perceptual and semantic processes in a superordinate categorization task. Images of natural objects elicited an enhanced negativity in a time window 160 to 200 ms after stimulus onset compared to artifacts. Furthermore, ERPs of natural objects and artifacts differed in a later time window (300 to 500 ms). Independent of the input modali- ty (picture vs. word), natural object showed less negativity at posterior electrodes, while for artifacts

16

(21)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings a reduced negativity was found over anterior sites.

Bentin et al. (1996) described a face-selective negativity around 170 ms after onset of a stimulus (N170), which was absent when other animate or inanimate stimuli were presented. However, subsequent studies reported N170 responses following the presentation of non-facial stimuli like birds, dogs or novel objects (Rossion et al. 2002a; Tanaka & Curran 2001; Tanaka et al 1999). Though smaller in amplitude than the N170 to faces, the findings that the N170 amplitude can be modulated by non-facial stimuli challenges the “face-specifity” hypothesis of the N170. Consequently, it was suggested that the N170 might rather reflect “subordinate level expertise” (Rossion et al. 2002b).

Supporting this view, an enhanced N170 was found when experts categorized objects in their domain of expertise compared to objects outside of their expertise (Tanaka & Curran 2001).

Only a few studies using MEG to investigate neuronal correlates of object or face processing have been published. Liu et al. (2000; 2002) examined face perception with MEG. Results identified an early component (M100) that discriminated between facial and non-facial stimuli and correlated with successful categorization as a ‘face’. The M100 was followed by an M170 (the magnetic coun- terpart of the N170 in the EEG) that indexed identification of the individual identity of a face. Such an early difference between faces and building (around 110 ms) was recently also reported using EEG (Herrmann et al., 2004).

Dipole modeling of the MEG response to facial stimuli by Sato et al. (1999) identified bilateral sources in the lingual or fusiform gyrus at a latency of 160 ms. In contrast, the prominent MEG deflection to control stimuli (pictures of 'scenes') had a latency of about 300 ms and originated most likely from the right parahippocampal gyrus and the right parieto-occipital junction. Differences between facial and non-facial stimuli around 160 ms were further found by others (Swithenby et al., (1998), Watanabe et al., 1999; Halgren et al., 2000; Sams et al., 1997).

In sum, EEG and MEG studies revealed differences between different (mostly facial vs non-facial)

(22)

A.1.2 : Introduction - Neuroanatomy of object processing and categorization: empirical findings classes of objects starting around 170 ms after stimulus onset. ERP differences between different levels of categorization have been reported mostly for later time windows (>300 ms), but were found as early as 140 ms after onset of stimuli.

A.1.3 Object categorization: models

Possible neuroanatomic substrates of object representations and mechanisms underlying object categorization have been studied with different methods in animals and humans.

Though results of functional imaging studies frequently are interpreted as clear evidence for the existence of object-specific 'modules' in the temporal cortex, this conclusion is not sufficiently covered by the data. Alternative models, differing in their degree of modularity, are also able to account for the reported findings and will be described in the following.

A.1.3.1 Category-based models

This widespread view under fMRI researchers claims more or less independent processing modules for each category of objects. Strongest support for this model results from studies focusing on face processing, which found evidence for a face-selective region in the fusiform gyrus ('fusiform face area'). In addition, areas activated distinctly by different categories like houses or tools were found.

The major problem with this view is that there exists a nearly endless list of basic-level categories (birds, dogs, cats, cars, ships, airplanes, bicycles, vegetables, fruits, chairs, tables, shirts, violins, guitars, pianos, lamps, houses, bridges, books, spoons, forks, knives, bottles and so on), but no theoretical approach can explain how such a large number of categories could be implemented as

“modules” on a cortical level.

A.1.3.2 Process-based models

This view proposes 'brain modules' dedicated to specific computations like fine-grained visual 18

(23)

A.1.3 : Introduction - Object categorization: models

discriminations or extraction of similarities. As objects differ in their processing demands, 'category-specific' brain areas can emerge by subtracting activity associated with two different objects from each other. For example, categorizing an animal might have higher demands on a 'visual discrimination module' than recognizing a tool, and after subtracting 'tools' from 'animals' the difference can be misinterpreted as an 'animal area'.

A.1.3.3 Feature-based models

These models posit that the neural representation of object categories is distributed throughout the extrastriate cortex. The object-form hypothesis postulates that the ventral temporal cortex consists of a “continuous, topologically arranged representation of information about object-form in which the representations of different categories are distributed and overlapping” (Haxby et al. 2000). In the ventral temporal and occipital cortex, different categories of objects evoke distinct patterns of activity (Ishai et al. 2000). However, the exact nature of the features or “object forms” being represented is still a matter to be defined.

A.1.4 Aim of the study and hypotheses

As described above, numerous studies have been undertaken to disentangle the brain mechanisms and areas underlying the categorization of different classes of objects. Though knowledge is accu- mulating, previous studies have a number of shortcomings. This study attempts to account for a number of the limitations of previous studies.

1. Stimuli: It is possible that at least some previous studies have limitations regarding the stimuli used. Different classes of objects (e.g. faces and buildings) might have differed in physical stimulus properties (spatial frequency, brightness, etc) or familiarity. This might have evoked differences in brain activity across categories. Furthermore, studies employing only two different classes of stimuli (e.g. animals and tools) have only limited value with regard to generalization to different cate-

(24)

A.1.4 : Introduction - Aim of the study and hypotheses

gories of objects. In this study, a stimulus set from 4 different categories was used and it was attempted to keep stimuli identical with regard to task-relevance and physical properties.

2. Category-specific activity: It is hypothesized that different categories of objects, will differ in their neuromagnetic response.

3. Spatial aspects of categorization: Compared to EEG, MEG has a higher spatial resolution. Neu- roimaging studies have shown category-specific activity in occipital and temporal regions of the brain. It is hypothesized that source modeling of differences between objects in the neuromagnetic response will reveal activity in brain areas previously shown to be involved in category-specific object processing.

4. Temporal aspects of categorization: As the majority of studies were undertaken with fMRI or PET, knowledge about temporal aspects of categorization is rather limited. In this study, magneten- cephalography with its high temporal resolution was used to uncover temporal properties of the categorization of objects. Based on previous research with EEG, it is hypothesized that around 170 ms after onset of a stimulus category-specific activity will be evident.

5. Previously shown categorical dissociations are not sufficient to demonstrate semantic categorization in the brain. In order to do so, it is necessary to reveal associations between different exemplars belonging to the same category. Consequently, in this study a design, which permitted the disentan- glement of associations between objects as well as dissociations across categories, was employed. It is hypothesized that associations between objects of the same superordinate category will be higher than between objects belonging to different superordinate categories.

6. In addition to category-specific findings, differences between the living and non-living domain are reported in the neuroimaging and neuropsychological literature. It is hypothesized that differences between the living and non-living domain will be reflected in the neuromagnetic response.

20

(25)

A.2 : Methods - Methods

A.2 Methods

A.2.1 Subjects

10 healthy controls subjects (5 women, 5 men; mean age: 26.2 years, range: 23 to 35 years) were examined in this study. All subjects were right-handed with a mean handedness score of 93.7 (SD:13.4) as indexed by a modified version of the Edinburgh Handedness Questionnaire (Oldfield, 1971).

Participants reported that they have never been treated for any neurological or psychiatric disease and that they are free of any psychotropic medication.

All subjects were informed about the experiment and gave their written consent. The amount of 25 DEM (about 13 EUR) was paid for participation.

A.2.2 Task

A.2.2.1 Stimuli

Pictures of four different super-ordinate categories were selected. Two of the categories were man- made objects (clothes, furniture), and two categories consisted of natural objects (forest animals, flowers). Each of the super-ordinate categories was represented by four different base-level concepts (see Table A-1). Examples of the stimuli can be found in Appendix B.

Object 1 Object 2 Object 3 Object 4

Clothes Pants Shirt Jacket Shoe

Furniture Chair Table Sofa Wardrobe

Forest Animals Fox Wolf Deer Bear

Flowers Sunflower Rose Tulip Orchid

Table A-1: Overview of objects used in the study.

For each of these 16 base-level concepts a set of 60 different exemplars was constructed, resulting in a total of 960 pictures. Pictures were gathered from the Internet (online shops, image databases,

(26)

A.2.2 : Methods - Task

etc.) and processed with standard image processing software. After processing all stimuli were 8- bit-gray-scaled, 300x360 or 360x300 pixel sized images of the object with a unicolored gray background.

A.2.2.2 Procedure

Stimuli were presented in randomized order for 750 ms with a constant stimulus onset asynchrony (SOA) of 1500 ms. Subjects where required to indicate for each stimulus by pressing one of two buttons with the middle or index finger (counterbalanced across subjects) of their right hand, if it was a man-made or a nature-made object. The total of 960 trials was presented in three runs separat- ed by two short breaks. Prior to the experiment, 16 practice trials were performed to make sure that the task was understood and to acquaint subjects with the presentation rate of the stimuli. The dura- tion of the experiment including explanation of the procedure, attachment of the ECG and EOG sensors and coils, instruction and the experimental runs was approximately 2 hours.

A.2.3 Recordings

Recordings were done in supine position with stimuli presented to the ceiling of the experimental room via a mirror system. The neuromagnetic response was registered with a Magnes 2500 Neuro- magnetometer system (4D Neuroimaging, San Diego, USA). The system was installed within a magnetically shielded room and consisted of an array of 148 uniformly arranged signal detectors covering the entire cranium. Data were recorded continuously with a sampling rate of 678 Hz, a bandwidth of 0.1 to 200 Hz and stored for offline analysis. ECG and EOG were recorded with the same parameters for offline removal of ocular and cardiac artifacts.

To determine the relative position of the head within the array of sensors, five coils were attached to each subject’s head and fed with a small current before and after each experimental run. This

allowed to control for the subjects’ head position during the later analysis and ensured that subjects 22

(27)

A.2.3 : Methods - Recordings

did not move within and between experimental runs.

A.2.4 Analysis

A.2.4.1 Behavioral response

Response latency was determined by the difference between the trigger signals recorded for the onset of a picture and the button press. Median response latencies of correct responses were determined for each object and were compared across objects and categories by means of Analysis of Variance (ANOVA) with the within-factors CATEGORY and OBJECT, respectively.

A.2.4.2 Data processing and artifact correction

In a first step, external noise was removed by applying a noise-reduction procedure. This procedure eliminates any correlation that the MEG sensors have with any of the reference magnetometer.

BESA2000 software (MEGIS, Munich) was used for all following data processing steps.

The continuously recorded data of each block were combined into one file and ocular and cardiac artifacts were corrected. This correction was based on the MSEC (multiple source eye correction) approach developed by Berg and Scherg (1994). This method uses empirically determined compo- nents to estimate artifact related ocular or cardiac activity from the topographic information of all sensors. For the correction of eye blinks one source vector was used, while for the removal of cardiac artifacts two independent source vectors were applied.

Next, data were converted to epochs of 1000 ms starting 100 ms prior to stimulus onset. Correction of possible baseline differences was achieved by subtracting the mean amplitude of each channel during the pre-stimulus period from each epoch. Each data set was visually inspected to detect sensors with a high amount of artifacts. Artifact-loaded channels were replaced by interpolated data based on spherical splines (Perrin et al., 1989). For each channel the mean amplitude during each

(28)

A.2.4 : Methods - Analysis

epoch was computed to estimate the goodness of data and to identify any trials with remaining artifacts due to movement or other sources of noise, which were excluded from further analysis by set- ting an artifact rejection threshold.

For every subject, the event-related magnetic field was averaged for each of the 16 base-level concepts and the individual grand mean across all concepts was subtracted from the grand mean of each single concept to remove activity not specifically related to it.

Trials in which the response was erroneous were not included in the averages.

A.2.4.3 Source space analysis

An important issue in the interpretation of electro- and magnetophysiological data is the inference from the recorded signals to the underlying generators in the brain. This problem, known as the 'in- verse problem' (Helmholtz, 1853), is ambiguous: it has no unique solution. That is, a given magnetic field recorded can be explained by an infinite number of different configurations of intracranial sources. A solution can only be found by making assumptions about these sources and volume con- duction (Fender, 1987).

In this study, the source current distribution was estimated with a distributed source model, the Minimum Norm Estimate (MNE). While single- or multiple dipole models presuppose that brain activity is localized to one or several small areas, the MNE represents the best solution for the current if only little a priori information about the source is available (Hämäläinen & Ilmoniemi, 1994;

Michel et al., 2004; Hauk, 2004). The MNE models the cortical source space as a dense grid of n independent, single equivalent current dipoles located on a sphere.

A measured magnetic field can be described by a vector B:

B=L P + ε

where L is a m x n lead field matrix, describing the sensitivity pattern of each of the m magnetome- 24

(29)

ters to the primary current P and ε is a noise component.

Even in the case of perfectly accurate data (with ε=0) this system has no unique solution. Several methods to solve the this problem have been suggested. Here, a unique solution was achieved by minimizing the ordinary quadratic L2 norm of the current distribution, or in other words by select- ing the current distribution with the shortest current vector capable of explaining the measured signals.

As the presence of noise can lead to distortions or unstable estimates of real activity, spatial regularization is necessary and Thikhonov-Philips regularization (Bertero et al., 1988) was applied during the pseudo-inversion of matrix L, which results in suppression of currents with poor coupling to the sensors. This means that the MNE does not represent exactly the measured signals, but deviations are within the range of measurement errors (Sarvas, 1987). The Minimum Norm model consisted of a concentric sphere with a radius of 0.6 relative to the average radius of the cortex. The source space contained a uniformly distributed grid of hypothetical dipoles, each with three orientations at the 655 locations. For illustration and further analysis 197 of the locations covering the whole sphere were selected. All results reported on the following are based on the norm activity (i.e. vector length) of the three orthogonal dipoles at each of the 197 locations. The MNE was computed with MATLAB-based in-house software developed by Olaf Hauk.

A.2.4.4 Spatio-temporal correlations

A.2.4.4.1 Method

As described in the introductory part of this thesis, objects might be represented in the brain in form of specialized areas or in distributed networks with a unique neural signature for a particular base level concept. Given that category-related activity is reflected in the distribution of magnetic fields, base level concepts of the same superordinate category should be more similar than base level con-

(30)

cepts of different superordinate categories. Determining spatio-temporal correlation coefficients permits to compare brain activity evoked by the different base level concepts.

The Pearson correlation coefficient r was computed for each pair wise combination of the 16 base- level concepts. Different subsets of dipole locations (combined to 17 areas, see Figure A-4) and time windows were used. Time windows had a width of 40 ms each and data were analyzed from 90 to 450 ms after stimulus onset (resulting in 9 time windows: 90-130 ms, 130-170 ms, etc).

The correlations were computed after all data points within an area (each including of 8 dipole locations) and time window were represented by a single vector for each concept. As Pearson’s r is not normally distributed, the values were Fisher z transformed to the variable z’:

z’=0.5 * ln(1+r)-ln(1-r)], where ln is the natural logarithm.

26

Figure A-4: Schematic representation of the distribution of 17 areas clustered for analysis. Each of the areas contained 8 dipole locations.Top view, nose up.

(31)

The distribution of z’ is normal with a standard error of: σz’= 3 1

− N .

The z’-transformed coefficients were averaged a) for pairs of base level concepts within the same super-ordinate category, b) for pairs of base-level concepts from different categories.

A.2.4.4.2 Contrast score, Topography of correlations

For graphical illustrations of the spatial distribution of differences between the correlations within and between superordinate categories for each of the areas a contrast score was determined. This contrast score consisted of the sum of the differences between each within-category correlation of a given category and the between-category correlations with the other categories. For each area one contrast score was obtained. Scores were 'mapped' using the center of gravity of each area as an approximate location of the area on the scalp.

A.2.4.5 Unsupervised classification

The primary goal of classification methods is the clustering and visualization of high-dimensional data, based on similarities among different patterns. In this study two different clustering algorithms were applied¹:

a) Self-organizing topographical mapping as a neural network approach b) Hierarchical clustering as a statistical classification approach

A.2.4.5.1 Hierarchical clustering

Hierarchical clustering techniques organize data in nested sequences of groups, which can be displayed in the form of dendrograms or trees. In this study, a hierarchical algorithm clustered the two most similar high-dimensional vectors in sequential passes (Duda et al., 2000). During each pass the number of vectors is reduced by one by replacing the two composing vectors by the mean of them.

1I am grateful to Prof. Dr. Shlomo Bentin and Dr. Yaron Silberman for providing algorithms and assistance in

(32)

Similarity was defined as the Euclidian distance between two data vectors.

This procedure is deterministic and enables to follow the dynamic formation of clusters.

Hierarchical clustering was performed on the grand average for each base-level concept for the time windows 120-210 ms and 210-450 ms. Each concept was represented by a vector including all data points within a time window and all dipole locations within area. The clustering was performed independently for each hemisphere. Selection of these time windows was based on visual inspection of the data and theoretical considerations. While during the first time window the initial stages of visual processing are completed (Thorpe & Fabre-Thorpe, 2001), the second one is most likely associated with semantic activity.

In addition, Symmetric Uncertainty Coefficients (see Appendix C) were computed (for the outcome of the hierarchical clustering after 12 passes) to quantify the results of the hierarchical clustering procedure.

A.2.4.5.2 Self-organizing topographical mapping (SOM)

SOM is a particular type of neural network, using an unsupervised stochastic iterative learning algorithm to discover patterns and categories in the input data by clustering high-dimensional vectorial data.

For each iteration, a data vector is randomly chosen (without replacement) and compared to a set of vectors of the same dimension organized on a (usually) two-dimensional grid. At each comparison, the vector on the grid that is the most similar to the data vector (based on a high-dimensional metric function) is algorithmically modified, becoming even more similar, and so are its neighboring vectors. The size of this neighborhood decreases as the number of iterations increases. Upon conver- gence of the algorithm, the original data vectors are mapped on the grid such that vectors that are similar in their properties are placed at topographically nearby nodes (Kohonen, 2001). In the

28

(33)

present study, the map was two-dimensional (10 by 10 nodes) and the 16 original vectors were mapped onto it to form geometrically meaningful clusters. For illustration purposes, the U-MAT algorithm was implemented on the organized map (Ultsch, 1993). This algorithmcolours the organized map using a grey scale, such that the actual similarity between the high-dimensional vectors (concepts) that are represented by the nodes in the two-dimensional map is reflected by colour dens- ity: Darker demarcation space reflects larger high-dimensional distance between the adjacent nodes.

The input to the SOM consisted of the same data vectors as for the unsupervised hierarchical clustering. The computation of the SOM is a stochastic process; several runs of the algorithm with the identical input may result in slightly different output. To account for this, the algorithm was applied four times to each input data set and based on visual inspection the ‘best’ map was selected.

A.2.4.6 Statistical analysis

The General Linear Model (GLM) as implemented in the SAS System for Windows (SAS Institute Inc., Cary, NC ) was used to perform statistical analysis. Versions 6.12 and 8.02 were used.

Statistical analyses were performed on time windows of 40 ms from 90 to 450 ms after stimulus onset to follow the time course of categorization. For each window, statistical comparisons were carried out by repeated measurement analysis of variance (ANOVA).

Tests were performed independently on different transformations of the data: a) the ‘raw” neuromagnetic fields, b) the MNE (after the previous subtraction of the overall grand average) and c) the z’-transformed correlation coefficients. Sensors (or dipole locations for the MNE) were grouped into regional means prior to analysis as illustrated in Figure A-4.

Based on findings presented in the introduction of this thesis and published reviews (Grill-Spector, 2003), category-related differences were expected in occipital and temporal brain areas and the areas 1,2,3,4,5,6,9,10,11,12,15 and 16 (Figure A-4) were included in statistical tests.

(34)

The within-factors CATEGORY or OBJECT tested for differences between the stimuli. Tests for differences in the topographical distribution were performed with the within-factors HEMISPHERE (right [areas 1,2,5,6,11,12], left [areas 3,4,9,10,15,16]), GRADIENT (central [areas 2,3,6,9,12,15], posterior [areas 1,4,5,10,11,16]) and DEPTH (3 levels in dorso-ventral direction: dorsal [areas 1,2,3,4], medial [areas 5,6,9,10], ventral [areas 11,12,15,16]). To control for Type I errors associated with inhomogeneity of variance, the degrees of freedom (df) were decreased using the Green- house-Geisser epsilon (ε) for all repeated measures with more than one df (Greenhouse & Geisser, 1959). Corrected P-values are reported.

Hierarchical clustering and SOM were performed on the group data and are therefore interpreted based on visual inspection.

A.3 Results

A.3.1 Behavioral data

A.3.1.1 Response times (RTs)

The average of the median response times across all objects was 557 ms (SD: 61.6 ms).

Figure A-5 illustrates the response times for each superordinate category (left panel) and each base- level concept (right panel).

Figure A-5: Median response times in milliseconds (ms) for each of the four categories (left panel) and for each of the 30

sixteen objects (right panel). Error bars represent one standard error.

(35)

A.3.1 : Results - Behavioral data

RTs did not differ between superordinate categories (CATEGORY: (F(3,27)=2.23, p=.16). Howev- er, the analysis of response times on the level of individual objects yielded a significant difference between objects (OBJECT: F(15,135)=3.18; p<.05). Post-hoc paired t-tests showed that (after ad- justing the α-level according to Sidak) the following pairs of objects differed significant: orchid- sunflower, shirt - chair, chair-sofa, shirt-table and shirt-wardrobe.

A.3.1.2 Error rates

The numbers of correct responses are represented in Figure A-6.

Subjects performed the task with an average error rate of 6.1 % (SD: 3.2 %). Neither categories nor concepts did differ with respect to error rate (CATEGORY (F(3,27)<1; OBJECT (F(15,135)<1).

Response times and error rates were uncorrelated (Pearson’s r =.00 for objects, r =.01 for categories), indicating that there was no speed-accuracy-trade-off.

A.3.2 Evoked magnetic fields

A.3.2.1 Signal space

The individual overlay plots of the 148 magnetometers (Figure A-7), averaged across all conditions, show a pronounced deflection starting around 100 ms and peaking at approximately 170 ms after

Figure A-6: Number of correct responses for each of the four categories(out of 240; left panel) and for each f the sixteen objects (out of 60; right panel). Error bars represent one standard error.

(36)

A.3.2 : Results - Evoked magnetic fields

stimulus onset. Additionally, some of the subjects show another activity peak about 60 ms later.

The Grand Average across all subjects (see Figure A-8) illustrates that the early peaks show their maximum deflection over posterior regions. A more central peak around 600 msec is most likely associated with the required motor response and theredoes does not reflect categorical processing of

Figure A-8: Grand Average waveforms for the four superordinate categories. Each of the 148 waveforms corresponds to 32

the approximate position with the sensor array (top: anterior, bottom: posterior)

Figure A-7: Overlay of all 148 MEG channels. Each overlay plot represents the neuromagnetic activity of one subject averaged across all concepts.

(37)

A.3.2 : Results - Evoked magnetic fields the stimuli.

The average evoked magnetic fields of the four categories show highly similar waveforms and to- pographies. A distinct topographical distribution is reflected in the interaction HEMISPHERE x DEPTH x GRADIENT across time in the intervals between 170 and 450 ms (170-210 ms:

F(2,18)=8.96; p<.01 ; 210-250 ms: F(2,18)=15.84. p<.01 ; 250-290 ms; F(2,18)=11.58, p<.01 ; 290-330 ms: F(2,18)=11.71, p<.01 ; 330-370 ms: F(2,18)=12.82, p<.01 ; 370-410 ms:

F(2,18)=12.12, p<.01; 410-450 ms: F(2,18)=13.71, p,.001). For each sensor group and time window, the mean magnetic flux is illustrated in Figure A-9.

Across time windows, the topographical distribution is similar. Over the left hemisphere, the magnetic flux differed between regions as indicated by the interaction DEPTHxGRADIENT (LH:

210-250 ms: F(2,18)=4.74, p<.04, 290-330 ms: F(2,18)=5.22, p<.03; 330-370 ms: F(2,18)=4.57, p<.04; 370-410 ms: F(2,18)=3.95, p<.06; 410-450 ms: F(2,18)=3.23, p<.08; all other time windows:

n.s.). However, separate contrasts between levels of DEPTH for posterior and central regions, did not yield any significant differences. Over the right hemisphere, the interaction DEPTHxGRADI- ENT was highly significant for several time windows (170-210 ms: F(2,18)=8.39, p<.01); 210-250 ms: F(2,18)=11.14,p<.01; 250-290 ms: F(2,18)=12.12, p<.01, 290-330 ms: F(2,18)=10.22, p<.01;

330-370 ms: F(2,18)=8.77, p<.01; 370-410 ms: F(2,18)=5.97, p<.03; 410-450 ms: F(2,18)=7.66,

Figure A-9: Illustration of the HEMISPHERExDEPTHxGRADIENT interaction. Four of the time windows are displayed.

Upper row: central areas, lower row: posterior areas; black line: right hemisphere, blue line: left hemisphere.

(38)

A.3.2 : Results - Evoked magnetic fields

p<.02). Post-hoc tests revealed that over the right central area for later time windows there was less magnetic flux over the dorsal and ventral area compared to the region between (DEPTH: 330-370 ms: F(2,18)=5.58, p<.03; 370-410 ms: F(2,18)=4.47, p<.04; 410-450 ms: F(2,18)=6.35, p<.02).

Right posterior, starting 210 ms after picture onset there was significantly higher positive flux over the dorsal compared to the ventral area (DEPTH: 210-250 ms: F(2,18)=12.37,p<.01; 250-290 ms:

F(2,18)=12.61, p<.01; 290-330 ms: F(2,18)=6.27, p<.03; 330-370 ms: F(2,18)=6.42, p<.03;

370-410 ms: F(2,18)=6.82, p<.02; 410-450 ms: F(2,18)=5.99, p<.03; contrasts between the dorsal and ventral area: all F(1,9)>7.0, all p<.03).

Furthermore, an interaction CATEGORY x HEMISPHERE x DEPTH was evident during early as well as later time windows (130-170 ms: F(6,54)=4.13, p<0.01, 170-210 ms: F(6,54)=3.12, p<.04, 250-290 ms: F(6,54)=3.79, p<.03, 290-330 ms: F(6,54)=3.25, p<.05); 330-370 ms: F(6,54)=3.70, p<.03, 370-410 ms: F(6,54)=2.97, p<.05, 410-450 ms: F(6,54)=3.65, p<.03).

Subsequent post-hoc tests revealed that this three-way interaction was due to differences between categories in the most ventral region and was more pronounced over the left hemisphere: the

interaction between CATEGORY and HEMIPSHERE over ventral areas was significant for all time windows starting 130 ms after stimulus onset (all F(3,27)>4.20, all p<.02).

Figure A-10 illustrates the mean magnetic flux for each category and time window for the ventral area over both hemispheres.

Moreover, for these later time windows (from 250 to 450 ms) an additional interaction CATEGO- RY x DEPTH x GRAD was found. Post-hoc testing revealed that categories differed in the posterior part of the ventral stream. The mean magnetic flux for each category as well as significant contrasts is shown in Figure A-11.

34

Neuromagnetic correlates of categorization in normal controls and schizophrenic patients