Systems that classify movements according to function

Among the coding systems that classify body movements according to their function, there are many gesture coding systems. As exposed in Chapter 2, ges-ture is class of body movements defined by its specific function.

One of the most influential systems for hand and head gestures has been devel-oped by the anthropologist David Efron (1941), who conducted research on tra-ditional and assimilated Eastern Jews and South Italians. Efron combined the analysis of the function of gesture with a spatio-temporal analysis and an inter-locutional analysis, which includes between-subjects phenomena. The spatio-temporal analysis refers the visual appearance of the gesture movement. It in-cludes the categories radius, form, plane, bodily parts involved in gesticulation, and tempo. The interlocutional analysis focuses on the topographic-gestural rela-tionships between speaker and auditor. It includes the categories familiarity with the physical person, simultaneous gesturing, conversational grouping, and ges-turing with objects. The analysis of the function of the gesture refers to referen-tial aspects: "A gestural movement may be ‘meaningful’ by (a) the emphasis it lends to the content of the verbal and vocal behavior it accompanies, (b) the connotation (whether deictic, pictorial, or symbolic) it possesses independently from the speech of which it may, or may not, be an adjunct. In the first case its

"meaning" is of logical or discursive character, the movement being, as it were, a kind of gestural portrayal, not of the object of reference, or "thought", but of the course of the ideational process itself (i.e., a bodily reenactment of the logi-cal pauses, intensities, inflections, etc. of the corresponding speech sequence).

This type of gesture may in turn be (a) simply baton-like, representing a sort of

"timing out" with the hand the successive stages of the referential activity, (b) ideographic, in the sense that it traces or sketches out in the air the "paths" and

"direction" of the thought pattern. The latter variety might also be called logico-topographic or logico-pictorial. In the second case the "meaning" of the gesture is "objective", and the movement may be (a) deictic, referring by means of a sign to a visually present object (actual pointing), (b) physiographic, depicting either the form of a visual object or a spatial relationship (iconographic gesture), or that of a bodily action (kinetographic gesture), (c) symbolic or emblematic, representing either a visual or a logical object by means of a pictorial of a non-pictorial form which has no morphological relationship to the thing repre-sented.” (Efron, p. 96, 1972). To summarize, the function-oriented analysis comprises the main values discursive and objective gestures. Discursive gestures

include the values baton (emphasizing the beat pattern of the speech) and ideo-graphic (sketching a thought pattern). Objective gestures comprise the values deictic (pointing to a real or imagined object or indicating a direction), physi-ographic with the two subvalues iconographic (depicting a form) and kineto-graphic (depicting a movement), and emblematic gestures (conventional signs having specific linguistic translation). As Efron conducted an independent spa-tio-temporal analysis of gestures, his descriptions of the referential gesture val-ues are accompanied by precise descriptions of the radius, form, plane, bodily parts involved in gesticulation, and tempo. However, his distinction between the values physiographic and ideographic is based on the analysis of the verbal statement that accompanies the gesture.

While Efron did not assess interrater agreement, he applied a fourfold method for analysing the data including direct observation, sketches by a painter, rough counting, and motion pictures studied by observation and judgments of naive observers and graphs and charts. Recent studies, applying Efron's referential values, with the exception of ideographic gestures, report good interrater agree-ment (e.g. Lausberg et al., 2000).

The psychoanalysts and psychiatrists Freedman and Hoffman (1967) designed a system to analyze the patients' hand movements in clinical interviews. The elaborated version of the coding system distinguished the two main values "ob-ject-focused movements, their major defining characteristic being the close link-age to the spoken word" and "body-focused activity which bears no apparent relation to speech" (Freedman et al., 1972, p. 240). Thus, the two main values were primarily defined by their relation to speech. Furthermore, assumptions were made about the validity of the two values: "While object-focused move-ment seem to function as part of the verbalizing and symbolizing process, the major function of body-focused activity appears to be self-ministration." (p.

240). However, as it will become evident below, the body-focused values were operationalized primarily based on the visual appearance of the movement.

Object-focused hand movements represent the content of the verbal utterance or accompany the rhythm of speech. With regard to the visual appearance, they are defined as being directed away from the gesturer's body towards the listener.

Object-focused hand movements are subdivided into speech primacy and motor primacy, depending on whether the message is primarily given verbally or whether it is primarily conveyed through gesture. An example for motor pri-macy gestures are emblematic gestures (as defined by Efron), such as the Vic-tory sign. Emblematic gestures convey the message independently from a verbal statement, which may be completely missing. An example for speech primacy gestures are batons (as defined by Efron). These gestures rhythmically accom-pany the verbal message, which is the primary medium of information transfer.

Speech primacy gestures are further subdivided into punctuating and minor qualifier, while motor primacy gestures are further subdivided into representa-tional, concretization, speech failures, and pointing.

Body-focused hand movements are touches of the own body or its adorn-ments. The body-focused movements are divided into discrete movements that last less than three seconds, and continuous ones that last longer than 3 seconds.

The latter value is subdivided into hand-to-hand, i.e., hands touching each other, direct, i.e., touching another part of the body with one hand, and indirect, i.e., the manipulation of a thing, typically articles of clothing or accessories.

In 1995, Wilke published a revised version of Freedman's Body Movement Coding System, in which some kinesically oriented terms of the hand movement values were replaced by more psychologically oriented terms. The most relevant changes were that object-focused hand movements were termed narrative rhythmicities, indirect body-focused became instrumentals, and hand-to-hand body-focused was modified to shielding.

Freedman`s analysis system has been used in psychiatric, psychotherapeutic and psychological research, as it registers movement entities that are sensitive to psychopathological states, clinical improvement, and personality traits. Objec-tive definitions of the values are available (e.g Freedmann, 1972), and the inter-rater agreement has been investigated in all studies applying the system. How-ever, highly arbitrary is the distinction between discrete and continuous body-focused movements based on the criterion of a duration of either less or more than 3 seconds. There is no empirical evidence why 3 seconds (and for example, not 5 seconds) represent a relevant duration in time that separates two movement entities. Kinematic criteria would be more appropriate than an arbitrary temporal criterion to distinguish movement entities.

A dichotomy as that between object-focused and body-focused movements is also fundamental to Kimura's coding system (1973), which has been used mainly for neuropsychological studies. Kimura's system is primarily descriptive, while she makes empirically based assumptions concerning the link of the two movement values to language production. The system is presented in this group, because it is more related to function-oriented gesture coding systems than to comprehensive descriptive systems.

Kimura's system comprises only the two broad values free movements, de-fined as "any motion of the limb which did not result in touching of the body or coming to rest”, and self-touch, defined as "any act resulting in the touching of the person’s own body” (p. 46). Thus, free movements include all gesture values described by Efron or by Freedman's value object-focused. The value self-touch corresponds to Freedman's value body-focused, with the exception that the value body-focused does not only include touching of the body but also manipulation of objects, while Kimura’s value self-touch explicitly excludes the latter move-ments. Kimura's system has been applied by several researchers; most of them were interested in hand preferences. Later studies using Kimura's system re-ported the interrater reliability (e.g. Lavergne & Kimura, 1987).

However, recent research has demonstrated that the value free movements is too broad for neuropsychological research, as it collapses hand movements that are associated with different neuropsychological functions (Lausberg & Kita, 2003;

Lausberg et al., 2007).

For the analysis of movement behaviour in social interaction, the psychologists Ekman and Friesen (1969) defined the main values illustrators, emblems, adap-tors, regulaadap-tors, and affect display. The definitions of these values are predomi-nantly functional. Thus, some main values contain several movement classes, e.g. regulators can be position shifts but also head nods. Accordingly, there is no precise definition referring to the visual appearance of the movement.

Illustrators (equivalent to Efron's physiographics and ideographics) illustrate verbal messages and are closely linked to content and form of language state-ments like phrasing, contour of voice, volume, etc. Within the main value of il-lustrates, several types are distinguished that are adopted from Efron: baton, ideograph, pictograph, spatial movements, kinetograph, and deictic movements.

Emblems are non-verbal signals which generally can be replaced by one or two words and which are known to all participants in a social group. Ekman and Friesen's definition of emblem is comparable but not perfectly matching Efron's definition, as they include gestures with a morphological relationship to the ref-erent. Affect display is defined for facial movements (and therefore, not dis-cussed here). Adaptors serve for self-regulation. There is some overlap with Freedman's body-focused movements. Regulators serve to maintain and regulate verbal interaction. The latter two main values each include various classes of movements. Particularly, the conceptualization of the system that illustrators, emblems, adaptors, and affect display may all function as regulators causes a methodological problem.

McNeill‘s classification system (1992), which has been primarily designed for psycholinguistic research, includes the gesture values beats, iconics, meta-phorics, and deictics. Beats and deictics are equivalent to Efron’s batons and deictics. Iconics are comparable to Efron‘s physiographs but only those depic-tive gestures are classified as iconic in which the concomitant verbal statement refers to a concrete content, e.g. gesturally depicting a circular form when talk-ing about a concrete ball. In contrast, metaphorics are depictive gestures that are accompanied by a verbal statement that refers to abstract contents, e.g. gestur-ally depicting a circular form when referring to a project that is rounded off or complete. It should be emphasized that the distinction between iconics and metaphorics actually refers to the verbal message that accompanies the gesture, and not to the gesture per se.

Recent studies applying McNeill's system or expanded versions examine in-terrater agreement (e.g. Kita et al., 2007).

To summarize, there are several coding systems for gestures, i.e., for body movements that have a communicative or expressive function. While gestures

can be head movements or foot movements, they are most often hand/arm movements. Therefore, most of the systems focus on hand movements. Some systems also include an analysis of self-touching behaviour of the hands.

Despite different scientific backgrounds and different terminologies, there is a substantial overlap between the values of the different systems. Using Efron's seminal system as a frame of reference, Table 1 provides an overview on how the gesture values of the different systems are related to each other. The main values of the respective coding systems are printed in capital letters. In anticipa-tion of part II of this book, the Module III values of the NEUROGES system are included in the table. In addition, the class self-touch is listed.

Obviously, not all gesture values of the other systems can be clearly allocated to Efron's values. Values in the same line of the table are comparable, but they do not necessarily match perfectly. However, Table 1 may be used as a form of a translation scheme to facilitate the comparison between the research studies using the different coding systems for hand movements and gesture.

Table 1 Comparison of the values of different gesture coding systems Efron (1941) Ekman &

OBJECTIVE

While there are no validity studies according to psychometric standards, the fact that there is basic consensus among different - more or less independent - searchers concerning the existence of certain gesture values supports the re-searchers' propositions that these values constitute functional entities.

The gesture coding systems define the values by describing their function.

They vary in the degree to which the visual appearance of the body movement is referred to. Kimura's system defines the two values with regard to the visual ap-pearance. Efron supplements the functional analysis by a spatio-temporal analy-sis, and Freedman provides particularly for the main value body-focused move-ments precise descriptions of the visual appearance of the movement. However, even when values are mainly defined by their function, raters will typically be able to code the same values based on the visual appearance of the movement alone. As an example, raters can reliably identify a deictic (pointing gesture) or an iconographic gesture (depicting a form), even if they cannot listen to the speech that accompanies the gesture. Therefore, it is argued here these function-oriented coding systems partly work based on the observers' (implicit and ex-plicit) knowledge about the meaning or the psychodynamic function that a body movement with a certain visual appearance has in a given culture. Therefore, while in the coding systems the values are primarily defined by a description of their function, they could likewise be defined by a description of their visual ap-pearance.

Some values such as ideographs, metaphorics, and iconics are confounded with linguistic assessments. Furthermore, since the systems focus on gestures and partly on self-touch, they are clearly less comprehensive than the coding systems exposed in Section 3.3. With regard to hand / arm movements, Freed-man's and Kimura's system are comprehensive insofar as they code gestures and self-touches, the latter class obviously including implicit movements. In most studies, applying Kimura's system, the laterality of the hand movement has been noted. Likewise, the laterality is reported in studies using Freedman's system (e.g. Souza-Poza et al., 1979).

While they did not conduct a segmentation of the ongoing stream of behav-iour, Kimura and Freedman coded all hand movements displayed in a certain interval of time. They counted the number of free movements and object-focused movements and of self-touches and discrete body-object-focused movements per 5 and 10 minutes, respectively. For continuous body-focused activity, Freedman and colleagues also registered the time spent with this behaviour in seconds / 10 minutes (see Section 2.4).

Thus, while Kimura's system contains too broad values, Freedman's system is well suited for empirical and even quantitative research. However, it is lim-ited to hand / arm movements. For Efron's referential coding system, some validity of the gesture values can be assumed, since - despite different termi-nologies - other researchers have later come to similar hypotheses concerning the meaning of certain gestures that can be identified by their visual appear-ance. As Efron employed a three-fold analysis, a detailed description of the visual appearance of the gesture values is provided.

3.5 Systems that register alterations in movement behaviour