difference ‘perceptoid’ of pictorial signs can be demonstrated particularly well by means of these domains.
As the most complete one of the three, a pragmatic investigation deals with the com-plex formed by a communication act and the other related acts (of communication or of any other sort), i.e., the embedding of the sign act in the “living practice” of the sign us-ers. Which other act or behavior can or must precede a certain sign act? Which others may or have to follow? That, for example, the utterance of an assertion can be answered either immediately by an accepting or refusing (doubting) act of communication of the interlocutor, or by first performing a referential anchoring with corresponding sensory-motor test routines, this is the structure of that particular “language game”, and hence a theme of pragmatics.
More restrictive in its perspective, semantics focuses on the relations between sign vehicle and what is represented by means of it – as far as those relations are relevant for the sign use and can be investigated essentially without looking at the pragmatics of the sign act. Expressive or appellative aspects of sign uses are not taken into account. Usu-ally, such relations are subsumed under the expression “meaning”. The links between a definite description (“this blurred photo”) and the object referred to, between a certain part of the distribution of color on a screen and the face depicted, or between a particu-lar arm movement and a child’s emotional state are typical examples for the “objects”
semantics is interested in. From a strictly semanticist point of view, indexical signs offer causality, and iconic signs similarity as candidates for a non-pragmatic meaning rela-tion. Following the linguistic turn, it is now widely accepted that the concept of a strictly semantic investigation of meaning is ill-formed (with the exception of formal and artificial languages), and that semantics forms a special part of pragmatics focusing on the representational aspects of sign acts.
The syntactic domain of questions has the most restricted horizon as it deals, strictly speaking, only with the relations between the sign vehicles of a sign system. Since sign vehicles are essentially some physical objects used in a particular manner, syntactic in-vestigations examine in consequence just the relations between and properties of physi-cal objects. Representational, expressive or appellative aspects are ignored. The main focus of interest lies in determining the range of deviation of physical properties not changing the identity of a sign, and the rules of composition forming vehicles for com-plex signs from vehicles for simple signs. Obviously, the criteria used to distinguish one sign from the other can only be derived from the sign system as a whole, i.e., from a pragmatic point of view.
Although the „natural“ order is, thus, to start with pragmatics, and then progress to the particular aspects of semantics and syntax, we shall go the other direction – a proce-dure quite familiar to computer scientists, as programming languages are usually ex-plained by starting with the syntax, adding the semantics of the constructs, and some-times complementing the explanation by style guides as a very weak form of pragmat-ics.
In our context, the section on pragmatics (4.4) serves the purpose of bringing the par-ticipants of the pictorial sign act, their interest in those images, the purpose of their in-teractions with the computer, and the general communicative setting into prominent fo-cus. Before doing so, section 4.2 investigates the options to reproduce pictures by means of a computer: (a) How can this particularly complicated but nevertheless closely re-stricted physical artifact provide the range of properties that renders it useful as a picture vehicle? (b) How does that effect the data type »image« that ultimately determines the
subject of computational visualistics? These syntactic considerations are followed by focusing on the representational aspects of pictorial communication: under the label
“semantics” we investigate in section 4.3 a particular set of relations between the data type »image« and other data types covering the image’s content.
4.2 Syntactic Aspects
Combining elementary sign vehicles into complex ones is often viewed as the central issue of syntactic investigations. In this tradition, STROTHOTTE & STROTHOTTE [1997, Sect. 3.1] have presented some thoughts about a combinatorial syntax for computational pictures. They have introduced analogies that may be drawn between linguistic and pictorial levels of sign elements we shall also use in this section. In particular, we have to distinguish between the non-autonomous elements combined into a picture (4.2.2.1), and the combination of autonomous pictures into pictorial (or other) signs of a higher order (sect. 4.2.2.2).
In the present discussion about the concept of pictures, the property of having a dense range of sign elements is considered prominent among the syntactic aspects: verbal signs are characterized in contrast by their discreet succession of elements. GOODMAN
has introduced in this context the concept »density« – intuitively related to the structure of rational numbers – by means of a strange bipartite negation concerning (i) the uniqueness of the relation between sign vehicle and sign, and (ii) the existence of an ef-fective procedure to prove the former relation. However, the two negations also hold for the concept »continuity«. As the distinction between the two concepts throws some light on the other two themes of pictorial syntax in computational visualistics, it is discussed first. 31
4.2.1 Pictorial Resolution and the Identity of Images
For GOODMAN [1976, 133] it is essentially the attribute of syntactic density that is characteristic for pictorial sign systems, and hence plays an important role for the corresponding data structures. A sign system is called syntactically dense, if the dimension of values for at least one of the syntactically relevant properties of the sign vehicles corresponds to the rational numbers: between any two values there are always more values. Sign vehicles with different values in that property are taken as different signs in that sign system. That is, two of the infinitely many signs of such a system can be “infinitely similar” to each other. If no such dimension of properties is given, that is, if all syntactically relevant attributes take values that can be separated from each other distinctly, the sign system is called syntactically discreet.
Syntactic characteristics of pictures are obviously defined by the visual properties of a marked surface of the picture vehicle. There are at least two different relevant dimen-sions that are apparently dense: (i) the positions of a point of color or a border between colors, and (ii) the perceived color (in a broad meaning). In the following, the range of positions and its connection to the concept »resolution« is investigated (for color cf.
sect. 4.2.3).
31 Let us, for the time being, restrict ourselves to flat, smooth, rectangular pictures. The other forms can usually be dealt with in analogy or by a simple projection to the basic form.
If syntactic density is accepted as an important criterion for pictorial sign systems, the author has not given pictures in the first row of Table 1 at all. The second row, however, is used in fact for exemplifying five pictures.32
The signs in the upper row belong to the discreet sign system of the international traf-fic signs. The first three sign vehicles depicted in the lower line can indeed be also used as vehicles for the first sign in the upper row. However, those three sign vehicles carry three quite distinct signs when viewed as pictures – the sign system actually intended in the second row. The syntactically discreet system of traffic signs is embedded in the syntactically dense system of pictures: each traffic sign forms the island of an equiva-lence class, so to speak, surrounded by pictures that are not used as traffic signs. Simi-larly, the sign vehicles of letters – “a”, “b”, “c”, etc. – can be conceived of as pictures (as in a font editor) and also as signs in a syntactically distinct sign system (as in a word processor) depending on the pragmatic context.
The syntactically characteristic property of density is of high significance for the pos-sibility of encoding, presenting, storing, and transferring pictures in/with a computer. Is it decidable whether two pictures are syntactically equal? Can we, with other words, de-termine whether the transmission of a picture through the Internet, for example, has been correct, or whether a stored image still corresponds to the original?
GOODMAN deduces from density as the relevant syntactic property of pictorial sign systems that sign vehicles cannot be associated uniquely to one sign alone. In conse-quence, an effective proof of correctness for any image transmission becomes impossi-ble on the base of syntax. He writes, referring to signs (“characters”) of a sign system that are determined by the lengths of sign vehicles (“marks”) [1976, 132]:
Corresponding to the different rational numbers, there will, then, always be two (or more precisely: infinitely many) characters such that measuring cannot determine that the mark does not belong to them independent of how precise the length of a mark can
be measured.
Here GOODMAN implicates a division of problems in classes of decidability well-known to any computer scientist. In the concrete example, a semi-decidable problem is considered. We have to decide whether a length is not equal to another length.33 The positive case (“not equal”) can be determined with a finite number of steps: by compar-ing with successively higher resolutions. That does not work for the negative case of the
32 The reference to the sender is of course quite essential here, as a finite set of example sign vehicles alone does not properly identify a sign system – we would at least have to add “that’s all”. The property of syntactic density or discreetness can be ascribed only to the intended sign system as a whole.
33 The same holds true if not lengths but the positions of spots of color are considered.
Table 1: Five picture vehicles of two quite different sign systems
question (not “not equal”, i.e., “equal” – here is indeed the reason for the complicated formulation with double negation GOODMAN uses).
4.2.1.1 Density, Continuity, and Decidability
From a formal perspective, density can be stated if there is a property of the sign vehicles (like position) that is structured as to allow us to speak about a “between”-relation of its values. This “between”-“between”-relation must again fulfill certain conditions. That is, syntactic density is a property of another property (“between”) of attributes (“position”) of objects. Rational numbers are the archetype for density. Real numbers are dense as well, but they have also another property of the same general structure:
they are continuous. A type of numbers is continuous if the limes of any infinite sequence of numbers also belongs to that type. This is not true for the rational numbers, as for example the sequence of numbers approximating the relation of a circle’s diameter and its circumference has as its limit not a rational number. The real numbers can indeed be conceived of as introduced by means of closing the rational numbers under the limes operation.
Being material objects, picture vehicles have surfaces we usually consider as being continuous, i.e., associated with the real numbers. Marks on that surface, spots of pig-ment, for example, may be the results of movements (of a brush, a droplet of ink, a jet of electrons). In physics, we have to consider a continuous range of locations in order to describe the interception of the movement with the surface adequately, i.e., without paradoxes of ZENO’s type haunting our conception.
The distinction between density and continuity for the range of positions of pictorial surfaces is particularly important because the two types of numbers are associated with different kinds of infinity. The rational numbers can be enumerated while the real num-bers cannot [CANTOR 1874]. As is well-known, many problems concerning the question whether an instance with a specific combination of attributes does exist can be decided if the members of the set considered can be enumerated: any member can, then, be reached for checking after a limited number of steps. Correspondingly, testing the iden-tity of any two numbers (e.g., in decimal notation) is a decidable problem only for the rational numbers. For real numbers, the test is only semi-decidable: we can find out in a finite number of steps whether two numbers (of usually infinitely many figures) are not the same, but in general not whether they are indeed the same.
The observation that picture vehicles must be viewed as a field of concept with a con-tinuous, hence over-enumerablely infinite range of locations due to the conditions of their production does by no means imply the type of infinity for the locations relevant for the concept »image«. Although the vehicles may be linked with locations by real numbers, it is still possible to assume that rational numbers are sufficient for the range of positions relevant for pictures: densely ordered equivalence classes in the continuous sea of possible picture vehicles.
4.2.1.2 Syntactic Types of Pictures in Computers
Based on the classes of decidability, three classes of pictures can be distinguished on syntactic grounds in computer science.