Conclusions for Computational Visualistics

The overview on image science given here was centered around the following idea: to isolate the essential properties of the concept »picture« by means of a logical reconstruction (“an implementation” see Sect. 2.1) of the corresponding field of concepts. Such an investigation deals with questions like “how can we motivate that such a form of sign acts with the special involvement of (visual) perception could have developed?” – “What are the general properties of sign use and perception that are necessary for such a conceptual combination?” – “To what purpose and under what

Figure 42: Ocean and Pier

P. MONDRIAN 1915

preconditions did creatures evolve that can use such a type of signs?”

Thus, we deal with the concepts

»perception« or »sign use« or

»picture« in the essence, so to speak, not with a particular form any such concept may take in a certain cultural environment; with the “conditions of their possibility” inKANT’s words.

Some characterizations, then, hold for pictures because of those general structures of perceptoid signs while others are just contingent conse-quences of one particular and idio-syncratic instance (of many equiva-lently possible such instances) of a

perception apparatus with sufficient complexity, or of the specific language and sign systems established for those picture users. The latter properties may change with cul-tural development, and may even be used to investigate culcul-tural differences and devel-opments (being the classical fields of history of arts, and cultural anthropology, of course). Modifications in the former attributes however do result in a different concept altogether, something that is not characterizing pictures anymore but something else.

The difference between the two types of properties is clearly expressed when corre-sponding forms of explanations are juxtaposed: “Since visual perception follows this or that rule (for us middle-aged Europeans at the beginning of the third millennium)” – think of perspective, for example – “that and such is true for pictures (for us)” vs. “Since perception (in general) can only be rationally conceived if this and that relation holds” – e.g., between sender, receiver, sign vehicle, and sign content – “perceptoid signs are only possible if that and such is granted.” Computational visualists are expected to know something about the first kind of explanations, but ultimately they must know everything about the second kind by heart if they want to earn their name.

From a general act-theoretic perspective, two lines of argumentation have been fol-lowed, associated with levels of understanding with growing complexities: for (i) per-ception and (ii) sign use. Combinations have been sketched on different levels, but only the most complex pair – perception of individual things and assertive language – allows us to reach the conceptual structure of perceptoid signs. The goal in the next chapter is to take the essential structures resulting for pictures as a guidance toward the specifica-tion of the “complete data type »image«” that underlies and structures every effort in computational visualistics, if one such thing can rationally be considered; or alterna-tively to motivate the set of distinct data structures necessary, and to clarify the relations they stand in.

The gigantic task of unfolding the new discipline on the basis of general characteriza-tions of visual perceptoid signs can indeed act as a research programme only in the pre-sent context. As has been mentioned in Chapter 2, computational visualistics, as a co-herent field of computer science, is a relatively new idea, a consequence of general visualistics – as a unified image science – not being established earlier. Up to now, deal-ing with pictures in computer science has been separated in several sub-disciplines with more or less loose methodological connections (mainly by means of computer science

Figure 43: An Exemplification of the Algorithm ‘envi-ronment mapping’ Using the Notorious “Utah Teapot”

but not by the common subject “picture”). Only aspects of the “complete data type” (or the structured set of several types) have been considered.

Let us conclude the condensed review of theory in image science by summarizing the most essential points for the computational visualist.

• An explication of pictorial communication must consider the double nature of per-ceptoid signs: their general symbolic aspect and their particular perceptual aspect.

The way pictures are interpreted is mainly influenced by the way these two aspects interfere: pictures stand in the diverging force fields between communication and immersion.

• When dealing with pictures, one has to answer in general the question “who is communicating with whom”; that relation between sender, sign, and receiver is fur-thermore complicated by the very nature of conscious communication – to internally anticipate the interlocutors. Generic models of senders and receivers anticipated by the actual communication participants regularly interfere in the sign process.

• Pictures are signs with representational function. However the naïve conception of a simple relation “similarity” per se between picture and objects or states of affairs as the basis for that function does not satisfy. Internalization of the resemblance re-lation leads to the integration of the complicated principles of primary and secon-dary object constitution into the act of signification.

• Pictures are neither true nor false; they are used in sign acts that are authentic, i.e., legitimate, or not. This is a relation not merely between a picture and what it is used to stand for (objects, states of affairs), but a relation that additionally (and more importantly) includes the participants of the sign act (sender and receiver).

• Pictures are basically used as context builders: their presentation is an unsaturated form of communication that refers to complementing sign acts. The complements are not part of the picture, and can therefore not be predicted by the picture vehicle alone.

• Pictures are not primarily used for referring to objects, exemplifying abstract enti-ties, or communicating states of affairs: they are employed to open up a medial dis-course universe with objects as partaking in states of affairs. Reference to objects is performed by means of nominations (that depend on contexts introduced priorily).

States of affairs are communicated by assertions.

• Objects (referred to by means of a nomination) may be used in a metonymy as ex-emplification for an abstract aspect. The metonymy can be extended to a corre-sponding picture leading to abstract pictures. Fields of concepts, which structure state of affairs, can be mapped on each other by metaphorical transfer. The meta-phor can also base the use of pictures leading to structural pictures. Finally, quoted pictures and pictures of art are employed in a very special mode of usage different from the usual un-reflected mode.