A Border Line Case: Immersion

Figure 110 presents three gray-scale examples: color and texture parameters are ig-nored here for simplicity.⁸⁹ Let us assume that the communicative intention given asso-ciates the background (a table top) as nominatoric, and a bunny’s shape and the con-figuration of its (body-) parts as predicative, while the relative position of the bunny with respect to the background objects is also marked as known already to the beholders in question. Then, it is sufficient for the background to be drawn in outline and without parts (“atomic”). For the bunny however, form and configuration parameters should be maximal (i.e., value “natural”). If only the bunny’s configuration is predicative, the form parameter changes to “outline”: the four components are clearly discernable.

The third example is more complicated since the place of an element can only be predicative if the configuration of the complete scene is emphasized, as well. Therefore, using the bunny’s place as predicative property has to be propagated up in the hierarchy of representation elements to the configuration of the scene, and from there down again to the place parameter of the other children, i.e., all the background elements. The inner contours of the tabletop are selected while the configuration of the bunny becomes atomic and only outlines need to be shown. The graphic clearly emphasizes the bunny being almost at the border (in consequence, its danger to fall off is here more evident).

The act of distancing is indeed not restricted to aesthetic considerations in the close sense but forms the core aspect of the symbolic mode: evoking contexts that are not the situational context at hand is the major function of signs. ‘Evoking a context’ means not only immediately activating certain spontaneous reactions, but also the ability of post-poning those reactions to the context evoked though not present (cf. Sect. 3.5.1). Thus, one of the components defining the concept »picture« is missing. If the use of pictures in highly immersive systems aims at a reception in the pure deceptive mode by sup-pressing any factors able to activate the viewer’s symbolic mode, then there are indeed no pictures used at all, at least for the “immersants” who do not spontaneously reflect about their situation.

Nevertheless, an important part of computational visualistics deals exactly with this borderline case of image use. As has been mentioned already in chapter 3, the producers of trompe l’œil pictures as well as of immersive systems cannot share the reduced recep-tion mode: they in fact deal with pictures as perceptoid signs that are intended to be mis-taken – at least for a time or to a certain degree. Usually, we do not meet immersants that do not at all reflect their specific situation: highly immersive systems still need so much technical effort and preparation that nobody simply find themselves in such a sys-tem without noticing it. Correspondingly, they are not in a pure deceptive mode but in the immersive mode where the postponing of spontaneous reactions is strongly reduced, not totally absent. Recall the situation in cinema when we appear to be confronted with a life-like Tyrannosaurus rex threatening to attack us: we know that it is an illusion (a sign we show to ourselves), consequently having a lot of spontaneous reactions that are generally postponed; but we allow those reactions to surface to some degree, which is one of the pleasures of viewing such films.

Long before expressions like ‘virtual reality’ or ‘immersive systems’ have become popular, S. LEM investigated different levels of immersion under the name of ‘phan-tomatics’ [LEM 1964]. Present attempts are based essentially on technical devices pro-jecting pictures on more or less flat smooth surfaces covering all of the field of view:

they can be viewed in the ordinary sense.⁹⁰ Sound is being emitted by speakers in more or less sophisticated manners, also to be heard in the traditional way. Quite obviously, only distance senses can be easily deceived by that form of immersive systems. Many contact senses are much harder to be deceived – recall all the sensations from our skins.

The feedback from the immersant’s actions is also mostly restricted to very specific and very small channels: a mouse, a data glove, or a data suit at most.

As an improvement of immersion, LEM imagines what he calls ‘peripheral phan-tomatics’, where technical devices directly manipulate the immersant’s peripheral nerv-ous system, feeding the sensoric nerves and taking signals from the effectoric nerves:

the pictures or sounds used can then be inspected only by additional devices. Still, the immersant’s body with its movements and the monitoring proprioceptors form a source of “disturbance” to the immersion. Therefore, another step is introduced: LEM’s expres-sion ‘central phantomatics’ refers to the hypothetical technology that allows technical devices to directly manipulate the immersant’s central nervous system, overwriting any bodily signals including those from the sense organs, and intercepting any nerve pulses controlling (real) motion. That is: for the immersant, the physical body is completely replaced by the avatar. He or she seemingly exists only in “cyberspace”.

90 Current experimental devices projecting light directly into the eye form the only exception known to the author so far.

Beside the aspect of plausibility, LEM is basically interested in the epistemological question whether (and how) an immersant of any of those levels would be able to detect the deception – indeed a modern pendant on DESCARTES’ reflection on the nature of truth under the assumption of a deceiving deity.⁹¹ This is not the place to investigate those problems.

Of course: the expression ‘immersion’ is not only a question of technology – more likely a question of purpose and concentration. Humans are very well able to be com-pletely absorbed in reading a novel ignoring most of their situational context (even up to the bodily needs to some degree) evoking mental imaginations of what they read. High degrees of technically induced deceptive mode are relevant in simulations, e.g. for pilot training. In other applications, the equilibrium between the symbolic and deceptive components of reception is more complicated. Different purposes lead essentially to dif-ferent possibilities of interactions. Two particular cases – virtual architecture and virtual institutes – are presented in the following together with a description of the particular conditions and intentions of application, demonstrating alternative needs for deceptive or immersive reception modes of the pictures used.

5.3.1 Virtual Architectur: The Atmosphere Projekt

The first example is the computational reconstruction of an important Jugendstil house built by PETER BEHRENS (1868–1940) in Darmstadt, which is not preserved in its original state. The house was designed as a piece of the exhibition Ein Dokument deutscher Kunst that was prepared by the artist colony of Darmstadt in 1901. It represents a unique Gesamtkunstwerk: Apart from the architecture of the façade and the exterior disposition, all details of the interior decorations are based on designs of BEHRENS himself. This does not only include the doors, windows, carpets and wallpapers. The form of furniture, lamps, glasses, chinas, and cutlery, even music instruments, inkstands, and jewelery fitting to the house’s aesthetic conception are based on inventions of BEHRENS. In this unique fashioning of human living space characteristic for Art Nouveau, a specific conception of life was expressed. It is the approach of using artistic abilities to transform the environment in a beautiful and reasonable manner in order to harmonize again humaneness and the technical development [BUCHHOLZ 2000; BEHRENS 1901, 3–6]. During the 19^th century, the two aspects had developed more and more into quite different directions in Europe, a disintegration of the manners of life that pushed forcefully into public conscience at the end of that century and gave rise to a large number of reform-oriented approaches, which to our days have a strong influence.⁹² The functional and aesthetic investigation of an integral ensemble like the House Behrens is an important building block toward a proper understanding of the characteristic conception of life of that time and its lingering influence on us.

91 While L^EM keeps an ironic distance to such technologies and is interested mainly in epistemological and ethical questions, other scientists are not so prudent: recall, for example, M. MINSKY’s public fan-tasy about all humans being “equipped” with an implanted computer interface to their brains. The crea-tivity of narrative artists had been excited by such scenarios, as well; beside FASSBINDER’s TV produc-tion Welt am Draht [1973], C^RONENBERG’s film eXistenCe [1999] is one of the more interesting results.

92 Just as a few examples: fitness studios and ecological agriculture, functional architecture and feminism, health food and artificial tanning; they all can easily be traced back to the broad movement of “Lebens-reform” about a century ago [BUCHHOLZ ET AL. 2001].

However, the house was destroyed severely during Word War II. The façade was re-constructed afterwards with only minor alterations, but the interior decoration is utterly lost.⁹³ The original partitioning of rooms has not been restored; all decorative elements are destroyed. A reconstruction by means of computational visualistics – a virtual archi-tecture – must, then, appear as a very plausible approach [FORTE & SILIOTTI 1997].

Such a virtual reconstruction has to be based on data about the original contexts as precise as possible. In the case of BEHRENS’ house in Darmstadt, there is a sufficient amount of details available at least for the two central rooms of the ground level: the fact that pieces of Art Nouveau have been broadly documented in illustrated papers of that time does not only demonstrate how important the underlying conception of life was rated then; it obviously is quite helpful for the virtual reconstruction, as well. The journal Deutsche Kunst und Dekoration appearing in Darmstadt published an extensive article about BEHRENS’ house with floor plans, sketches, and many large black-and-white photographs [BEHRENS & BREYSIG 1901/02]. A similar paper appeared in the journal Dekorative Kunst [SCHEFFLER 1902]. A special catalogue was produced for the exhibition covering exclusively BEHRENS’ house: it contains an introduction by BEHRENS and a list of all the enterprises that constructed the objects designed [BEHRENS

1901]. Finally, numerous modern color photographs are available of those parts of the furniture that have survived World War II and are exhibited in several museums (e.g., [INSTITUT MATHILDENHÖHE DARMSTADT w/o y., 6–27].

The two rooms in our focus of attention have been discussed extensively in contem-porary architecture critiques: they are the music room and the dining room just beside

93 Furniture and other movable parts had been already removed from the house since BEHRENS moved to Düsseldorf in 1903. Fortunately, some of the furniture was therefore spared from the destruction.

Figure 111: Contemporary Photography of the Music Room. The door to the dining-room is just outside the right frame border

each other (Fig’s 111 & 112). Both rooms serve social purposes and are connected by a large double door with partitioned wings. The music room is higher in order to evoke a particular atmospheric effect described by BEHRENS [1901, 8 f.]:

In order to heighten the music room in accordance with its purpose with respect to the rooms around it – the true festive room of the house – it was necessary to lead down two steps from the hall and simultaneously to lift the ceiling approximately for the same dis-tance compared to the adjacent dining room. The two steps have the practical purpose named; but also the other, spiritualized one: to lend a rhythmical movement to the traffic between dining room and music room. Stepping down gives us the feeling of being pre-pared for something; stepping up evokes the one of lifting to something. And in those feel-ings, very essential moods of humanity can be recognized.

Numerous details and materials have to be considered: The floor of the music room consists of a parquet with seven different woods forming a linear geometric pattern. The dining-room has a floor of mosaic in a curved pattern. The steps leading from music room to dining-room are of a pink marble, the music room’s walls are covered with grey marble and blue reflecting glas. Above the door to the dining room and at both sides, additional mosaics are placed. The walls of the dining-room are interrupted by large windows above silver-coated heating grills. Between some of the windows, crystal mirrors are installed. Other parts of the walls are panelled in white-lacquered wood below a frieze of damast. The ceilings of the two rooms are richly decorated: in the music room, it consists of gold-painted wood with another linear ornament; the dining-room’s ceiling has curved stucco ornaments with some of the intermediate spaces coated with silver. The doors are also particularly elaborated. The door from the music

Figure 112: Contemporary Photography of the Dining-Room. The door to the music room is par-tially visible at the left side

room to the hall is adorned with a linear ornament in golden aluminium bronze similar to the music room ceiling. The door between music room and dining-room is white on the dining-room side with a simple curved ornament; toward the music room it shows

“an uninterrupted surface of noble silver-maple without a single ornamental line”

[BEHRENS & BREYSIG 1901/02, 148].

The interior decorations of the House Behrens vary a small number of different or-naments. The decorative design of the dining room is developed from the basic figure of two intersected curves. They appear in the simple form in the gratings of the cupboard glasses and in the mirror cuttings. A more complex derivation is shown in the heating grills, where different degrees of sinuosity are combined. The curved line induces verve and movement into the dining room. In contrast, all forms in the music room are devel-oped from a linear base, a rhomb, which develops into the complicated form of a crys-tal. Decoration and furniture of the music room induce the impression of static calm-ness.

Taking into account the conceptual background of Jugendstil design mentioned above and the particular emphasis BEHRENS put on designing a complex but uniquely coherent whole, any alteration of a detail in the virtual reconstruction can destroy the intended use. Unfortunately, many of the colors and texture are uncertain. Black and white pho-tographs only hint at the relative luminescence. Verbal descriptions of the colors are of-ten rather exuberant but obviously of limited help for the computational visualist.

The success of a project like the reconstruction of the House Behrens by means of computational visualistics depends, thus, on an intense cooperation between computa-tional visualist and art scientist. Decisions have to be made concerning the colors or tex-tures to be used. Has a given texture to be modified? In which way? What are the crite-ria? Is it technically possible? Answers depend essentially on the precise purpose of the immersive images to be produced, and in particular on the addressee. Detecting the au-thenticity of colors and texture falls, of course, essentially in the domain of the art

scien-Figure 113: Screenshot from the Virtual House Behrens: View from the Dining-Room into the Mu-sic Room (with door to the the hall)

tist; but “trial shots” generated by the computational visualist are certainly quite helpful even if they are still far from an acceptable end result. In a way, the visual intermediate results (e.g., (Fig’s 113 & 114) serve as a virtual experiment in which the effects of de-cisions about colors and textures can be concretely studied. Thus, the sign character ex-plicitly controls the deception.

The border of present immersive technology can be clearly demonstrated when we consider the broad use of reflective surfaces in the two rooms under investigation: that a material is highly reflective is, of course, not a major problem for computational visual-istics: but what happens if the viewer looks at one of the blue glasses in the music room from an angle of approximately 90°? In the original (not virtual) room, she could then see her own mirror image – an effect BEHRENS certainly has taken into account when planning the house. It is, of course, impossible to model for every user of an interactive system that provides virtual strolls through the reconstructed House Behrens a fitting avatar that could be seen in the virtual mirrors. The geometrical and optical specifica-tions of the body of that user in its present bearing would have to be included in the model. On the other hand: being invisible in the virtual context or having no body is certainly not a satisfying solution, too. Finally, a “digital dummy” could appear in the virtual mirror representing the user, though it is probable that a user may mistake it not for the own reflection but for the avatar of another user. A solution is not yet palpable.

In the perspective of pictorial pragmatics, the problem offers an interesting aspect apart from aesthetical considerations: How does the missing (or wrong) reflection dis-turb the intention of the reconstruction of a lost Gesamtkunstwerk: i.e., to maintain the deceptive mode of reception? In general, the virtual reconstruction of the House Behrens has to investigate the problem of how much intentional deviation from the original is possible without upsetting the integral atmosphere of the environment. On the other side: how much deviation is necessary in order to mark clearly those parts that are not (or not certainly) integrated in their original appearance?

Figure 114: View of the Reconstructed Music Room toward the Dining Room

No doubt: virtual architecture can provide experiences over and above those of con-temporary photographs: for example, views from arbitrary perspectives, and – up to a degree – the atmospheric synthesis of the colors and materials used. The effects of dif-ferent lighting provide another prominent example (compare the two parts of Fig. 115).

Mediating further atmospheric aspects in an adequate manner is more complicated: the experience of stepping up or down between music room and dining-room, for example, can hardly be gained in front of a computer screen with the mouse as means of naviga-tion. The same holds true for the authentical impression of the size of the rooms and the objects within: in order to “really” walk though the computationally reconstructed house, we have to employ a highly immersive system, like a CAVE [CRUZ-NEIRA ET AL. 1992] – a cubic room with stereo projections on (almost) all walls on which pictures in the correct perspective relative to the position of the user in the room are projected. The position and view direction of the user are registered; they influence in real-time the projections. Shutter glasses let the beholders see objects in stereovision. Similarly, cor-responding sounds – e.g., of steps, closing doors, or drawn curtains – adapted to the ac-tual position of the immersant with a surround sound system enhance the deceptive feel-ing of befeel-ing present in that virtual reality.

The computational effort for the different projections of the House Behrens simulta-neously necessary in sufficient detail is still rather too high. Apart from that, using a CAVE for presentation certainly gives a better basis for studying atmospheric aspects of such an artistic ensemble. At least, the proportions of the rooms or their acoustics can be perceived in an adequate manner, quite close to the original. But even for that form of presentation of a virtual architecture, we can easily find problematic aspects of atmos-pheric effects. Although the visual impression paired with corresponding stepping sounds suggest that we are stepping up from the music room to the dining room, we do not use the muscles of our legs in the same way as in real life.

Figure 115: Two Screenshots from Approximately the Same Perspective – Daylight Atmosphere …

With the questions of aesthetical atmosphere, we meet, it seems, a fundamental diffi-culty of virtual architecture and its use of computer-generated images. The problem is connected with BEHRENS’ aim of artistically permeating all aspects of life in the house.

In the House Behrens, everything is designed towards one unique homogeneous effect.

That impression of aesthetical homogeneity has been emphasized by the contemporary critiques (cf., e.g., [MEIER-GRAEFE 1901, 482–484] or [SCHÄFER 1901, 39]). In order to adequately reproduce that impression, it is however necessary to rather use replicas in-stead of images since the specific distance that always separates the beholders from the image referents due to the sign character of the picture easily may “poison” the atmos-phere and disturb the integral impression intended.

However, it is not the task of virtual architecture to provide room that can be (virtu-ally) inhabited: We are interested in gaining a sensible impression of the house that is sufficient to understand how the abstractly described desire of BEHRENS (and other art-ists of Art Nouveau) – to extensively embellish everyday life – is put into concrete ef-fect. It is quite unclear, how close we have to come to the original to gain a “sufficient”

impression of the characteristic atmosphere.

5.3.2 Types of Use of Virtual Architecture

The degree to which the original atmosphere has to be evoked varies with the different aims of such a virtual architectural reconstruction. There are essentially three goals determining the design: the presentation should be mainly (1) educational, (2) scientific or (3) entertaining.

Among the educational goals, the focus can either be on the mediation of the histori-cal appearance of one individual object, which together with other exhibits and apart from the virtual architecture is expected to lead to corresponding new knowledge of the beholders/users: “That’s how that object did look like, that approximately was the inte-gral effect.” Quite easily, we can imagine to have a guided tour in a museum through

... vs. Nocturnal Atmosphere (and a few more alterations, e.g. drawn curtains)