In the hybrid interface of mobile screens, tagging, plotting and stitching converge in augmented reality browsing. Augmented reality is a container term for the use of data overlays on real-time camera view of a location, a term coined by Claudell and Mizell (1992). Originating from developments in virtual reality, using bulky head-mounted displays and later backpacks with equipment, the use of augmen-ted reality is currently taking off in applications for mobile phones. This is a fast-developing field at the moment of writing: from marker-based augmented reality (Rekimoto 1996) and QR codes, to image recognition technologies and experi-ments with haptic feedback to create a sensation of material depth of objects. AR browsers Layar and Wikitude and, more recently, Junaio are rapidly expanding the possibilities of (consumer) AR browsing for smartphones such as the iPhone.
They offer browser applications on devices that have a video camera, GPS, a com-pass, and an orientation sensor, entailing a new way of engaging with screen-space and navigation of digital screen-space, by effacing the map representation and using direct camera feed with a layer of data superimposed. AR browsing entails a new way of engaging with screenspace that converges the practices of touring, tagging and navigation of digital maps.
5. performative cartography 157
Augmented reality browsers make it possible to browse data directly within
‘reality’as it is represented on the screen, showing objects within their spatial context. The camera eye on the device registers physical objects on location, and transmits these images in real time on the screen. On-screen this image is com-bined with different layers of data in different media: still image, text, moving image. These layers have various scales and dimensions within one master frame.
We see information superimposed on a real-time image on screen.
Fig. 5.2: Flashmob in augmented reality. Image: Sander Veenhof, 2010
The screen is not actually transparent, but in effect, through real-time, simulta-neous display of the camera feed, it seems to be. It looks like and functions as a transparent window, framed only by the edges of the screen. This framing is temporary and directly changeable by the user wielding the screen. As such, in terms of screen-based representation, augmented reality browsing provides a complex sort of framing of this‘reality’. We could say that the screen itself frames the video image on screen, yet the information is layered on the image, and in a sense frameless. The frame is the camera image that brackets off the contours of the world-as-image. With this new mode of‘reality browsing’by means of cam-era feed, the map on the screen has been rescaled to the same proportion as our vision through the camera lens. Like that vision, it depends on the relative
dis-tance between ourselves and the objects seen, and the perspective changes according to our movements.
AR browsers such as Layar,Wikitude andJunaiooffer platforms with different uses for this layering, ranging from commercial applications of location-based services showing where restaurants, banks or shops are located, or what real estate is for sale, to more artistic interventions such as virtual expositions, gal-leries on location, or museum tours. Augmented reality offers a new platform for exhibition in public space, as is being discovered by museums, archives and other cultural institutions.ARtours, for example, is an initiative by the Stedelijk Museum in Amsterdam to develop an AR infrastructure for art tours.
Fig. 5.3:ARtotheque, a virtual gallery on location at the Lowlands music festival, orga-nized by the AR project of the Stedelijk Museum, 2010. Image: Stedelijk Museum/Tabworld-media.
In the summer of 2010, the Stedelijk Museum collaborated with the MediaLAB to hold a virtual exhibition of AR art on the Museumplein. The first augmented rea-lity flashmob was organized in April 2010, also in Amsterdam. There and then, people could ‘encounter’ all kinds of virtual statues or other characters on the street by wielding their mobile phones. These initiatives explore ways to bring AR applications to the public space for (scheduled) public events that can be shared.
5. performative cartography 159
There is, however, a tension between the size of the individual screen, and the space available for multiple participants.16
The location specificity of augmented reality based on the tagging and plotting of space is, paradoxically perhaps, highly transportable to other locations. Tags can be moved easily. Time- and space-specific events–like festivals–can be used as settings for temporary virtual exhibitions, asARtoursexperimented with such concepts at a music festival. In augmented reality, exhibitions can travel, infinitely multiplying and coexisting in space.
Fig 5.4:Can You See Auras?by artist Marieke Berghuis at the Stedelijk Museum virtual art exhibitionIk op het Museumplein[Me on the Museumplein]. Image: Hein Wils, 2010 A less time-based programming of augmented reality tours, but dealing with time nonetheless, is the Urban Augmented Reality (UAR) application launched by the Netherlands Architecture Institute (NAi). The tour shows large 3D buildings that were either once there in the past, will be there in the future, or were designed but were never actually built at all. In the hybrid screenspace this tour establishes, the present, past, future, and even the‘past future’do in fact coincide.
Fig. 5.5: Urban Augmented Reality (UAR) in Rotterdam. Image: Netherlands Architecture Institute (NAI), 2011
Using the reality browsing property of real-time camera vision, the navigation software Wikitude Drive shows new directions in consumer navigation. The map has disappeared in favor of direct on-screen visual and acoustic feedback. While this application uses the mobile screen of phones for live camera vision layered with data, this combination of real-time video feed and on-screen layering on transparent screens has been developed in the military and aviation, much like the integration of data-layering into special glasses or contact lenses, and heads-up display(HUD) layering of information on the windshield of our cars. The pos-sibilities for commercial applications of this type of on-screen navigation are readily apparent. In boasting rhetoric, CEO of Mobilizy (developers of Wikitude Drive) Philipp Breuss-Schneeweis suggests:
[Wikitude Drive] is a light-weight navigation system which takes a different approach than all other navigation systems: You see the real street on your mobile phone, instead of 2D or 3D maps. […] There is a lot of room to grow in this area when you imagine the possibilities by having access to the huge number of mobile services and points of interest that are already available on mobile devices. Imagine driving by virtual billboards of your favorite fast food chain, or simply having an alert when one of them is nearby. This is going to happen within Wikitude Drive. The Wikitude platform offers […] a fantastic base to sell premium content or to display location based ads.”17
5. performative cartography 161
According to this rhetoric, the device will be a true competitor, rivaling for first place with spatial reality itself. This disregards the question of whether anyone really needs“millions of POIs”.
Despite the commercial nature of the latter application, these examples gener-ally demonstrate that augmentation is a form of creative contribution, which not only adds to space but inherently also modifies it. It creates hybrid space. I wrote above that this use of the wordcreativedoes not always imply artistic creation, but simply the act of making. Nevertheless, the word also suggests that the categories are porous. The possibility of activating the more traditional sense of creativity has the advantage of debunking an exalted, romantic vision of art that tradition-ally accompanies the qualifier‘creative’and bringing out the participatory poten-tial of creation. Margriet Schavemaker (2010), Head of Research and Collections at the Stedelijk Museum in Amsterdam and initiator of the earlier mentioned ARtours, the museum’s project for AR tours for modern art, has pointed this out.
She intimates that augmentation itself is at the core of art in general.
The mash-up logic we can recognize in the navigation of a layered reality entails mnemonic, temporal and experiential aspects of mobility. First of all, it engages with objects in their specific place, while adding temporal layers: a form of mnemonic spacing. This logic requires some sort of spatial stability: objects need to be in their place for some time in order to function as markers for their tags. As such, the logic relies on archival information attached to a spatial pres-ence. AR applications are built on databases (archives) of metadata attached to geospatial information. Secondly, the mash-up logic provides means to experi-ence a‘different’city. It adds, changes, enhances and constructs a city of differ-ence. The augmented reality navigator is an interactive performer, erecting a city of difference.18
These AR browsers and applications construct a particular kind of cartography.
Like any kind of cartography they are information-based, but this information can be modified and personalized. Moreover, it is an interactive cartography in that it is responsive to input. The navigator operates it individually on a small, handheld touchscreen and the cartography activates a subjective perspective on the directly surrounding space, unlinking the abstracted bird’s-eye view of space in tradi-tional paper maps. The layeredness of the augmented-reality image is a superim-posing of different spatial representations: one based on photographic/filmic framing, and the other a dataspace.
Discussing new-generation AR navigation systems for cars, Tristan Thielman makes the comparison with Edward Soja’s (1996) conception of first, second, and thirdspace:
In accordance to [Soja] the new generation of navigation systems that project the travel route onto the windscreen can also be described as the rise of the perspective of a third space. The driver is himself in the first space and
through the windscreen sees a first space that can be experienced physically.
Via the head-up display, a second space is simultaneously projected before his eyes as a mental concept of space. These spaces, when overlaid and integrated into each other, represent something like a“both/and”instead of an“either/
or”through this hybridity, mobility and simultaneity. Such a complex under-standing of space opens up new spaces. (2007: 70)
The analogy with Soja’s thirdspace is that of a conceptual and experiential cate-gory. As Thielmann seems to suggest here, this spacing is a quality of the experi-ence of hybridity. Augmented reality thus brings about not the sum of layers, but a whole new dimension to the experience of space.19
Like cartography in de Certeau’s sense, augmented reality provides a practiced narrative in that it tells spatial stories in the making: it makes experiences unfold in space at the moment of their occurrence. Hence, it is procedural, in the sense that movement through space and interaction with on-screen layers of digital information to off-screen geographical and material presence unfolds in time.
But not only does it take time, it becomes over time. A conception of time that includes the productive, or literally creative aspect of time, is relevant here; it includes changeintime.
This puts us with our historical feet back on the ground. This new technology has much in common with the age-old habit of walking, and in this links back to, say, Baudelaire’s flâneur as leisurely stroller. The cartographic principle of (AR) browsing is a synthesis of the two other models, that of touring and tagging.
Incorporating geotagging as a principle, the spatial logic is that of cut-outs and layers of information. Being structured as tours, the engagement is not visual, fixated and distanced, but haptic, fluid and procedural.