A Distributed Staged Architecture for Multimodal Applications (Extended Abstract)
∗Alessandro Costa Pereira1,2, Falk Hartmann1,2, Kay Kadner1,2 1Department of Computer Science 2SAP Research CEC Dresden
Technische Universit¨at Dresden SAP AG
Dresden, Germany Dresden, Germany
{alessandro.costa.pereira, falk.hartmann, kay.kadner}@sap.com
The Services for Nomadic Workers project (SNOW) aims at enabling the widespread use of multimodal documentation for mobile operations in an industrial environ- ment. One of the use cases covered by the project is that of a service worker in the aircraft maintenance domain. In order to get rid of paper-based maintenance documentation (so-called procedures), the worker should get electronic access to it.
Because of the requirements (hands-free operation) and the working environment (noisy, changing light conditions, restricted), access to procedures must be multi- modal. This was a major requirement of the SNOW project besides the design of an architecture that is as domain-neutral as possible, i.e., the number of parts to be exchanged when switching to another domain had to be minimized.
The SNOW consortium decided to follow approaches like Nightingale [WAQ04]
and to use the Multimodal Interaction Framework (MMI-F, [W3C03]) as the base for the SNOW architecture. This specification defines basic building blocks of a multimodal application, their responsibilities and collaboration partners.
The MMI-F is only influencing architectures with respect to multimodality, it is not an architecture itself (see [W3C03]). It soon became clear that the MMI-F is a role model [RG98] that could be used to introduce the multimodality aspect to existing architectures. Understanding the MMI-F this way makes it easier to incorporate it into a concrete architecture.
During the mapping of the MMI-F role model onto the SNOW architecture it was neccessary to map some roles to multiple components. This kind of denormalization can have several reasons: performance improvements, special requirements and/or deficiencies in the role model.
In contrast to existing multimodal architectures like QuickSet [CJM+97] or Embassi [ERMS03], the SNOW architecture is specifically designed to be as domain-neutral
∗This work has been partially supported by the European Union within the FP6 IST STREP SNOW (FP6-511587).
255
as possible. This has been achieved by using a staged architecture [Aßm05], which allows to implement separation of concerns in an architectural sense. In SNOW, the stages separate domain-specific from domain-neutral componentsas well asthe device-specific from device-independent components.
The staged architecture in SNOW is formed by the following components: The Doc- umentation Application represents the domain-specific, device-independent stage, where the procedures are loaded from a database and transformed into XML docu- ments in an intermediate language (D3ML, [GHKP06]). This is the only component that needs to be exchanged when switching to a different domain. Next, the Dialog Manager implements a domain-neutral, device-independent dialog model by inter- preting requests for documents (which might be procedures), fetching documents from the Documentation Application and transforming them with dialog model specific content into new D3ML documents. Finally, the domain-neutral, device- independent Adaptation Manager transforms these into device-specific documents.
The SNOW architecture has proven to be quite stable and mature at the end of the project in 2006. It has been shown that the MMI-F is in fact a role model. As users of the MMI-F can benefit from this knowledge, the specification should state this explicitly. Furthermore SNOW shows that a staged architecture is a powerful architectural style to implement domain-neutral multimodal applications.
References
[Aßm05] Aßmann, U. Architectural styles for active documents. Science of Computer Programming, 56(1-2):79–98, 2005.
[CJM+97] Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. QuickSet: Multimodal Interaction for Distributed Applications. InProceedings of ACM Multimedia 1997, pages 31–40, 1997.
[ERMS03] Elting, C., Rapp, S., M¨ohler, G., and Strube, M. Architecture and imple- mentation of multimodal plug and play. InICMI ’03: Proceedings of the 5th international conference on Multimodal interfaces, pages 93–100, New York, NY, USA, 2003. ACM Press.
[GHKP06] G¨obel, S., Hartmann, F., Kadner, K., and Pohl, C. A Device-Independent Multimodal Mark-up Language. InINFORMATIK 2006: Informatik f¨ur Men- schen, Band 2, pages 170–177, 2006.
[RG98] Riehle, D. and Gross, T. Role Model Based Framework Design and Integra- tion. ACM Press, pages 117–133, 1998.
[W3C03] The World Wide Web Consortium. Multimodal Interaction Framework, 2003. http://www.w3.org/TR/2003/NOTE-mmi-framework-20030506/ (vis- ited 2007, February 2nd).
[WAQ04] West, D., Apted, T., and Quigley, A. A context inference and multi-modal approach to mobile information access. In Artificial Intelligence in Mobile Systems, pages 28–35, Nottingham, England, 2004.
256
