In this chapter we have provided an introduction into the treatment of parsing as a constraint satisfaction problem. Weighted-constraints have been presented as a particularly useful refinement to the formalism of symbolic constraint-based parsing.
The main advantages of a symbolic parser operating with weighted constraints rather than generation-rules lie in its capability to a) model graded preferences rather than simply to categorise as grammatical or ungrammatical, b) provide analytic feedback, and c) react more robustly to unknown input.
We have outlined the benefits of WCDG’s dependency formalism and its relational representation. Its disadvantage lies in the limitation that it expresses linguistic de-pendencies between individual words rather than between bracketed, more complex linguistic entities such as phrases.
We have also provided a detailed description of the system capabilities of WCDG1, an implementation of a weighted-constraint dependency parser. WCDG1 is of par-ticular interest to our modelling challenge because of its generic predictor interface.
Within clearly defined limitations, this interface permits to integrate parser-external, non-linguistic information into the parsing process. Access to the predictions is achieved via suitably formulated integration constraints in the grammar that are processed at parse time.
argument to thepredict()function isX@id, which normally references a specific homonym in the input sentence unambiguously. However, at code level, thepredict()function has been implemented such that the homonym-specific referenceX@idis abstracted into a reference to the homonym’s slot only.
Additional modelling requirements based on WCDG1’s capabilities and limitations have been motivated in order to achieve the integration of cross-modal context into the parsing process. The application-focused requirements identified in this chapter complement the collection of modelling requirements from the preceding Chapters2 and 3 and conclude the process of requirements collection in this thesis.
It needs to be noted that the list of collected requirements cannot make a claim to completeness, nor are the requirements structurally homogeneous or uniform in granularity. The main use of the requirements will be as a benchmark for the func-tional scope of the model we intend to argue for in PartII of this thesis. The com-pilation of a comprehensive, uniform and homogeneous collection of requirements for the interaction between vision and language certainly requires further rigorous investigation and warrants to undertake a separate research effort in its own right.
In the following part of this thesis, we describe in detail our model implementation which was designed with the intention to achieve maximum coverage of the iden-tified requirements while ensuring actual implementability of the specified system.
We also engage in a discussion to what extent our model meets the identified mod-elling requirements. For ease of revision, the list of all 32 modmod-elling requirements is given in Appendix I.
Model Implementation
71
The WCDG2 Parser
The implementation of our model for the interaction of non-linguistic modalities with language centres around WCDG2, a functionally enhanced version of the weighted-constraint dependency parser WCDG1 described in Chapter4. This chapter outlines WCDG2’s functional enhancements over WCDG1 and discusses the parser’s inter-actions with the other components in our model.
As a general guideline in the design and specification of WCDG2, we have kept the number and extent of functional changes over WCDG1 to a minimum. Modifica-tions were only made in cases where WCDG1’s features made the implementation of vital aspects of our model difficult or impossible. Another guideline was to leave the syntactic processing of WCDG1 unchanged. Any additional capabilities included in WCDG2 are add-ons to WCDG1’s existing functionality.
The enhancements implemented in WCDG2 comprise semantic extensions to the constraint base and the lexicon, modifications to the predictor interface and an ex-pansion of the argument structure for prediction access in the grammar. In line with our second design guideline, no changes were made to WCDG’s central con-straint satisfaction algorithms nor to the heuristic search routines of frobbing (cf.
Section 4.2.4). In our model implementation, we build on WCDG1’s large-coverage grammar of German and leave its constraints for syntactic processing unchanged.
5.1 Architectural Overview
After loading an extended lexicon and a semantically enhanced grammar, WCDG2 receives its input sentence. In the preprocessing phase, WCDG2 integrates a plau-sibility predictor component (PPC) via an extended version of WCDG1’s predictor interface (cf. Section4.2.5). The purpose of the PPC is to score semantic dependen-cies in the input sentence based on a representation of visual context information.
To this end, the PPC establishes communication with a reasoner component. The reasoner accesses a knowledge representation of visual context. The knowledge rep-resentation is made up of two components: a situation-invariant ontology containing hierarchical lexical and world knowledge (theT-Box) and a situation-specific repre-sentation of situation information (the A-Box).
73
Figure 5.1: Components and their interaction in the Context Integration Architecture (CIA).
Based on this context information, the PPC computes score predictions for semantic dependencies between words in the input sentence and returns them to the parser for access at parse time. The PPC’s score predictions are based on semantic con-text information and directly affect the assignment of the semantic dependencies in WCDG2’s semantic representation of the input sentence. Syntactic analysis is affected indirectly by these predictions via the correspondence rules between se-mantic and syntactic representation as specified in the syntax-sese-mantics interface in WCDG2’s enhanced grammar. The syntax-semantics interface contains correspon-dence rules between the representations of syntax and semantics which ensure that semantic and syntactic representations align. Thus, the contextual influence upon semantic representation is propagated into syntactic analysis with semantic medi-ation. WCDG2 optimises the semantic and syntactic dependency structures by a heuristic search for the minimum of the severities of all constraint violations.
We refer to this entire framework as our Context Integration Architecture (CIA).
A schematic overview over the CIA is given in Figure 5.1. The CIA components and their interaction with each other will now be outlined in detail in the following sections.