Compositional Approach

Despite the significant improvement in coverage, the accuracy of the grammar was not investigated for the “words-with-spaces” approach in Zhang et al. (2006).

As Sag et al. (2002) pointed out, such an approach has several major limitations. First, the approach suffers from a flexibility prob-lem. For example, Sag et al. (2002) pointed out that a parser which lacks sufficient knowledge of verb-particle constructions might cor-rectly assign look up the tower two interpretations (“glance up at the

tower” vs. “consult a reference book about the tower”), but fail to treat the subtly different look the tower up as unambiguous (“consult a reference book . . . ” interpretation only). For highly variable MWEs with such or other kinds of flexibility, the linguistic precision will not be fully captured in the words-with-spaces approach. Moreover, this simple approach also suffers from a lexical proliferation problem.

For example, light verb constructions often come in families, e.g., take a walk, take a hike, take a trip, take a flight, . . . . Listing each such expression results in considerable loss of generality and lack of prediction.

A closer look at the MWEs not properly handled by the grammar reveals that only a small proportion of them can be handled appro-priately by the “words-with-spaces” approach of Zhang et al. (2006).

Simply adding new lexical entries for all MWEs can be a workaround for enhancing the parser coverage, but the quality of the parser output is clearly linguistically less adequate.

On the other hand, we also find that a large proportion of MWEs that cannot be correctly handled by the grammar can be covered properly in a compositional way by adding one lexical entry for the head (governing) word of the MWE. For example, the expression foot the bill will be correctly handled with a standard head-complement rule, if there is a transitive verb reading for the word foot in the lexicon. Some other examples are: to put forward, the good of, in combination with, . . . , where lexical extension to the words in bold will allow the grammar to cover the MWEs. In this section, we focus on the constructional approach for the acquisition of new lexical entries for the head words of the MWEs.¹

It is arguable that such an approach may lead to some poten-tial grammar overgeneration, as there is no selectional restriction ex-pressed in the new lexical entry. However, as far as the parsing task is concerned, such overgeneration is not likely to reduce the accuracy of the grammar significantly as we will show through a thorough

1The combination of the “words-with-spaces” approach with the constructional ap-proach we propose here is an interesting topic that we want to investigate in future research.

evaluation.

With a similar setting to the one described in Section 4.4.1, we did two more acquisition experiments with two lists of candidate MWEs.

The first list (L-I henceforth) contains 200 randomly selected MWEs from the high permutation entropy, high WWWHITSn-grams. The sec-ond list (L-II henceforth) is selected with a more thorough validation step by combining three statistical measures: the mutual information (MI), χ² test, and permutation probability (PP). The candidateMWEs are ranked according to all these measures, and the top 30 MWEs with the highest average rankings are selected into the second list. The difference with Section 4.4.1 is that these evaluations are done on a slightly more recent version of the ERG ( jul-06)².

With both lists of MWEs, the following steps are taken to find the head word with heuristics:

• the n-grams are PoS tagged with an automatic tagger;

• finite verbs in the n-grams are extracted as head words;

• nouns are also extracted if there is no verb in the n-gram.

Occasionally, tagger errors might introduce wrong head words. How-ever, the lexical type predictor of Zhang and Kordoni (2006) that we used in our experiments did not generate interesting new entries for them in the subsequent steps. As a result, we obtained 52 head words for L-I and 20 for L-II.

With the two lists of MWEs, we extracted two sub-corpora from the BNC, with each sentence containing at least one of the MWEs in the corresponding lists. The sub-corpora contain 2,463 sentences and 674 sentences, respectively. The lexical acquisition technique described by Zhang and Kordoni (2006) was used with these sub-corpora in order to acquire new lexical entries for the head words. The lexical

2The major difference between the ERG version jan-06 and jul-06 is the size of the lexicon. jul-06 shows much better overall coverage on the BNC, largely due to the semi-automatic lexical extension. But no significant change in the grammar accuracy is observed.

acquisition model was trained with the Redwoodstreebank, following the techniques presented in Chapter 3.

The lexical prediction model predicted the most plausible lexical type in that context for each occurrence of the head words. Only those predictions that occurred 5 times or more were taken into con-sideration for the generation of the new lexical entries. As a result, we obtained 50 new lexical entries for L-I and 21 for L-II.

These new lexical entries were later merged into the ERG lexicon.

To evaluate the grammar performance with and without these new lexical entries, we

1. parsed the sub-corpora with and without new lexical entries and compared the grammar coverage;

2. inspected the parser output manually and evaluated the gram-mar accuracy.

In parsing the sub-corpora, we used the PET parser (Callmeier, 2001). For the manual evaluation of the parser output, we used the treebanking tools of the [incr tsdb()] system (Oepen, 2001).

Table 4.4 shows that for L-I the grammar coverage improved sig-nificantly (from 4.3% to 16.7%) with the acquired lexical entries for the head words of the MWEs. With a more carefully validated list L-II, the coverage improvement is even more noticeable (over 15%, see Table 4.5). These improvements in coverage are largely comparable to the result we observed in the “words-with-space” approach (from 4.3% to 18.7%). Also, we discovered that with a more carefully fil-tered list of candidates, the average analysis number drops when new lexical entries are added, somehow indicating that those new entries do not lead to terribly more readings with higher lexical ambiguity.

Comparing the numbers of the new lexical entries added, we no-ticed that the compositional approach achieved comparable coverage improvement with fewer new lexical entries. This suggests that the lexical entries acquired in our experiment are of much higher linguis-tic generality.

To evaluate the grammar accuracy, we manually checked some of the parser outputs from each sub-corpus after the lexical extension.

item # +entry # parsed # analysis φ coverage %

-MWE 2463 – 107 127.88 4.3%

+MWE 2463 52 412 178.33 16.7%

Table 4.4: ERG coverage with and without lexical acquisition for the head words of L-I MWEs (compositional)

item # +entry # parsed # analysis φ coverage %

-MWE 674 – 48 335.08 7.1%

+MWE 674 21 153 285.01 22.7%

Table 4.5: ERG coverage with and without lexical acquisition for the head words of L-II MWEs (compositional)

A sentence was marked as “accepted”, if one of the analyses was correct, otherwise it was marked as “rejected”. The results are listed in Table 4.6.

lists sentence # accepted # %

L-I 100 81 81.0%

L-II 153 124 81.0%

Table 4.6: ERG accuracy after lexical acquisition for the head words of MWEs

Baldwin et al. (2004) reported earlier that, for BNC data, about 83% of the sentences covered by the ERG have a correct parse. In our evaluation, we observed very similar accuracies. We also found that the disambiguation model as described by Toutanova et al. (2002) performed reasonably well, and the best analysis is ranked among top-5 for 66% of the cases, and top-10 for 75%.

All of these results indicate that our approach for lexical acquisition of head words of MWEs achieves a significant improvement in grammar coverage without damaging the grammar accuracy. Optionally, the grammar developers can check the validity of the lexical entries before they are added into the lexicon. This semi-automatic procedure can largely reduce the manual work of grammar writers.