Comparison with CLARE

The evaluation in this section was done with the earlier 1995 release of the grammar. This section compares the performance of XTAG against that of the CLARE-2 system ([#!clare-report92!#]) on the ATIS corpus. Table  shows the performance results. The percentage parsed column for both systems represents the percentage of sentences that produced any parse. It must be noted that the performance result shown for CLARE-2 is without any tuning of the grammar for the ATIS domain. The performance of CLARE-3, a later version of the CLARE system, is estimated to be 10% higher than that of the CLARE-2 system.^29.6

 

Table: Performance of CLARE-2 and XTAG on the ATIS corpus

System	Mean length	% parsed
CLARE-2	6.53	68.50%
XTAG	7.62	88.35%

 

In an attempt to compare the performance of the two systems on a wider range of sentences (from similar genres), we provide in Table  the performance of CLARE-2 on LOB corpus and the performance of XTAG on the WSJ corpus. The performance was measured on sentences of up to 10 words for both systems.

 

Table: Performance of CLARE-2 and XTAG on LOB and WSJ corpus respectively

System	Corpus	Mean length	% parsed
CLARE-2	LOB	5.95	53.40%
XTAG	WSJ	6.00	55.58%