Next: Comparison with CLARE
Up: Evaluation and Results
Previous: Comparison with IBM
The evaluation in this section was done with the earlier 1995 release
of the grammar. This section compares XTAG to the Alvey Natural
Language Tools (ANLT) Grammar. We parsed the set of LDOCE Noun Phrases
presented in Appendix B of the technical report ([#!Carroll93!#])
using XTAG. Table G.7 summarizes the results of this
experiment. A total of 143 noun phrases were parsed. The NPs which
did not have a correct parse in the top three derivations were
considered failures for either system. The maximum and average number
of derivations columns show the highest and the average number of
derivations produced for the NPs that have a correct derivation in the
top three. We show the performance of XTAG both with and without the
tagger since the performance of the POS tagger is significantly
degraded on the NPs because the NPs are usually shorter than the
sentences on which it was trained. It would be interesting to see if
the two systems performed similarly on a wider range of data.
System |
# of |
# parsed |
% parsed |
Maximum |
Average |
|
NPs |
|
|
derivations |
derivations |
ANLT Parser |
143 |
127 |
88.81% |
32 |
4.57 |
XTAG Parser with |
143 |
93 |
65.03% |
28 |
3.45 |
POS tagger |
|
|
|
|
|
XTAG Parser without |
143 |
120 |
83.91% |
28 |
4.14 |
POS tagger |
|
|
|
|
|
- {Comparison of XTAG and ANLT Parser
Next: Comparison with CLARE
Up: Evaluation and Results
Previous: Comparison with IBM
XTAG Project
http://www.cis.upenn.edu/~xtag