Across-genres and empirical evaluation of state of- the-art treebank-style parsers

• Autorzy: Rus, V. , Hempelmann, C. F.
• Konferencja: Human Language Technologies as a challenge for Computer Science and Linguistics (2; 21-23.04.2005; Poznań, Poland)
• Języki publikacji: EN

Abstrakty

EN

This paper evaluates a series of freely available, state-of-the art parsers on a standardbenchmark as well as with respect to a set of data impotrant to measure text cohesion in the context of learning technology. We outline advantages of the existing parsing technologies and make recommendations. The performance reported uses traditional measures as well as novel dimensions for parsing evaluation to develop a gold standard for narrive and expository texts. To our knowledge this is the first attempt to evaluate parsers across genres.

Słowa kluczowe

EN

parsing evaluation; bracketing; learning technologies

• Czasopismo: Archives of Control Sciences
• Rocznik: 2005
• Tom: Vol. 15, no. 4
• Strony: 645-654
• Opis fizyczny: Bibliogr. 16 poz., tab.

Twórcy

autor

Rus, V.

• Institute for Intelligent Systems, University of Memphis, TN 38120, USA,

autor

Hempelmann, C. F.

• Institute for Intelligent Systems, University of Memphis, TN 38120, USA,

Bibliografia

• [1] S. Abney, R.E. Schapire and Y. Singer : Boosting applied to tagging and pp attachment. In Proc. Joint SIGDAT Conf. on Empirical Methods in Natural Language Processing and Very Large Corpora, (1999), 38-45.
• [2] E. Brill and P. Resnik : A rule-based approach to prepositional phrase attachment disambiguation. In Proc. 15th Int. Conf. on Computational Linguistics, (1994).
• [3] J. Carroll , E. Briscoe and A. Sanfilippo : Evaluation of Natural Language Processing Systems: Final Report, chapter Parser evaluation: current practice, ECDG-XIII LRE EAGLES Document EAG-II-EWG-PR.1., (1999), 140-150.
• [4] E. Charniak : Statistical parsing with a context-free grammar and word statistics. Proc. Fourteenth National Conf. on Artificial Intelligence, AAAI Press/MIT Press, Menlo Park, (1997).
• [5] E. Charniak. A maximum-entropy-inspired parser. In Proc. North-American Chapter of Association for Computational Linguistics, Seattle, Washington, (2000).
• [6] M. Collins : A new statistical parser based on bigram lexical dependencies. In Proc. 34th Annual Meeting of the ACL, Santa Cruz, (1996).
• [7] M. Collins : Three generative, lexicalised models for statistical parsing. In Proc. 35th Annual Meeting of the Association for Computational Linguistic, Madrid, Spain, (1997).
• [8] M. Collins and J. Brooks : Prepositional phrase attachment through a backed-off model. In Proc. Third Workshop on Very Large Corpora, Cambridge, (1995).
• [9] C. Cremers : On Parsing Coordination Categorially. PhD thesis, Leiden University, 1993.
• [10] A. C. Graesser , D.S. Mcnamara , M. M. Louwerse and Z. Cai : Cohmetrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36(2), (2004), 193-202.
• [11] C. MacLeod , R. Grishman and J. Sterling : Evaluating parsing strategies using standardized parse files. In Proc. 3rd Conf. on Applied Natural Language Processing, (1992), 156-161.
• [12] M. Grootveld : Parsing Coordination Generatively. PhD thesis, Leiden University, 1994.
• [13] D. Klein and C. Manning : Accurate unlexicalized parsing. In Proc. 41st Annual Meeting of the Association for Computational Linguistic, Sapporo, Japan, (2003).
• [14] D. Lin : A dependency-based method for evaluating broad-coverage parsers. In Proc. Int. Joint Conf. on Artificial Intelligence, (1995), 1420-1427.
• [15] A. Ratnaparkhi , J. Renyar and S. Roukos : A maximum entropy model for prepositional phrase attachment. In Proc. ARPA Workshop on Human Language Technology, (1994).
• [16] S. Sekine and R. Grishman : A corpus-based probabilistic grammar with only two non-terminals. In Proc. Int. Workshop on Parsing Technologies, (1995), 216-223.