Relational evaluation schemes

Briscoe, Ted, Carroll, John, Graham, Jonathan and Copestake, Ann (2002) Relational evaluation schemes. In: Beyond PARSEVAL Workshop at the 3rd International Conference on Language Resources and Evaluation, Las Palmas, Gran Canaria.

Full text not available from this repository.


We describe extensions to a scheme for evaluating parse selection accuracy based on named grammatical relations between lemmatised lexical heads. The scheme is intended to directly reflect the task of recovering grammatical and logical relations, rather than more arbitrary details of tree topology. There is a manually annotated test suite of 500 sentences which has been used by several groups to perform evaluations. We are developing software to create larger test suites automatically from existing treebanks. We are considering alternative relational annotations which draw a clearer distinction between grammatical and logical relations in order to overcome limitations of the current proposal.

Item Type: Conference or Workshop Item (Paper)
Schools and Departments: School of Engineering and Informatics > Informatics
Depositing User: John Carroll
Date Deposited: 06 Feb 2012 19:24
Last Modified: 12 Apr 2012 14:30
📧 Request an update