SENSEVAL

Evaluating Word Sense Disambiguation Systems

Under the auspices of ACL-SIGLEX, EURALEX, ELSNET and EU projects ECRAN and SPARKLE

The project

There are now many computer programs for automatically determining which sense a word is being used in. One would like to be able to say which were better, which worse, and also which words, or varieties of language, presented particular problems to which programs. SENSEVAL is designed to meet this need.

The first SENSEVAL is taking place in the summer of 1998, for English, French and Italian.

For specifications of the French and Italian tasks see the ROMANSEVAL site.

On this website:

GENERAL

Workshop Programme
Directions to Herstmonceux
Registration Form
Planning and Programme Committee
proceedings.asc Contains details for submission of paper versions of workshop presentations
Call for participating systems (old)

ENGLISH: specifications and resources

Participating Systems
Scoring Specifications for scoring
mr.asc Melamed-Resnik proposals for probabilistic scoring
mne-uid.map Mapping between sense identifiers, sense mnemonics and structured sense numbers
output.asc Output format specifications
wn1.5_hector
wn1.6_hector

PAPERS

SENSEVAL: An Exercise in Evaluating Word Sense Disambiguation Programs A. Kilgarriff. In Proc. LREC, Granada, May 1998. Pp 581--588. Postscript, 6 pages.
Gold Standard Datasets for Evaluating Word Sense Disambiguation Programs A. Kilgarriff. To appear in Computer Speech and Language, Special Issue on Evaluation. Postscript, 24 pages.

The SENSEVAL committee wishes to thank Oxford University Press, Cambridge University Press, the UK Engineering and Physical Sciences Research Council, ELRA and Addison Wesley Longman for their contributions to the initiative.