English SENSEVAL resources
in the public domain

Introduction

HECTOR

For a sample of a few hundred words, the OUP lexicographers used the corpus to develop dictionary entries, and in the course of the lexicography, they associated a sense-tag with every instance of the word in the corpus. The rules and strategies they followed in the corpus-tagging and dictionary-writing are described in their Policy Guide.

Lexical sample

Nouns		Verbs		Adjectives		Indeterminates
-n	N	-v	N	-a	N	-p	N
accident	267	amaze	70	brilliant	229	band	302
behaviour	279	bet	177	deaf	122	bitter	373
bet	274	bother	209	floating	47	hurdle	323
disability	160	bury	201	generous	227	sanction	431
excess	186	calculate	217	giant	97	shake	356
float	75	consume	186	modest	270
giant	118	derive	216	slight	218
knee	251	float	229	wooden	195
onion	214	invade	207
promise	113	promise	224
rabbit	221	sack	178
sack	82	scrap	186
scrap	156	seize	259
shirt	184
steering	176
TOTAL	2756	TOTAL	2501	TOTAL	1406	TOTAL	1785

"Indeterminates" were words for which the task involved determining major word class, as well as determining word sense. For the other tasks, major word class was given.
For word selectin strategy, see bibliography.

Dictionary entries

A postscipt versions of the full 35-word dictionary is available (gzipped) here.

Test Corpus

700001 
John Dos Passos wrote a poem that talked of `the <tag>bitter</> beat look, the scorn on the lip."   

700002
The beans almost double in size during roasting. 
Black beans are over roasted and will have a <tag>bitter</> flavour and insufficiently roasted beans are pale and give a colourless, tasteless drink. 

• All files are simple ASCII.
• Instances are separated by a blank line.
• The first line of the instances contains a reference number (a six-digit number beginning with 7)
• Each sentence is on a new line; the default number of sentences of context is two, but sometimes there are more, sometimes just one.
• The word to be disambiguated is preceded by <tag> and followed by </>

Gold Standard Taggings

700002:kind
700003:unstint or kind
700004:copious

Each sense had a numerical unique identifier, as well as a mnemonic. The mapping between the two is available here.

The files are available, tar'd and gzipped, here. A sample of three items (rabbit, generous and shake) is also available uncompressed here.

Training corpus

800004
Mr Purves is tight-lipped about what happens then. 
He recently vexed rumour-mongers, who <tag "520051">bet</> on a bid for Midlan sooner rather than later, by declining to disclose Hongkong Bank's inner reserves when the ban reported its 1989 results on March 13th.   

800005
Of our other leisure facilities, Blackbird Leys Leisure Centre, opened last autumn was given a six-month event-free trial and will now begin to attract the large events and exhibitions to subsidise its running costs. 
Finally, Mr Hugh-Jones loses his <tag "519914?">bet</>: 400,000 people attended Temple Cowley pools last year.  

• All files are simple ASCII.
• Instances are separated by a blank line.
• The first line of the instances contains a reference number (a six-digit number beginning with 8)
• Each sentence is on a new line; the default number of sentences of context is two, but sometimes there are more, sometimes just one.
• The word to be disambiguated is preceded by <tag sensenum> and followed by </>
• The sense number is given as a reference number, rather than a mnemonic (mapping available here) and may have 'suffixes' (such as the ? in the second example above): see the Policy Guide for details.

A further dataset in this format is available on request.

Results

WordNet mappings

• Framework and Results for English Adam Kilgarriff and Joseph Rosenzweig

For the HECTOR project, see:

• Sue Atkins. Tools for corpus-aided lexicography: the HECTOR project. In Acta Linguistica Hungarica 41, 1992--93. Budapest. Pp 5--72.

A background paper on SENSEVAL:

• SENSEVAL: An Exercise in Evaluating Word Sense Disambiguation Programs A. Kilgarriff. In Proc. LREC, Granada, May 1998. Pp 581--588. Postscript, 6 pages.

• Gold Standard Datasets for Evaluating Word Sense Disambiguation Programs A. Kilgarriff. Computer Speech and Language 12 (4), 1998. Special Issue on Evaluation. Edited by R. Gaizauskas. Pp 453--472. Postscript, 24 pages.

This page is maintained by Adam Kilgarriff and was last updated on 6 July 1999.