Referring Expression Generation Challenge 2008
----------------------------------------------

Following the success of the 2007 Attribute Selection for Generating
Referring Expressions (ASGRE) Challenge, a second Referring Expression
Generation Evaluation event is being organised, to be presented and
discussed during a special session at INLG'08.  This year, both the range
of shared tasks and the evaluation programme will be extended.  The REG
Challenge 2008 will involve two different data sets:

  I. TUNA Corpus of Referring Expressions: This is the same data as in
  the ASGRE Challenge, but with new, previously unseen test data.

  II. GREC-2 Corpus: This is a new corpus of references to named
  entities in free text, with a test set that has additional
  human-selected referring expressions.

The submission tracks will be as follows:

  1. Task 1 (TUNA): Attribute selection for distinguishing descriptions.

  2. Task 2 (TUNA): Realisation of referring expressions from a given
     semantic representation.

  3. Task 3 (TUNA): Attribute selection and realisation combined.

  4. TUNA Open Track: Any work involving the TUNA data.

  5. TUNA Evaluation Methods: Any work involving evaluation of Tasks
     1-3.

  6. Task 4 (GREC-2): Given a referent, a discourse context and a list
     of possible referring expressions, select the referring expression
     most appropriate in the context.

  7. GREC Open Track: Any work involving the GREC data.

  8. GREC Evaluation Methods: Any work involving evaluation of Task 4.

The assessment criteria used will range from automatically computed
metrics to human evaluation.  The criteria envisaged for the TUNA tasks
are minimality, humanlikeness, and identification/comprehension by human
readers; those for the GREC task are humanlikeness and the DUC criteria
of coherence and referential clarity. Subject to feasibility, system
processing efficiency is another possible criterion.

A separate call for participation in the REG Challenge will be issued. In
the meantime, please contact Anja Belz and Albert Gatt at gre-stec (at)
itri.brighton.ac.uk for further information.