Anja Belz Reader in Computer Science School of Computing, Engineering and Mathematics University of Brighton, Lewes Road, Brighton, BN2 4GJ, UK Email: A dot S dot Belz at brighton dot ac dot uk Tel: +44 (0)1273 642909; Fax: +44 (0)1273 642908 Visiting Senior Research Fellow Natural Language & Computational Linguistics Group Department of Informatics, University of Sussex

Research Team:

1. V&L Net (EPSRC): EPSRC Network on Vision and Language, 2010-2013
2. Language Generation Benchmarking Tasks:
   1. Generation Challenges 2011 (EPSRC)
   2. Generation Challenges 2010 (EPSRC)
   3. Generation Challenges 2009 (EPSRC)
3. Referring Expression Generation Benchmarking Tasks: 2007 and 2008 (EPSRC)
4. Prodigy (EPSRC): Probabilistic Deep Generation, 2007-2010
5. CoGenT (EPSRC): Controlled Generation of Text, 2003-2006
6. NECA (EC, IST): Net Environment for Embodied Emotional Conversational Agents, 2001-2003
7. PILLS (EC, ECD): Patient Information Language Localisation System, 2001
8. LCG (EC, TMR), Learning Computational Grammars, 1998-2001

1. Surface Realisation Task 2011: Information for Participants
2. Prodigy-METEO Corpus pre-alpha release; data as used in Belz & Kow, 2009; join Prodigy-METEO user mailing list by emailing Anja Belz (address see above).
3. Generation Challenges 2009 Participants Packs:
   1. TUNA-REG PACK [test data]
   2. GREC-MSR PACK [test data]
   3. GREC-NEG Task (to be made available after completion of GREC'10)
4. REG'08 Participants Packs:
   1. TUNA Tasks (TUNA-AS, TUNA-R, TUNA-REG) PACK; [TUNA-AS test data], [TUNA-R test data], [TUNA-REG test data]
   2. GREC Task (GREC-MSR) PACK; [test data]
5. ASGRE'07 Participants Pack (TUNA-AS Task) PACK; [test data]

• Memberships:
  • Editorial board, Computational Linguistics, 2011-2014
  • EPSRC Early Careers Focus Group
  • EPSRC College, 2010-
  • Generation Challenges Steering Committee
  • EPSRC Network on Vision and Language Steering Committee


• Organisation of benchmark tasks in natural language processing:
  • Generation Challenges 2011: the fifth NLG evaluation event involving multiple shared tasks in language generation; organiser, with Albert Gatt, Alexander Koller and Kristina Striegnitz; results presented at ENLG'11.
  • Generation Challenges 2010: the fourth NLG evaluation event involving multiple shared tasks in language generation; organiser, with Albert Gatt and Alexander Koller; results presented at INLG'10.
  • Generation Challenges 2009: the third NLG evaluation event involving multiple shared tasks in language generation; organiser, with Albert Gatt and Eric Kow; results to be presented at ENLG'09.
    1. TUNA-REG'09 Report
    2. GREC-NEG'09 Report
    3. GREC-MSR'09 Report.
  • The REG Challenge 2008: the second NLG evaluation event with multiple shared tasks in Referring Expression Generation; organiser, with Albert Gatt; results to be presented at INLG'08.
  • First NLG shared-task evaluation: The ASGRE Challenge: Organiser, with Albert Gatt and Ehud Reiter, Aberdeen University, and Jette Viethen, Macquarie University; results session at UCNLG+MT in 2007. [Overview and Evaluation Results] [Participants Reports].
    
    
• Chairing and organisation of conferences and workshops:
  • Fourth UCNLG Workshop: Language Generation and Evaluation (UCNLG+Eval): Organiser and co-chair (with Roger Evans, Brighton, Albert Gatt, Malta, and Kristina Striegnitz, Union College), EMNLP'11, Edinburgh, 30-31 July.
  • Third UCNLG Workshop: Language Generation and Summarisation (UCNLG+Sum): Organiser and co-chair (with Roger Evans, Brighton, and Sebastian Varges, Trento), ACL-IJCNLP 2009, Singapore, 6 August. [Proceedings]
  • Second UCNLG Workshop: Language Generation and MT (UCNLG+MT): Organiser and co-chair (with Sebastian Varges, Trento University), Copenhagen, 11 September 2007. [Proceedings].
  • INLG'06 Special Session on Sharing Data and Comparative Evaluation, Sydney, 15-16 July, 2006: Organiser, with Robert Dale, Macquarie University.
  • Workshop on Using Corpora for Natural Language Generation (UCNLG 2005): Organiser and co-chair (with Sebastian Varges, Stanford University). [Proceedings].
  • Third International Conference on Natural Language Generation (INLG 2004): Organiser and co-chair (with Roger Evans and Paul Piwek).
    
  
  
• Recent membership of programme committees and review committees:
  • 2012: ACL'12 Area Chair for Summarisation and Generation; EACL'12 mentor; CICLing'12 PC member; NAACL-HLT'12 PC member; LREC'12 PC member; ACL-SRW'12 PC member; EMNLP-CoNLL'12 PC member.
  • 2011: ACL'11 (Summarisation and Generation), ACL'11 (NLP Applications), IWCS'11, ACL'11 Workshop on Monolingual Text-To-Text Generation, ENLG'11, EMNLP'11 (Summarisation and Generation).
  • 2010: ACL'10 (Summarisation and Generation), NAACL'10 (Generation), LREC'10, INLG'10, EMNLP'10 (Summarisation and Generation; EMNLP'10 Best Reviewer Award).
  • 2009: ACL-IJCNLP'09 Area Chair for Summarisation and Generation; EACL'09 Area Chair for Summarisation, Generation and Multimodality; Reviewer for ENLG'09, LTC'09.
  • 2008: ACL'08 (Natural Language Generation), COLING'08 (Summarisation and Natural Language Generation), INLG'08, LREC'08, EMNLP'08 (Generation and Summarisation).
  • 2007: ACL'07 (Summarisation and Generation), EMNLP-CoNLL'07 (Grammar Learning (empirical); Phonology, Finite State Methods and Morphology), ENLG'07, L&TC 2007, NSF Workshop on Shared Tasks and Comparative Evaluation in Natural Language Generation 2007.
  
  
• Peer reviewing for funding bodies and journals: EC CIP, EC FP7, EPSRC (UK), ESRC (UK); Natural Language Engineering, Computational Linguistics, International Journal of Human-Computer Studies, Journal of Artificial Intelligence Research, Pattern Recognition, Linguistics.
  

(2007-2010): Probabilistic Deep Generation

1. Project home
   
   
2. Anja Belz and Eric Kow (2010), Assessing the Trade-Off between System Building Cost and Output Quality in Data-to-Text Generation. In Krahmer, E., Theune, M. (eds.) Empirical Methods in Natural Language Generation, Vol. 5980 of Lecture Notes in Computer Science, Springer, pp. 180-200. (Extended version of Belz and Kow, 2009.) [pre-proof PDF]
   
   
3. Anja Belz and Eric Kow (2010) Extracting Parallel Fragments from Comparable Corpora for Data-to-text Generation. In Proceedings of the 6th International Natural Language Generation Conference (INLG'10). [PDF]


4. Anja Belz and Eric Kow (2009), System Building Cost vs. Output Quality in Data-to-Text Generation, Proceedings of the 12th European Workshop on Natural Language Generation (ENLG'09).
   
   
5. Anja Belz (2008). Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. In Natural Language Engineering, 14(4), pp. 431-455. Cambridge University Press. [preproof: .pdf]
   
   
6. Anja Belz (2007). Probabilistic generation of weather forecast texts. In Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT'07). [.pdf]
   
   

Evaluation of NLG:


1. Anja Belz and Robert Dale (2006). Introduction to the INLG'06 Special Session on Sharing Data and Comparative Evaluation. To appear in Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). [.pdf]
   
   
2. Ehud Reiter and Anja Belz (2006). GENEVAL: A proposal for shared-task evaluation in NLG. To appear in Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). [.pdf]
   
   
3. Anja Belz and Adam Kilgarriff (2006). Shared-task evaluations in HLT: Lessons for NLG. To appear in Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). [.pdf]
   
   
4. Anja Belz and Ehud Reiter (2006). Comparing automatic and human evaluation of NLG systems. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL'06), pp. 313-320. [.pdf]
   
   
5. INLG'06 Special Session on Sharing Data and Comparative Evaluation, Sydney, 15-16 July, 2006: [Call for papers]
   
   
6. ELRA 10th anniversary workshop on HLT Evaluation, Malta, 1-2 December 2005:
   • presentation on Evaluation in NLG
   • informal report on the workshop.
     

Statistical Generation:


1. Anja Belz (2005). Statistical generation: Three methods compared and evaluated. In Proceedings of the 10th European Workshop on Natural Language Generation (ENLG'05). See list of publications below.
   
   
2. Anja Belz (2005). Corpus-driven generation of weather forecasts. In Proceedings of the 3rd Corpus Linguistics Conference (CL'05). See list of publications below.
   
   
3. Workshop on Using Corpora for NLG, Birmingham, 14 July 2005.
   
   
4. Implementation of probabilistic NLG technology will be available shortly as pCRU-1.0 (contact me if interested in helping with beta-testing).

Underspecification for Natural Language Generation:


1. Anja Belz (2004). Towards a General Framework for Underspecification in NLG and an Underspecification Language for MRS. Technical Report No. ITRI-04-17. Information Technology Research Institute, University of Brighton, UK. [.pdf]
   
   
2. Anja Belz (2004). Context-Free Representational Underspecification for NLG. Technical Report No. ITRI-04-18. Information Technology Research Institute, University of Brighton, UK. [.pdf]
   
   
3. Implementation of technology described in above TR available as CRU-1.0 (contact me if interested).
   
   
4. Anja Belz (2004). Underspecification for NLG. In: Belz et al. (eds.) INLG04 Posters: Extended abstracts of posters presented at the 3rd International Conference on Natural Language Generation (INLG 2004), pp. 9-13. Technical Report No. ITRI-04-01. Information Technology Research Institute, University of Brighton, UK.
   
   
5. Underspecification Bibliography 2004 (subsumes IMS Underspecification Bibliography, maintained until 1997)

NECA (2001-2003): A Net Environment for Embodied Emotional Conversational Agents

Anja Belz (2003). And Now with Feeling: Developments in Emotional Language Generation. Technical Report No. ITRI-03-21. Information Technology Research Institute, University of Brighton. 17 pages. [.pdf]

Nadjet Bouayad-Agha, Richard Power, Donia Scott and Anja Belz (2002). PILLS: Multilingual generation of medical information documents with overlapping content. Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC 2002), pp. 2111-2114. [.ps.gz]

Learning Computational Grammars (1998-2001): EU TMR Scheme

Other project partners: Groningen University (coordinating partner), Antwerp University, Tuebingen University, University College Dublin, Geneva University and Xerox Research Centre Europe.

Project Report:


John Nerbonne, Anja Belz, Nicola Cancedda, Herve Dejean, James Hammerton, Rob Koeling, Stasinos Konstantopoulos, Miles Osborne, Franck Thollard and Erik F. Tjong Kim Sang (2001). Learning Computational Grammars, In: Walter Daelemans and Remi Zajac (eds.) Proceedings of CoNLL-2001, pp. 97-104. [cs.CL/0107017]


Grammar Learning by Nonterminal Merging and Splitting:


1. Anja Belz (2001) Optimisation of corpus-derived probabilistic grammars, Proceedings of Corpus Linguistics 2001, pp. 46-57. [.ps.gz file ]
   
   
2. Anja Belz (2002). PCFG learning by nonterminal partition search. In P. Adriaans, H. Fernau and M. van Zaanen (eds.) Grammatical Inference: Algorithms and Applications. Proceedings of the 6th International Colloquium on Grammatical Inference (ICGI 2002), pp. 14-27. Berlin: Springer. [publication page] [.ps.gz (copyright Springer-Verlag)]
   
   
3. Anja Belz (2002). Learning grammars for different parsing tasks by partition search. In Proceedings of the 19th International conference in Computational Linguistics (COLING 2002), pp. 78-84. San Francisco: Morgan Kaufman. [.ps.gz]
   
   
4. Anja Belz (2002). Grammar learning by partition search. Proceedings of the LREC 2002 Workshop on Event Modelling for Multilingual Document Linking, pp. 9-16. [.ps]
   
   
5. Anja Belz (2002). Learning Grammars for Noun Phrase Extraction by Partition Search. Proceedings of the LREC 2002 Workshop on Linguistic Knowledge Acquisition and Representation: Bootstrapping Annotated Language Data, pp. 63-70. [.ps]