DutchSemCor: Aiming at the ideal sense-tagged corpus

TitleDutchSemCor: Aiming at the ideal sense-tagged corpus
Publication TypeJournal Article
Year of Publication2013
AuthorsVossen, P, Izquierdo, R, Görög, A
JournalComputational Linguistics in the Netherlands Journal
Date Published12/2013

The most-frequent-sense and the predominant domain sense play an important role in the debate on Word Sense Disambiguation (WSD). This discussion is, however, biased by the way sense-tagged corpora are built. In this paper, we argue that current sense-tagged corpora neglect rare senses and contexts and, as a result, do not represent a good corpus for training and testing word-sense-disambiguation. We defined three quality criteria for sense-tagged corpora and a methodology to satisfy these criteria with minimal effort. Following this method, we built a Dutch sense-tagged corpus that tried to meet these criteria. The corpus was evaluated by deriving word-sense-disambiguation systems and testing these on different subsets of the corpus in various ways. The performance of our systems and the quality of the derived data are equal to state-of-the-art English systems and corpora. Finally, we used the systems to annotate a chunk of the Dutch SoNaR-corpus and create a subcorpus of over 47 million sense-tagged tokens spread over a large variety of genres, domains and usages of Dutch. The results of the project can be downloaded freely from the project website.

Refereed DesignationRefereed
PDF icon Vossen2013.pdf748.68 KB