Gender Bias and the Role of Context in Human Perception and Machine Translation

Abstract

This paper investigates human gender bias and its relation to bias in machine translation (MT), focussing on the role of context in gender interpretation. To this end, we measured human implicit gender bias and conducted an annotation study, followed by a linguistic and computational analysis to compare human gender perceptions among themselves and with a machine translation system. We created a dataset of 60 gender-ambiguous sentences and collected annotations to understand human gender perceptions and specifically which trigger words in context lead to this perception. The study shows that, unlike the MT system tested in this study, humans exhibit highly varied perceptions of gender in ambiguous contexts. A linguistic analysis on annotated trigger words reveals that proper nouns, nouns and adjectives frequently affect human gender perception.