Analyzing Gender Translation Errors to Identify Information Flows between the Encoder and Decoder of a NMT System

Guillaume Wisniewski (1) , Lichao Zhu (2) , Nicolas Ballier (2) , François Yvon (3)

1 LLF - UMR7110 - Laboratoire de Linguistique Formelle
2 CLILLAC-ARP (URP_3967) - Centre de Linguistique Inter-langues, de Lexicologie, de Linguistique Anglaise et de Corpus
3 TLP - Traitement du Langage Parlé - LISN

Multiple studies have shown that existing NMT systems demonstrate some kind of "gender bias". As a result, MT output appears to err more often for feminine forms and to amplify social gender misrepresentations, which is potentially harmful to users and practioners of these technologies. This paper continues this line of investigations and reports results obtained with a new test set in strictly controlled conditions. This setting allows us to better understand the multiple inner mechanisms that are causing these biases, which include the linguistic expressions of gender, the unbalanced distribution of masculine and feminine forms in the language, the modelling of morphological variation and the training process dynamics. To counterbalance these effects, we formulate several proposals and notably show that modifying the training loss can effectively mitigate such biases.

Domains: Computation and Language [cs.CL]; Artificial Intelligence [cs.AI]; Document and Text Processing

Guillaume Wisniewski, Lichao Zhu, Nicolas Ballier, François Yvon. Analyzing Gender Translation Errors to Identify Information Flows between the Encoder and Decoder of a NMT System. BlackboxNLP 2022, Dec 2022, Abu Dhabi, United Arab Emirates. ⟨hal-03912438⟩