Reducing named entity hallucination risk to ensure faithful summary generation - Aix-Marseille Université Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Reducing named entity hallucination risk to ensure faithful summary generation

Résumé

The faithfulness of abstractive text summarization at the named entities level is the focus of this study. We propose to add a new criterion to the summary selection method based on the "risk" of generating entities that do not belong to the source document. This method is based on the assumption that Out-Of-Document entities are more likely to be hallucinations. This assumption was verified by a manual annotation of the entities occurring in a set of generated summaries on the CNN/DM corpus. This study showed that only 29% of the entities outside the source document were inferrable by the annotators, leading to 71% of hallucinations among OOD entities. We test our selection method on the CNN/DM corpus and show that it significantly reduces the hallucination risk on named entities while maintaining competitive results with respect to automatic evaluation metrics like ROUGE.
Fichier principal
Vignette du fichier
Eunice___Sigdial_INLG_2023.pdf (219.09 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04181194 , version 1 (15-08-2023)

Identifiants

  • HAL Id : hal-04181194 , version 1

Citer

Eunice Akani, Benoit Favre, Frederic Bechet, Romain Gemignani. Reducing named entity hallucination risk to ensure faithful summary generation. 16th International Natural Language Generation Conference, Sep 2023, Prague, Czech Republic. ⟨hal-04181194⟩
115 Consultations
17 Téléchargements

Partager

Gmail Facebook X LinkedIn More