Towards an Exhaustive Evaluation of Vision-Language Foundation Models - Aix-Marseille Université Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Towards an Exhaustive Evaluation of Vision-Language Foundation Models

Résumé

Vision-language foundation models have had considerable increase in performances in the last few years. However, there is still a lack of comprehensive evaluation methods able to clearly explain their performances. We argue that a more systematic approach to foundation model evaluation would be beneficial to their use in real-world applications. In particular, we think that those models should be evaluated on a broad range of specific capabilities, in order to bring awareness to the width of their scope and their potential weaknesses. To that end, we propose a methodology to build a taxonomy of multimodal capabilities for visionlanguage foundation models. The proposed taxonomy is intended as a first step towards an exhaustive evaluation of vision-language foundation models.
Fichier principal
Vignette du fichier
Towards_an_Exhaustive_Evaluation_of_Vision_Language_Foundation_Models.pdf (299.09 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-04181190 , version 1 (15-08-2023)

Identifiants

  • HAL Id : hal-04181190 , version 1

Citer

Emmanuelle Salin, Stéphane Ayache, Benoit Favre. Towards an Exhaustive Evaluation of Vision-Language Foundation Models. ICCV workshop on Multimodal Foundation Models (MMFM), Oct 2023, Paris, France. ⟨hal-04181190⟩
34 Consultations
28 Téléchargements

Partager

Gmail Facebook X LinkedIn More