Tootfinder

Visual Hallucinations of Multi-modal Large Language Models
Wen Huang, Hongbin Liu, Minxin Guo, Neil Zhenqiang Gong
https://arxiv.org/abs/2402.14683 https:/…

Visual Hallucinations of Multi-modal Large Language Models
Visual hallucination (VH) means that a multi-modal LLM (MLLM) imagines incorrect details about an image in visual question answering. Existing studies find VH instances only in existing image datasets, which results in biased understanding of MLLMs' performance under VH due to limited diversity of such VH instances. In this work, we propose a tool called VHTest to generate a diverse set of VH instances. Specifically, VHTest finds some initial VH instances in existing image datasets (e.g., COCO)…