中文不看而见:视觉-语言基准真的在测试视觉吗?
ENSeeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
arXiv:2605.22903v1 指出,视觉语言模型(VLM)在幻觉基准上的高准确率常被误认为依赖视觉证据,但移除大量图像token后性能仅轻微下降。研究通过多粒度视觉退化分析,系统揭示了开源VLM对非视觉线索的过度依赖,警示基准分数并不等同于真正的视觉理解。
arXiv:2605.22903v1 Announce Type: new Abstract: Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image tokens only degrades model performance very slightly on a widely used hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs. Our analysis spans multiple levels of granularity, spanning global visual degradatio