Tootfinder

Cin\'{e}aste: A Fine-grained Contextual Movie Question Answering Benchmark
Nisarg A. Shah, Amir Ziai, Chaitanya Ekanadham, Vishal M. Patel
https://arxiv.org/abs/2509.14227 h…

Cinéaste: A Fine-grained Contextual Movie Question Answering Benchmark
While recent advancements in vision-language models have improved video understanding, diagnosing their capacity for deep, narrative comprehension remains a challenge. Existing benchmarks often test short-clip recognition or use template-based questions, leaving a critical gap in evaluating fine-grained reasoning over long-form narrative content. To address these gaps, we introduce $\mathsf{Cin\acute{e}aste}$, a comprehensive benchmark for long-form movie understanding. Our dataset comprises 3,…

A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation
Ye Shen, Junying Wang, Farong Wen, Yijin Guo, Qi Jia, Zicheng Zhang, Guangtao Zhai
https://arxiv.org/abs/2509.14886

A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation
The rapid progress of Multi-Modal Large Language Models (MLLMs) has spurred the creation of numerous benchmarks. However, conventional full-coverage Question-Answering evaluations suffer from high redundancy and low efficiency. Inspired by human interview processes, we propose a multi-to-one interview paradigm for efficient MLLM evaluation. Our framework consists of (i) a two-stage interview strategy with pre-interview and formal interview phases, (ii) dynamic adjustment of interviewer weights …

ParaEQsA: Parallel and Asynchronous Embodied Questions Scheduling and Answering
Haisheng Wang, Weiming Zhi
https://arxiv.org/abs/2509.11663 https://arxiv.o…

ParaEQsA: Parallel and Asynchronous Embodied Questions Scheduling and Answering
This paper formulates the Embodied Questions Answering (EQsA) problem, introduces a corresponding benchmark, and proposes a system to tackle the problem. Classical Embodied Question Answering (EQA) is typically formulated as answering one single question by actively exploring a 3D environment. Real deployments, however, often demand handling multiple questions that may arrive asynchronously and carry different urgencies. We formalize this setting as Embodied Questions Answering (EQsA) and prese…

Rare Event Simulation of Quantum Error-Correcting Circuits
Carolyn Mayer, Anand Ganti, Uzoma Onunkwo, Tzvetan Metodi, Benjamin Anker, Jacek Skryzalin
https://arxiv.org/abs/2509.13678

Rare Event Simulation of Quantum Error-Correcting Circuits
We describe a practical approach for accessing the logical failure rates of quantum error-correcting (QEC) circuits under low physical (component) failure rate regimes. Standard Monte Carlo is often the de facto approach for studying the failure rates of quantum circuits. However, in the study of fault-tolerant error-correcting circuits, the ability to extend this approach to low physical failure rates is limited. In particular, the use of Monte Carlo to access circuits that are relatively larg…

HistoryBankQA: Multilingual Temporal Question Answering on Historical Events
Biswadip Mandal, Anant Khandelwal, Manish Gupta
https://arxiv.org/abs/2509.12720 https://

HistoryBankQA: Multilingual Temporal Question Answering on Historical Events
Temporal reasoning about historical events is a critical skill for NLP tasks like event extraction, historical entity linking, temporal question answering, timeline summarization, temporal event clustering and temporal natural language inference. Yet efforts on benchmarking temporal reasoning capabilities of large language models (LLMs) are rather limited. Existing temporal reasoning datasets are limited in scale, lack multilingual coverage and focus more on contemporary events. To address thes…

VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage
A. Alfarano (University of Zurich, Max Planck Society), L. Venturoli (University of Zurich, Max Planck Society), D. Negueruela del Castillo (University of Zurich, Max Planck Society)
https://arxiv.org/abs/2510.12750

VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage
Multimodal Large Language Models (MLLMs) have demonstrated significant capabilities in joint visual and linguistic tasks. However, existing Visual Question Answering (VQA) benchmarks often fail to evaluate deep semantic understanding, particularly in complex domains like visual art analysis. Confined to simple syntactic structures and surface-level attributes, these questions fail to capture the diversity and depth of human visual inquiry. This limitation incentivizes models to exploit statisti…

HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems
Spandan Anaokar, Shrey Ganatra, Harshvivek Kashid, Swapnil Bhattacharyya, Shruti Nair, Reshma Sekhar, Siddharth Manohar, Rahul Hemrajani, Pushpak Bhattacharyya
https://arxiv.org/abs/2509.11619

HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems
Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 69% outperforming baseline detectors by 25.44%. Benchmarking five chatbot architectures, we find that…

ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering
Francesco Maria Molfese, Luca Moroni, Ciro Porcaro, Simone Conia, Roberto Navigli
https://arxiv.org/abs/2510.09351

ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering
While Small Language Models (SLMs) have demonstrated promising performance on an increasingly wide array of commonsense reasoning benchmarks, current evaluation practices rely almost exclusively on the accuracy of their final answers, neglecting the validity of the reasoning processes that lead to those answers. To address this issue, we introduce ReTraceQA, a novel benchmark that introduces process-level evaluation for commonsense reasoning tasks. Our expert-annotated dataset reveals that in a…

Agentic LLMs for Question Answering over Tabular Data
Rishit Tyagi, Mohit Gupta, Rahul Bouri
https://arxiv.org/abs/2509.09234 https://arxiv.org/pdf/2509.09…

Agentic LLMs for Question Answering over Tabular Data
Question Answering over Tabular Data (Table QA) presents unique challenges due to the diverse structure, size, and data types of real-world tables. The SemEval 2025 Task 8 (DataBench) introduced a benchmark composed of large-scale, domain-diverse datasets to evaluate the ability of models to accurately answer structured queries. We propose a Natural Language to SQL (NL-to-SQL) approach leveraging large language models (LLMs) such as GPT-4o, GPT-4o-mini, and DeepSeek v2:16b to generate SQL queri…