Tootfinder

Forschung: Verschmelzen Menschen und KI zu einem "evolutionären Individuum"?
Mit der Entwicklung von KI-Technik könnte die Menschheit einen großen evolutionären Übergang eingeleitet haben. Das meinen zumindest zwei Evolutionsbiologen.

A methodology for clinically driven interactive segmentation evaluation
Parhom Esmaeili, Virginia Fernandez, Pedro Borges, Eli Gibson, Sebastien Ourselin, M. Jorge Cardoso
https://arxiv.org/abs/2510.09499

A methodology for clinically driven interactive segmentation evaluation
Interactive segmentation is a promising strategy for building robust, generalisable algorithms for volumetric medical image segmentation. However, inconsistent and clinically unrealistic evaluation hinders fair comparison and misrepresents real-world performance. We propose a clinically grounded methodology for defining evaluation tasks and metrics, and built a software framework for constructing standardised evaluation pipelines. We evaluate state-of-the-art algorithms across heterogeneous and…

MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
Jiapeng Wang, Changxin Tian, Kunlong Chen, Ziqi Liu, Jiaxin Mao, Wayne Xin Zhao, Zhiqiang Zhang, Jun Zhou
https://arxiv.org/abs/2510.09295

MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
Reliable evaluation is fundamental to the progress of Large Language Models (LLMs), yet the evaluation process during pre-training is plagued by significant instability that obscures true learning dynamics. In this work, we systematically diagnose this instability, attributing it to two distinct sources: \textit{Parameter Instability} from training stochasticity and \textit{Evaluation Instability} from noisy measurement protocols. To counteract both sources of noise, we introduce \textbf{MaP}, …

What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment
Allison Sihan Jia, Daniel Huang, Nikhil Vytla, Nirvika Choudhury, John C Mitchell, Anupam Datta
https://arxiv.org/abs/2510.08847

What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment
We introduce the Agent GPA (Goal-Plan-Action) framework: an evaluation paradigm based on an agent's operational loop of setting goals, devising plans, and executing actions. The framework includes five evaluation metrics: Goal Fulfillment, Logical Consistency, Execution Efficiency, Plan Quality, and Plan Adherence. Logical Consistency checks that an agent's actions are consistent with its prior actions. Execution Efficiency checks whether the agent executes in the most efficient way to achieve …

The Sonora Substellar Atmosphere Models VI. Red Diamondback: Extending Diamondback with SPHINX for Brown Dwarf Early Evolution
C. Evan Davis, Jonathan J. Fortney, Aishwarya Iyer, Sagnick Mukherjee, Caroline V. Morley, Mark S. Marley, Michael Line, Philip S. Muirhead
https://arxiv.org/abs/2510.08694…

The Sonora Substellar Atmosphere Models VI. Red Diamondback: Extending Diamondback with SPHINX for Brown Dwarf Early Evolution
We extend the Sonora Diamondback brown dwarf evolution models to higher effective temperatures to treat the evolution of younger, higher mass objects. Due to an upper temperature limit of $T_\mathrm{eff}=$2400 K in the original Sonora Diamondback model grid, high mass objects ($M\geq$ 0.05 $M_\mathrm{\odot}=$ 52.4 $M_\mathrm{J}$) were limited to ages of $\gtrsim$ 100 Myr. To include the early evolution of brown dwarfs at $T_\mathrm{eff}>$ 2400 K, we use existing and new SPHINX cloud-free model …

Beyond the Binary: The System of All-round Evaluation of Research and Its Practices in China
Yu Zhu, Jiyuan Ye
https://arxiv.org/abs/2509.08546 https://arx…

Beyond the Binary: The System of All-round Evaluation of Research and Its Practices in China
The lack of a macro-level, systematic evaluation theory to guide the implementation of evaluation practices has become a key bottleneck in the reform of global research evaluation systems. By reviewing the historical development of research evaluation, this paper highlights the current binary opposition between qualitative and quantitative methods in evaluation practices. This paper introduces the System of All-round Evaluation of Research (SAER), a framework that integrates form, content, and …

TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation
Yincen Qu, Huan Xiao, Feng Li, Hui Zhou, Xiangying Dai
https://arxiv.org/abs/2510.09011

TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation
Travel planning is a valuable yet complex task that poses significant challenges even for advanced large language models (LLMs). While recent benchmarks have advanced in evaluating LLMs' planning capabilities, they often fall short in evaluating feasibility, reliability, and engagement of travel plans. We introduce a comprehensive benchmark for travel planning that unifies fine-grained criteria into a single reward, enabling direct comparison of plan quality and seamless integration with reinfo…

ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering
Francesco Maria Molfese, Luca Moroni, Ciro Porcaro, Simone Conia, Roberto Navigli
https://arxiv.org/abs/2510.09351

ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering
While Small Language Models (SLMs) have demonstrated promising performance on an increasingly wide array of commonsense reasoning benchmarks, current evaluation practices rely almost exclusively on the accuracy of their final answers, neglecting the validity of the reasoning processes that lead to those answers. To address this issue, we introduce ReTraceQA, a novel benchmark that introduces process-level evaluation for commonsense reasoning tasks. Our expert-annotated dataset reveals that in a…

Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation
Xiangxu Zhang, Lei Li, Yanyun Zhou, Xiao Zhou, Yingying Zhang, Xian Wu
https://arxiv.org/abs/2510.09275

Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation
Medical diagnostics is a high-stakes and complex domain that is critical to patient care. However, current evaluations of large language models (LLMs) are fundamentally misaligned with real-world clinical practice. Most of them rely on static benchmarks derived from public medical exam items, which tend to overestimate model performance and ignore the difference between textbook cases and the ambiguous, varying conditions in the real world. Recent efforts toward dynamic evaluation offer a promi…

Reveal-Bangla: A Dataset for Cross-Lingual Multi-Step Reasoning Evaluation
Khondoker Ittehadul Islam, Gabriele Sarti
https://arxiv.org/abs/2508.08933 https://

Reveal-Bangla: A Dataset for Cross-Lingual Multi-Step Reasoning Evaluation
Language models have demonstrated remarkable performance on complex multi-step reasoning tasks. However, their evaluation has been predominantly confined to high-resource languages such as English. In this paper, we introduce a manually translated Bangla multi-step reasoning dataset derived from the English Reveal dataset, featuring both binary and non-binary question types. We conduct a controlled evaluation of English-centric and Bangla-centric multilingual small language models on the origin…