Tootfinder

Kryptowährung: Neun führende europäische Banken entwickeln Euro-Stablecoin
Neun europäische Banken, darunter DekaBank, entwickeln Euro-Stablecoin als Antwort auf US-Dominanz im digitalen Zahlungsverkehr.

Kyler Murray has the 'ultimate confidence' in Trey Benson taking over for James Conner https://www.nfl.com/news/kyler-murray-has-the-ultimate-confidence-in-trey-benson-taking-over-for-james-conner

When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity
Benjamin Feuer, Chiung-Yi Tseng, Astitwa Sarthak Lathe, Oussama Elachqar, John P Dickerson
https://arxiv.org/abs/2509.20293

When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity
LLM-judged benchmarks are increasingly used to evaluate complex model behaviors, yet their design introduces failure modes absent in conventional ground-truth based benchmarks. We argue that without tight objectives and verifiable constructions, benchmark rankings can produce high-confidence rankings that are in fact largely noise. We introduce two mechanisms to diagnose these issues. Schematic adherence quantifies how much of a judge's overall verdict is explained by the explicit evaluation sc…

🇺🇦 #NowPlaying on KEXP's #Early
Benjamin Gibbard:
🎵 Ichiro’s Theme
#BenjaminGibbard
https://benjamingibbard.bandcamp.com/track/ichiros-theme
https://open.spotify.com/track/34O8x4AdwyKJFQdUKue5j5

TIL: ratarmount at https://binblog.de/2025/08/14/benchmarking-ratarmount/
Thanks @…

#Armut anders bemessen
Arm ist nicht nur, wer wenig Geld hat. Um Armut besser bemessen zu können, haben Wissenschaftler*innen nun ein Bewertungssystem entwickelt, das viele Dimensionen der Armut mit einschließt.
#Energie

The Illusion of Readiness: Stress Testing Large Frontier Models on Multimodal Medical Benchmarks
Yu Gu, Jingjing Fu, Xiaodong Liu, Jeya Maria Jose Valanarasu, Noel Codella, Reuben Tan, Qianchu Liu, Ying Jin, Sheng Zhang, Jinyu Wang, Rui Wang, Lei Song, Guanghui Qin, Naoto Usuyama, Cliff Wong, Cheng Hao, Hohin Lee, Praneeth Sanapathi, Sarah Hilado, Bian Jiang, Javier Alvarez-Valle, Mu Wei, Jianfeng Gao, Eric Horvitz, Matt Lungren, Hoifung Poon, Paul Vozila

The Illusion of Readiness: Stress Testing Large Frontier Models on Multimodal Medical Benchmarks
Large frontier models like GPT-5 now achieve top scores on medical benchmarks. But our stress tests tell a different story. Leading systems often guess correctly even when key inputs like images are removed, flip answers under trivial prompt changes, and fabricate convincing yet flawed reasoning. These aren't glitches; they expose how today's benchmarks reward test-taking tricks over medical understanding. We evaluate six flagship models across six widely used benchmarks and find that high lead…

EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models
Botai Yuan, Yutian Zhou, Yingjie Wang, Fushuo Huo, Yongcheng Jing, Li Shen, Ying Wei, Zhiqi Shen, Ziwei Liu, Tianwei Zhang, Jie Yang, Dacheng Tao
https://arxiv.org/abs/2509.20146

EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models
Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety. We study sycophancy -- models' tendency to uncritically echo user-provided information -- in high-stakes clinical settings. We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs. It contains 2,122 images across 18 departments and 20 modalities with 90 prompts that simulate biased inputs from patients, medical students, and ph…

Verbraucher stellen Ansprüche an digitalen Euro
Wer bargeldlos bezahlt, will auch bei einem digitalen Euro keine Kompromisse machen. Warum Banken und Sparkassen das EZB-Projekt trotzdem mit Skepsis begleiten.
https://www.

What Does Your Benchmark Really Measure? A Framework for Robust Inference of AI Capabilities
Nathanael Jo, Ashia Wilson
https://arxiv.org/abs/2509.19590 https://

What Does Your Benchmark Really Measure? A Framework for Robust Inference of AI Capabilities
Evaluations of generative models on benchmark data are now ubiquitous, and their outcomes critically shape public and scientific expectations of AI's capabilities. Yet growing skepticism surrounds their reliability. How can we know that a reported accuracy genuinely reflects a model's true performance? Evaluations are often presented as simple measurements, but in reality they are inferences: to treat benchmark scores as evidence of capability is already to assume a theory of what capability is…