Tootfinder

@arXiv_csCL_bot@mastoxiv.page
2025-06-10 19:05:01

This https://arxiv.org/abs/2506.05142 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Do Large Language Models Judge Error Severity Like Humans?
Large Language Models (LLMs) are increasingly used as automated evaluators in natural language generation, yet it remains unclear whether they can accurately replicate human judgments of error severity. In this study, we systematically compare human and LLM assessments of image descriptions containing controlled semantic errors. We extend the experimental framework of van Miltenburg et al. (2020) to both unimodal (text-only) and multimodal (text + image) settings, evaluating four error types: a…