PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI AssistantsZheng Zhao, Clara Vania, Subhradeep Kayal, Naila Khan, Shay B. Cohen, Emine Yilmazhttps://arxiv.org/abs/2506.09902
PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI AssistantsLarge language models (LLMs) have advanced conversational AI assistants. However, systematically evaluating how well these assistants apply personalization--adapting to individual user preferences while completing tasks--remains challenging. Existing personalization benchmarks focus on chit-chat, non-conversational tasks, or narrow domains, failing to capture the complexities of personalized task-oriented assistance. To address this, we introduce PersonaLens, a comprehensive benchmark for evalu…