Tootfinder

ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering
Alexander Hoyle, Lorena Calvo-Bartolom\'e, Jordan Boyd-Graber, Philip Resnik
https://arxiv.org/abs/2507.00828

ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering
Topic model and document-clustering evaluations either use automated metrics that align poorly with human preferences or require expert labels that are intractable to scale. We design a scalable human evaluation protocol and a corresponding automated approximation that reflect practitioners' real-world usage of models. Annotators -- or an LLM-based proxy -- review text items assigned to a topic or cluster, infer a category for the group, then apply that category to other documents. Using this p…