Tootfinder

Generative Information Retrieval Evaluation
Marwah Alaofi, Negar Arabzadeh, Charles L. A. Clarke, Mark Sanderson
https://arxiv.org/abs/2404.08137 https://<…

Generative Information Retrieval Evaluation
In this chapter, we consider generative information retrieval evaluation from two distinct but interrelated perspectives. First, large language models (LLMs) themselves are rapidly becoming tools for evaluation, with current research indicating that LLMs may be superior to crowdsource workers and other paid assessors on basic relevance judgement tasks. We review past and ongoing related research, including speculation on the future of shared task initiatives, such as TREC, and a discussion on t…

I give up. Crowd sourcing time. What's the name of that lookup site that lists traffic violations by plate number?
#FindMyPlate
#CrowdSource

Crowdsourcing Task Traces for Service Robotics
David Porfirio, Allison Saupp\'e, Maya Cakmak, Aws Albarghouthi, Bilge Mutlu
https://arxiv.org/abs/2403.14014

Crowdsourcing Task Traces for Service Robotics
Demonstration is an effective end-user development paradigm for teaching robots how to perform new tasks. In this paper, we posit that demonstration is useful not only as a teaching tool, but also as a way to understand and assist end-user developers in thinking about a task at hand. As a first step toward gaining this understanding, we constructed a lightweight web interface to crowdsource step-by-step instructions of common household tasks, leveraging the imaginations and past experiences of …

This https://arxiv.org/abs/2404.08137 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Generative Information Retrieval Evaluation
In this chapter, we consider generative information retrieval evaluation from two distinct but interrelated perspectives. First, large language models (LLMs) themselves are rapidly becoming tools for evaluation, with current research indicating that LLMs may be superior to crowdsource workers and other paid assessors on basic relevance judgement tasks. We review past and ongoing related research, including speculation on the future of shared task initiatives, such as TREC, and a discussion on t…