Tootfinder

Interesting explanation of LLM training frameworks and the incentives for confident guessing.
"The authors examined ten major AI benchmarks, including those used by Google, OpenAI and also the top leaderboards that rank AI models. This revealed that nine benchmarks use binary grading systems that award zero points for AIs expressing uncertainty.
" ... When an AI system says “I don’t know”, it receives the same score as giving completely wrong information. The optimal strategy under such evaluation becomes clear: always guess. ...
"More sophisticated approaches like active learning, where AI systems ask clarifying questions to reduce uncertainty, can improve accuracy but further multiply computational requirements. ...
"Users want systems that provide confident answers to any question. Evaluation benchmarks reward systems that guess rather than express uncertainty. Computational costs favour fast, overconfident responses over slow, uncertain ones."
=
My comment: "Fast, overconfident responses" sounds a bit similar to "bullshit", does it not?
#ChatGPT #LLMs #SoCalledAI

Hadn't seen this variation before.
Recently, a colleague has had that thing where, un-asked-for by them, their "AI notetaker" tries to go to all the meetings in their calendar. This time it was double trouble: they somehow had _two_ bots trying to come into every meeting citing their name. Consequently, a few of us have had to boot out these bots at the start of a meeting, when the person themself wasn't there.
Yesterday, the _real_ person tried to go to a meeting. The host (who was a bit flustered due to connection problems) mistook their name for the bot appearing yet again, and kicked them out! so they missed the meeting!
(I found out only afterwards that they'd been trying to get on, and what had happened.)
=
Luckily it wasn't a meeting which hinged on this one person's presence, so not super high stakes. I was just thinking about it again now, and the phenomenon of "unwanted bot-behaviour causes knock-on problem". It reminded me of this more-troublesome episode, which I'd also read about yesterday:
#software #bots #SoCalledAI

"I dislike the attempt to gaslight people into swearing fealty to a sickly and frail psuedo-industry where everybody but NVIDIA and consultancies lose money."
#SoCalledAI #business #bubble #AIBubble #LLMs #NVidia