Tootfinder

Opt-in global Mastodon full text search. Join the index!

@hacksilon@infosec.exchange
2026-06-10 17:22:21

Reading through the reasoning of Fable, I am quite impressed as well - it shows a fairly deep understanding of how Keycloak works, and evaluated different methods, considered the tradeoffs, and changed track after encountering issues with its proposals several times. (This was on the "Extra" effort level).
If you haven't tried to get a good answer about a non-trivial Keycloak issue from Claude in the past, you don't know just how much of an improvement this is - Sonnet has consistently been a trainwreck when it comes to these things, and Opus has been hit-or-miss.

@hacksilon@infosec.exchange
2026-06-10 17:11:45

Another anecdotal datapoint on Fable: it's the first model that passes my (fairly informal) benchmark of "will not hallucinate features when asked a non-trivial question about Keycloak." - it actually one-shot a working solution for a problem I'd been having, and I just verified that it seems to actually work, so I even learned something new. I'm going to validate the answer some more with a colleague tomorrow - if this actually works, we may write it up as a blog post, as this will be a helpful reference to keep in our pocket for future Keycloak security audit reports.
For comparison:
- Opus partially solved the problem in a less elegant and incomplete way, but at least it used actually-existing features. It also did not find the complete solution that closes the last holes that Fable proposed.
- Sonnet hallucinated a feature, then proposed an incomplete solution when I pointed this out, then proposed an answer that sounded plausible but was wrong in a subtle way, which could have bitten me badly, had I implemented it.