@aredridel@kolektiva.social2026-04-14 14:22:42
So to follow up on this, I've caught it in action. Models, when quantized a bit, just do a bit more poorly with short contexts. Even going from f32 (as trained) to bf16 (as usually run) to q8 tends to do okay for "normal" context windows. And q4 you start feeling like "this model is a little stupid and gets stuck sometimes” (it is! It's just that it's still mostly careening about in the space of "plausible" most of the time. Not good guesswork, but still in the zone). With long contexts, the probability of parameters collapsing to zero are higher, so the more context the more likelihood you are to see brokenness.
And then at Q2 (2 bits per parameter) or Q1, the model falls apart completely. Parameters collapse to zero easily. You start seeing "all work and no play makes jack a dull boy” sorts of behavior, with intense and unscrutinized repetition, followed by a hard stop when it just stops working.
And quantization is a parameter that a model vendor can turn relatively easily. (they have to regenerate the model from the base with more quantization, but it's a data transformation on the order of running a terabyte through a straightforward and fast process, not like training).
If you have 1000 customers and enough equipment to handle the requests of 700, going from bf16 to q8 is a no-brainer. Suddenly you can handle the load and have a little spare capacity. They get worse results, probably pay the same per token (or they're on a subscription that hides the cost anyway so you are even freer to make trade-offs. There's a reason that subscription products are kinda poorly described.)
It's also possible for them to vary this across a day: use models during quieter periods? Maybe you get an instance running a bf16 quantization. If you use it during a high use period? You get a Q4 model.
Or intelligent routing is possible. No idea if anyone is doing this, but if they monitor what you send a bit, and you generally shoot for an expensive model for simple requests? They could totally substitute a highly quantized version of the model to answer the question.
There are •so many tricks• that can be pulled here. Some of them very reasonable to make, some of them treading into outright misleading or fraudulent, and it's weirdly hard to draw the line between them.
@tinoeberl@mastodon.online2026-04-09 20:22:30
Das EnergiePortal des Kreises Schleswig-Flensburg informiert über aktuelle Entwicklungen zur #Energiewende.
Ein integriertes #Solardachkataster zeigt, ob sich #Photovoltaik auf…
@mia@hcommons.social2026-04-02 15:48:01
Good news for anyone working on their proposals for the next Fantastic Futures conference - the deadline is extended to April 16! #FF2026
@aufsmaulsuppe@chaos.social2026-02-25 19:36:43
Offener Brief der Deutschen Filmakademie zu dem scheinbar geplanten kulturpolitischen Eingriff in die Leitung der Berlinale
https://openletter.earth/die-deutsche-filmakademie-positioni…
@aral@mastodon.ar.al2026-02-03 09:45:06
Wow, man, imagine being the kind of person that suspends anyone he sees from Gaza during a genocide.
The inhumanity of some Germans is really quite remarkable. Didn’t you get enough of genocide the first time around?
Add social.tchncs.de to the list of Zionist, pro-genocide Mastodon servers.
#germany #israel
@cdarwin@c.im2026-04-08 01:17:57
The nonstop parade of luxury that is
La Première, Air France’s first-class trans-Atlantic service,
begins when a Mercedes limousine collects you from your hotel and whisks you to an exclusive entrance at Charles de Gaulle International Airport.
It ends at J.F.K., when an Air France employee personally escorts you from your seat through a special customs line.
Each new indulgence seems more lavish than the one before.
The bespoke departure lounge, where you can or…
@Tupp_ed@mastodon.ie2026-02-06 09:37:03
This week’s Gist is about how the U.K. Labour Party unknowingly has been blowing itself up by following Morgan McSweeney’s FG electoral instincts.
They don’t know the patterns, but we do.
https://www.thegist.ie/the-gist-uk-labours-fg-mastermind/
@johnleonard@mastodon.social2026-03-02 13:41:48
Rebuilding public trust in AI requires meaningful citizen engagement, transparent governance, and robust legislation. Technology itself is not the problem. The issue is that few people trust institutions to deploy it wisely and for their benefit. This makes the first step to answer the following question: What’s it in for me?
@ginevra@hachyderm.io2026-02-08 00:00:32
I've a small account here, same as for all SM platforms I've been on: infrequent posting, no big following. I know bigger accounts get lots of fake followers/bots, but it's rare for me. Maybe it averages to 1 per month on BSky and Insta (when I used it). Anyway, achievement unlocked:
🎉 1st follower that appears to be a bot or a spammer on mastodon (afaik)
Soz if it's a real person, but boasting of your income & faith & few/no posts earns a block from me
@azonenberg@ioc.exchange2026-03-30 12:31:22
Sadly couldn't join myself (wife wasn't feeling good and wanted me to stay home and help parent while she napped) but my SAR team participated in a great inter-agency training this past weekend.
It's always nice to practice alongside volunteers from other teams that we work with on real incidents and get to know each other without the pressure of a real emergency.
One note: I half feel like there should be a CW on this post for mentioning law enforcement in a positiv…
