Tootfinder

I senaste avsnittet av Hälsoveckan diskuterar jag och Erik de nya kostråden i USA. Det är ett fantastiskt motsägande dokument där du hittar gott om saker som går emot varandra tillsammans med en premiss som gör att i princip all forskning som finns på nutrition blir irrelevant i förhållande till vad de anser vara naturligt.
USAs kostråd har något som alla kan tolka som positivt | Podcast

So to follow up on this, I've caught it in action. Models, when quantized a bit, just do a bit more poorly with short contexts. Even going from f32 (as trained) to bf16 (as usually run) to q8 tends to do okay for "normal" context windows. And q4 you start feeling like "this model is a little stupid and gets stuck sometimes” (it is! It's just that it's still mostly careening about in the space of "plausible" most of the time. Not good guesswork, but still in the zone). With long contexts, the probability of parameters collapsing to zero are higher, so the more context the more likelihood you are to see brokenness.
And then at Q2 (2 bits per parameter) or Q1, the model falls apart completely. Parameters collapse to zero easily. You start seeing "all work and no play makes jack a dull boy” sorts of behavior, with intense and unscrutinized repetition, followed by a hard stop when it just stops working.
And quantization is a parameter that a model vendor can turn relatively easily. (they have to regenerate the model from the base with more quantization, but it's a data transformation on the order of running a terabyte through a straightforward and fast process, not like training).
If you have 1000 customers and enough equipment to handle the requests of 700, going from bf16 to q8 is a no-brainer. Suddenly you can handle the load and have a little spare capacity. They get worse results, probably pay the same per token (or they're on a subscription that hides the cost anyway so you are even freer to make trade-offs. There's a reason that subscription products are kinda poorly described.)
It's also possible for them to vary this across a day: use models during quieter periods? Maybe you get an instance running a bf16 quantization. If you use it during a high use period? You get a Q4 model.
Or intelligent routing is possible. No idea if anyone is doing this, but if they monitor what you send a bit, and you generally shoot for an expensive model for simple requests? They could totally substitute a highly quantized version of the model to answer the question.
There are •so many tricks• that can be pulled here. Some of them very reasonable to make, some of them treading into outright misleading or fraudulent, and it's weirdly hard to draw the line between them.

Apples KI-Strategie: Auch Datenschutz sorgt für Verzögerungen
Auch nach dem Beginn der Kooperation mit Google für Gemini-basierte Sprachmodelle ändert sich an Apples Strategie wenig. Die Ansage kommt von ganz oben.

>500.000 Gegenstände sollen am Tatort auf dem Boden liegen.
#SPIEGEL Plus vom 15.01.202
Einbruch in Gelsenkirchen
Wie die Täter 3000 Schließfächer knackten – und niemand etwas merkte
"Gut möglich, dass das Diebesgut einen Wert von mehr als hundert Millionen Euro hat."
"Sie müssen sich fragen lassen, wie es sein kann, dass sie in einer der ärmsten Stä…

Aus für Websperren: Gericht kippt Sperrverfügung gegen Pornhub & Co.
Das Verwaltungsgericht Neustadt gibt den Klagen von 1&1 und Aylo statt: Der europäische Digital Services Act hebelt deutsche Alleingänge beim Jugendschutz aus.

C09 - Sarcophagus
TARRANT: [Approaches Avon and hands him the calculator.] Here, take a look at this. That asteroid's less than thirty- two hours away, but we'll still be cutting it fine.
AVON: Let's get going then. There's nothing keeping us here.
https://blake.torpidity.net/m/309/22

iTunes-Wunschliste: Apple löscht das Feature jetzt ganz
iTunes gibt es eigentlich gar nicht mehr, doch ein Feature hatte Apple in eine andere App migriert. Die Wunschliste für Filme und Serien verschwindet nun ganz.

Series C, Episode 12 - Death-Watch
VILA: [Pulls Orac's key] They don't write poetry like that anymore. What this electronic pain is trying to say is-
TARRANT: -is that unless we get a break, there's going to be a fatal foul-up.
VILA: Right.
https://blake.torpidity.net/m/312/52

„Everspace“: Rockfish Games bekommt 8 Millionen Euro Games-Förderung
Das Studio hinter „Everspace 2“ bekommt 8 Millionen Euro von der Bundesregierung, um einen Nachfolger zu entwickeln – eine Rekordsumme.

Apple nimmt zu neuerlicher Siri-Verzögerung Stellung
Die Börse nahm es gar nicht gut auf: 5 Prozent ging es für Apple nach Berichten zu Siri-Problemen am Donnerstag herunter. Der Konzern äußert sich.
https://www.