People often assume that LLM output is some kind of neutral "average" of "all knowledge" (or at least, most of the internet), and that perhaps it picks up and amplifies biases already present in the training data, but these are merely innate human biases to begin with. Already, that shouldn't be comforting, given how biased LLM resume-scanning systems are against female-sounding names, for example, but let's put that aside. We can actually see that LLMs also have intentionally engineered biases added to them in custom training steps:
You know all the LLM syncophancy? The way it agrees with whatever the user says, even if that contradicts itself, or something else the user said earlier? Where did that come from? How often in the internet pre-LLMs did you see someone disagree, only for the other person to immediately apologize and back down? Is that the average behavior on Reddit? On other internet forums? Even in private chat messages or on IRC?
In fact, the internet is legendary for the exact opposite behavior? Where did the collective reply-guy tendency of the training data go? It must have been painstakingly trained out of the model by OpenAI. There's a few viable approaches to this, but likely, it involved optimizing for continued conversation and/or some kind of "pleasantness" score, and that kind of optimization always has side effects. More to the point, if we know the big models are doing this kind of intentional tuning, there probably tuning for other stuff too, and in general, the output of the models is much closer to "things OpenAI wants it to say" than "things the internet would naturally have said" than we might suspect.
#AI #LLMs
@aredridel@kolektiva.social
