Bigger Isn't Just Bigger

More practice usually helps

Someone who has read a thousand books generally has sharper instincts for language and ideas than someone who has read ten, all else being equal. That's a fair starting intuition for why bigger language models, ones with more parameters, trained on more text, using more computing power, tend to guess better than smaller ones.

But "all else being equal" is doing a lot of work in that sentence, and it's worth taking apart what's actually growing when people say a model got "bigger."

Three things that can each get bigger, separately

Turning up just one of these without the others tends to underdeliver. A model with an enormous number of dials but not nearly enough training text to meaningfully tune all of them ends up with a lot of dials sitting at close to their starting position, contributing little. The gains tend to come from scaling these together, roughly in balance.

Diminishing returns are real

Doubling a model's size doesn't double its usefulness. Early gains from scaling up tend to be dramatic: a model goes from barely coherent to genuinely useful. Later gains tend to be more incremental: a much larger, much more expensive model might handle a modestly harder set of tasks a bit more reliably, without the qualitative leap that earlier scaling produced.

This matters because it means "make it bigger" isn't a free path to fixing every weakness. A vastly larger model built the same way, on the same kind of data, will still fundamentally be a guessing machine, still without a built-in fact-checker, still limited by whatever context window it's given. Scale sharpens the guessing. It doesn't change what kind of thing is doing the guessing.