How It Learned to Guess So Well
How training actually works: showing the model a guess, telling it the right answer, and nudging it a tiny bit closer, billions of times over.
Learning to catch a ball
Nobody catches a ball perfectly the first time. You reach, you miss, the ball hits your palm at the wrong angle, and something in your brain quietly adjusts for next time. Do that a thousand times and your hands start getting there before you've consciously decided to move them. Nobody handed you a formula. You just got a huge number of tiny corrections, one throw at a time.
That's a close analogy for how a language model learns to guess text well. It isn't given rules about grammar or facts. It's shown a piece of real text with the next word hidden, asked to guess that word, and then shown the real answer. If the guess was close, nothing much changes. If it was way off, something inside the model shifts slightly so that next time, a similar situation gets a slightly better guess.
The dials being adjusted
Inside the model are an enormous number of adjustable numbers, called parameters (people often just call them "the model's weights"). Think of them as an almost unimaginably large panel of dials. None of them means anything on its own; a single dial isn't "the dial for knowing about dogs." But together, millions or billions of them, tuned just right, encode the patterns needed to guess text well.
Training is the process of tuning those dials. Show it a sentence, hide the last word, let it guess, compare the guess to the real word, and nudge every dial that contributed to the guess a tiny bit in the direction that would have made the guess better. Repeat that with a staggering amount of text, essentially a meaningful fraction of everything ever written that could be gathered, and the dials slowly settle into a configuration that guesses remarkably well.
Two very different phases
This is worth keeping straight, because it explains something people often find confusing: the model doesn't "learn" from your conversation with it the way it learned during training. Talking to it doesn't adjust its dials. It's using the instincts it already has, the same way your hands already know how to catch a ball without you re-practicing mid-catch.
A person can learn a language from a few years of hearing it. A model starts from nothing, no built-in sense of grammar or meaning at all, and has to discover every pattern purely from examples. Making up for that head start takes a genuinely enormous amount of text and an equally enormous amount of computation to actually process it.