How AI Actually Thinks
A first-principles guide to what's really happening inside a large language model, built from simple analogies up to the real mechanics.
Start with What Is a Word-Guessing Machine? →Basics3 questions
What Is a Word-Guessing Machine?
The single core idea behind every large language model: it isn't thinking in the way a person does, it's guessing the next word, extremely well.
How It Learned to Guess So Well
How training actually works: showing the model a guess, telling it the right answer, and nudging it a tiny bit closer, billions of times over.
Turning Words Into Numbers
Computers only do math, so before any guessing can happen, words get converted into lists of numbers that capture what they mean.
Intermediate3 questions
Paying Attention to What Matters
Not every earlier word matters equally for the next guess. Attention is how a model decides which ones to weigh most heavily.
Its Short-Term Memory
Why a model can lose track of something said earlier in a long conversation: there's a hard limit on how much text it can actually see at once.
Why the Same Question Gets Different Answers
The model doesn't always pick its single most likely next word. A setting called temperature controls how much it's willing to take a chance.
Advanced3 questions
Why It Sometimes Makes Things Up
A guessing machine doesn't have a built-in fact-checker. When it doesn't actually know something, it still has to guess, confidently and fluently.
Bigger Isn't Just Bigger
More dials, more training text, and more computing power tend to make a model better, but not automatically, and not without limits.
Where It Helps, and Where It Doesn't
Putting the whole picture together: what a next-token guessing machine is genuinely good at, where it struggles, and how the other two guides fit in.