Turning Words Into Numbers
Computers only do math, so before any guessing can happen, words get converted into lists of numbers that capture what they mean.
A computer can't read
That sounds strange to say about something that clearly does read and write, but underneath, a computer only ever works with numbers. Before a model can do anything with the word "dog," it has to turn "dog" into numbers first. The interesting part isn't that conversion itself; it's what those numbers are chosen to represent.
A map of meaning
Picture a map of a city where similar kinds of places sit near each other: coffee shops cluster in one neighborhood, hardware stores in another. Now imagine a similar kind of map, except instead of coffee shops and hardware stores, every point on it is a word, and words with similar meanings sit close together. "Dog" and "puppy" would be neighbors. "Dog" and "cat" would be a bit further apart, but still in the same general part of town (both animals, both pets). "Dog" and "spreadsheet" would be on opposite sides of the map entirely.
That map is really what's called an embedding, and each word's "location" on it is a list of numbers, its coordinates. Instead of two numbers like a real map (latitude and longitude), a word's location typically uses hundreds or thousands of numbers, which lets the map capture far more relationships than just physical distance: not just "similar meaning" but things like "same category," "opposite," or "related action."
Why this makes guessing possible
Once every token has a location on that map, the model can work with relationships mathematically instead of just matching exact words it has seen before. It can notice that "king" relates to "man" roughly the way "queen" relates to "woman," because that relationship shows up as a consistent pattern in how those points sit relative to each other on the map, not because anyone told it the rule explicitly.
This is also why a model can handle a sentence it has never seen before. It doesn't need an exact match in its training data; it needs the new sentence's words to land in familiar neighborhoods of the map, close enough to patterns it has already learned to guess well.
Which word feels closer to "hot": "warm" or "spreadsheet"? You answered instantly, because your own brain keeps something like this same kind of map. A language model's version of that map was built the same way yours effectively was, from seeing enormous numbers of examples of which words tend to show up in similar situations.