A large language model works by turning text into tokens, converting those tokens into vectors, passing them through many transformer layers that use…
See that an LLM's whole job is predicting the next token, repeated.
Underneath the fluent paragraphs, a large language model does one thing: given the text so far, it predicts the probability of every possible next token. It picks one, appends it, and feeds the longer text back in to predict the next — again and again until it stops.
That is the entire mechanism. There is no separate 'understanding' step or database of answers; the appearance of reasoning and knowledge emerges from a model that has become extremely good at predicting what token comes next.
text = prompt
repeat:
probs = model(text) # probability of each possible next token
next = sample(probs) # choose one
text = text + next # append and feed back in
until next == STOP or length limitThe model outputs a probability for every next token; one is sampled and appended; the extended text is fed back in. This loop, run token by token, is how an LLM writes an entire response.
Follow how text is prepared for the model to process.
A model does math, not letters, so text is first tokenized — split into tokens — and each token is mapped to an embedding, a vector of numbers that captures its meaning. Similar tokens get similar vectors, so meaning enters the model as geometry.
Because word order matters, the model also adds position information to each token's vector. The result is a sequence of vectors, one per token, ready to flow through the network.
Understand how the transformer builds context across the sequence.
The vectors pass through many transformer layers, and the key mechanism in each is attention: for every token, the model weighs how much each other token matters to its meaning and blends them in. This is how 'it' finds its referent and how context accumulates.
Stacking dozens of these layers lets the model build up from surface patterns to abstract meaning. After the final layer, the model turns the last position's vector into probabilities over the whole vocabulary — the prediction for the next token.
See how choosing the next token shapes the output, and what training gives.
The model gives probabilities, but something must choose. Always picking the single most likely token is safe but can be repetitive; sampling introduces controlled randomness. Temperature tunes this: low temperature makes the model focused and deterministic, high temperature makes it more varied and creative.
This is also why an LLM can give different answers to the same prompt, and why it is not a fixed lookup — the output depends on the sampling as well as the probabilities.
All of this ability is learned in pretraining by predicting next tokens over huge text, then refined by instruction and preference tuning. The model has no live access to the world — it predicts from patterns learned in training, which is why it can be confidently wrong (hallucinate) and why tools and retrieval are added to ground it.
A large language model works by repeated next-token prediction. Text is tokenized and embedded into vectors with position, which flow through stacked transformer layers where attention blends context across tokens. The final layer produces probabilities over the vocabulary; a token is sampled (with temperature controlling randomness), appended, and the loop repeats. All of this ability is learned in pretraining and refined by tuning — the model predicts from learned patterns, which is why it can hallucinate and why tools and retrieval ground it.
Explain to a colleague why the same prompt can produce two different answers from an LLM. Walk through where randomness enters, what temperature does, and why the model can sound confident yet be factually wrong.
What is the core operation of a large language model?
An LLM generates text by repeated next-token prediction; its apparent reasoning emerges from being extremely good at that one task.
Why is text turned into embeddings before the model processes it?
Embeddings represent each token as a meaning-bearing vector, letting the network do math over the sequence.
What does attention do inside a transformer?
Attention lets each token gather relevant context from the others; stacking many attention layers builds abstract meaning.
What does temperature control?
Temperature tunes how the next token is chosen from the probabilities, shaping whether outputs are safe and repetitive or varied.