What is chain-of-thought prompting?
Chain-of-thought (CoT) prompting asks a language model to write out its intermediate reasoning steps before giving a final answer. Because each generated step becomes context for the next, the model can break a hard problem into easier pieces, which improves accuracy on maths, logic and multi-step questions.
A model can only think in the text it writes. Asking for the answer immediately forces it to jump straight to a conclusion; asking it to work through the steps gives it room to compute.
Why does chain-of-thought prompting work?
It works because a language model generates one token at a time, and each token can use everything written before it. When the model writes "First, 23 apples minus 20 used is 3", that intermediate result is now in its context, and the next step can build on it. Without those steps, the model has to produce the final answer in a single leap, with all the working done invisibly in one pass through the network, which fails on problems that need several dependent steps.
What did the research show?
| Paper | Year | Finding |
|---|---|---|
| Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" | 2022 | Few-shot examples that include worked reasoning greatly improved large models on maths and logic benchmarks |
| Kojima et al., "Large Language Models are Zero-Shot Reasoners" | 2022 | Just adding "Let's think step by step" raised one model's accuracy on the MultiArith benchmark from 17.7% to 78.7% |
| Wang et al., self-consistency | 2022 | Sampling several reasoning paths and taking the most common answer improved accuracy further |
The 2022 results also showed the benefit was small or negative for small models and grew with model size. Chain of thought helps models that already have the underlying ability but need space to use it.
How do you use chain-of-thought prompting?
There are three common forms, from simplest to most controlled:
- 1Zero-shot CoTAdd an instruction such as "Think through this step by step before answering."
- 2Guided CoTName the steps: "First list the relevant facts, then check each condition, then give the answer."
- 3Structured CoTAsk for reasoning inside tags, such as <thinking> and </thinking>, and the final answer in <answer> tags, so code can extract just the answer.
Structured chain of thought is the most useful in applications, because users and downstream code usually want only the final answer:
A customer ordered 3 items at $18.50 each, used a 15% discount code,
and pays $6 shipping (not discounted). What is the total?
Reason step by step inside <thinking> tags.
Then give only the final amount inside <answer> tags.
When should you not use chain of thought?
Skip chain of thought when the task is simple, when latency matters more than a small accuracy gain, or when you are using a model that already reasons internally. The reasoning adds output tokens, so it adds cost and delay.
- Do: Use it for arithmetic, logic, multi-step analysis and decisions with several conditions
- Do: Ask for reasoning before the answer, never after, or the answer is written before the thinking
- Do: Separate reasoning from the answer with tags so you can parse and hide it
- Avoid: Use it for simple lookups, rewrites or classification with obvious labels
- Avoid: Add "think step by step" to prompts for reasoning models, which already do this internally
- Avoid: Assume the written reasoning is always a faithful record of how the answer was reached
How is chain of thought related to reasoning models?
Reasoning models are what happens when chain of thought moves from the prompt into training. Models such as OpenAI's o-series and DeepSeek-R1 are trained with reinforcement learning to produce long reasoning before every answer, and they reason far better than prompting a standard model can achieve. With these models, you describe the problem well and let the model decide how to think; detailed step-by-step instructions can even get in the way.
Frequently asked questions
What is an example of chain-of-thought prompting?
Adding "Think through this step by step, then give the answer" to a maths word problem is the simplest example. A more controlled version asks for reasoning in <thinking> tags and the final answer in <answer> tags.
Does "let's think step by step" still work?
It still helps standard models on multi-step problems, though modern models often reason without being asked. For reasoning models it adds little, since they already reason before answering.
Does chain of thought reduce hallucinations?
It can reduce reasoning errors on multi-step problems, but it does not add missing knowledge, and a model can reason confidently from a wrong premise. Grounding with retrieval is the main fix for factual errors.
Is the model's chain of thought its real reasoning?
Not always. Studies have found that written reasoning can leave out factors that influenced the answer. Treat it as helpful for debugging, not as a guaranteed explanation.
Last checked for accuracy on . Written by the solidcoder team.