Chain-of-Thought Prompting: Does Asking AI to Think Step by Step Help?

Asking a chatbot to “think step by step” was one of the best-known prompting tricks of 2022, and on many of today’s models it adds little. Chain of thought prompting means asking an AI model to write out the intermediate steps of a problem before it gives the answer. Early tests found large gains on multi-step math and logic. Current models often work through a problem on their own before replying, so use the request differently: ask for the steps when you plan to check them, split a multi-part job into separate prompts yourself, and check the final answer outside the chat.

For the basics of asking clearly, start with how to write a good prompt, and if chatbots are new to you, this beginner’s guide to AI assistants covers how they work and where they slip.

What is chain-of-thought prompting?

Chain-of-thought prompting is a way of getting a language model to produce a series of intermediate reasoning steps before its final answer. The name comes from a 2022 paper by Google researchers led by Jason Wei, which showed that this improved large models on arithmetic, commonsense and symbolic reasoning tests.¹

1. 2022Worked examples with steps lift large models on math
2. 2022One line, "Let's think step by step", does much the same
3. 2023Written steps can hide what swayed the answer
4. 2024Gains found mainly on math and logic
5. 2025Little added gain on recent and reasoning models

In the original method, the promptprompt: The instruction or input a person gives a generative AI system, such as a question, a description of a task or a block of pasted text. The same request worded differently can produce noticeably different answers.Full entry in the glossary carried a few worked examples, each a question followed by its working and then its answer, so the model picked up the habit of showing its work. The authors offered a reason it helps: a problem with several steps gets broken into pieces, and the model spends more text, and so more computation, on problems that need more steps.¹

A question such as “Notebooks cost 4 euros each and every third one is free; what do 7 cost?” shows the idea. Answered in one leap, the total is easy to get wrong. Written out, the steps are small: 7 notebooks, so 2 are free, so 5 are paid for, so 20 euros, and a slip in any line is easy to spot.

On GSM8K, a set of grade-school math word problems, Google’s 540-billion-parameter PaLM model solved about 18 percent with ordinary examples and about 57 percent when the examples showed their working, roughly three times as many. Only models of about 100 billion parameters or more gained; smaller ones wrote fluent but illogical steps and did worse than when they answered directly.¹ The technique pays off on problems with several dependent steps, such as splitting a bill with discounts or working back from a deadline through a chain of dates.

“Let’s think step by step”: the one-line version

The phrase most people know comes from a second 2022 paper, written at the University of Tokyo and Google by a team led by Takeshi Kojima. They dropped the worked examples and added one line, “Let’s think step by step”, before each answer, and called the result zero-shot chain of thought.²

On the same grade-school math set, OpenAI’s InstructGPT model (text-davinci-002) went from about 10 percent correct to about 41 percent with that single line.² To Kojima’s team, this suggested the reasoning ability was already in the model, waiting for one plain sentence to draw it out, with no hand-written examples.

The same paper also showed where the line did little. On simple arithmetic sets that need only one step, results were on par with answering directly, and on commonsense questions the line brought no gain in scores.² The original Google paper saw the same on the easiest one-step problems, where gains were very small or negative.¹ For a quick fact or a one-step sum, ask the plain question.

Chain of thought prompting works best on math and logic

Chain-of-thought prompting helps mainly with math and formal logic, a 2024 review led from the University of Texas at Austin, with Johns Hopkins and Princeton, concluded. The team’s meta-analysismeta-analysis: A study that combines the results of earlier studies on the same question into one overall estimate. Pooling makes the estimate more precise, but it cannot repair the studies it pools: a meta-analysis of surveys is still survey evidence.Full entry in the glossary of over 100 papers, plus their own tests of 14 models on 20 datasets, found strong gains on math and symbolic tasks and much smaller ones elsewhere.³

On MMLU, a widely used test that mixes many different kinds of questions, asking for steps gave almost the same accuracy as answering directly, unless the question or the model’s answer contained an equals sign. Much of the step-by-step benefit came from carrying out calculations, and the authors also found that it still trailed handing the same calculation to a dedicated solver.³

In ordinary work, that splits tasks into two groups: those whose steps depend on each other, and those where a request to think step by step mainly makes the reply longer.

The left-hand tasks are the ones worth a request for steps on an older or smaller model, and the ones whose working is worth reading on any model.

Myth

Asking an AI to think step by step always gives better answers.

Fact

It helped large 2022-era models on multi-step math and logic, did little on general-knowledge questions, and adds little on many current models that already reason before answering.

Does “think step by step” still work with new AI models?

It works less than it did. In a 2025 report by Generative AI Labs at Wharton, the University of Pennsylvania’s business school in the US, the added gain from asking for step-by-step thinking was small or mixed for recent models, and the requests took longer. The main reason is that models changed. The Wharton team noticed that many ordinary chat models already wrote a short chain of reasoning before answering, even when nobody asked, so the instruction had less left to add. To measure the prompt’s effect at all, they had to compare it with a version that told the model to answer without any thinking.⁴

The study

Limited evidence

Eight AI models, 198 science questions, with and without a step-by-step line

The team asked eight models from OpenAI, Google and Anthropic each of 198 hard multiple-choice science questions, 25 times per prompt. In five ordinary chat models, “Think step by step” generally raised average accuracy by a small amount over a forced direct answer, but in three of them it also made errors more likely on easy questions the model otherwise got right. Compared with the model’s default behavior, gains were small or absent. In three reasoning models, average accuracy rose by about 3 percentage points in two and fell by about 3 in the third, while requests took 20 to 80 percent longer.⁴

The test used one benchmark of PhD-level science questions and a simple prompt, and the report is a short technical paper without peer review; OpenAI supplied credits for running the models, and the authors say no company took part in the analysis. Office tasks may behave differently. The report also warns that telling an ordinary chat model to give only the answer, with no explanation, is likely to hurt its accuracy, because it stops the model from working through the problem.⁴ If you want short replies, let the model show its working and ask for a one-line final answer at the end.

Further reading

• 

  Co-Intelligence: Living and Working with AI

  by Ethan Mollick

  By an author of the 2025 test of chain-of-thought prompts. A plain guide to working with AI chatbots and judging their answers.

As an Amazon Associate WiserHours earns from qualifying purchases.

Do reasoning models need chain-of-thought prompts?

A reasoning model works through a problem in a hidden or summarized thinking stage before it answers, so it generally does not need a step-by-step request. OpenAI’s guide for its reasoning models advises avoiding chain-of-thought prompts, because the models reason internally, and warns that “think step by step” may not help and can sometimes hinder.⁵

Google’s documentation says Gemini models think by default and adjust how much reasoning they do to the complexity of the request.⁶ Anthropic’s guide for Claude goes a step further: a general instruction such as “think thoroughly” often produces better reasoning than a hand-written step-by-step plan, and on some of its newest models, a request to write out the reasoning in the reply may be declined in favor of the built-in thinking. The same guide keeps manual step-by-step prompting as a fallback for when thinking is switched off.⁷

An operations lead comparing three supplier quotes on a reasoning model gets little from “think step by step” and much more from describing the job: which costs to include, which delivery dates matter, what the finished comparison should contain. If your tool offers a thinking or reasoning mode, choosing it for a hard problem does what the old phrase was trying to do.

How to use chain-of-thought prompting today

On a current chatbot, step-by-step prompting is most useful as a checking tool, because written working lets you find the wrong step before anyone acts on the answer.

1. Ask for the steps when you need to check the working

The original Google paper pointed out that written steps give a window into how a model reached an answer and a chance to spot where the reasoning went wrong.¹ Ask for them when you plan to read them: a cost calculation, a date sum, a staffing plan.

2. Set out the parts yourself for a multi-part task

A task with distinct stages, such as gathering figures, comparing options and writing a recommendation, goes better as separate requests you check in turn. Anthropic’s guide says breaking a task into a sequence of prompts is still useful when you need to inspect the intermediate outputs, even though current models handle most multi-step reasoning internally. Splitting the task is different from scripting the model’s thinking within each part, where the same guide prefers general instructions.⁷

A team planning an offsite might send three prompts: list the costs from the venue quotes, convert them to one currency, then draft the summary for the budget holder. An error in the second step shows up before it reaches the summary.

3. Verify the final answer either way

Neat steps do not guarantee a right answer. The 2022 paper that named the method noted there is no guarantee the reasoning path is correct,¹ and the 2024 review found step-by-step reasoning still trailed a dedicated solver on calculations.³ Recalculate figures in a spreadsheet or calculator, and open the source for any fact the answer rests on.

1. Ask for the steps when you need to check the working: read each step, not only the result
2. Split a multi-part task into parts yourself, and check each output before you send the next part
3. Verify the final answer outside the chat, with a calculator, a spreadsheet or the source itself

Run through the list below before a step-by-step answer goes into a budget, a plan or a message.

Checks for a step-by-step answer you plan to use

• The task has several steps that depend on each other
• On a reasoning model, I described the problem instead of adding think step by step
• I asked for the steps before the answer, and read each one
• I rechecked the figures and facts outside the chat

Is the written reasoning what the model did?

A chain of thought can look like a record of how a model reached its answer and still leave out what drove it. Researchers from New York University and Anthropic showed in 2023 that models nudged toward a wrong answer often wrote plausible steps that justified it, without mentioning the nudge. The nudges were simple: worked examples reordered so the right answer was always the first option, or a line in which the user suggested an answer. Accuracy fell sharply on 13 reasoning tasks with OpenAI’s GPT-3.5 and Anthropic’s Claude 1.0. When the team read 426 explanations that supported a biased answer, only 1 mentioned the bias.⁸

A 2025 study by Anthropic’s own researchers found that reasoning models mentioned such hints more often than ordinary chat models, but still left them out most of the time. Given a hint inside the question, Claude 3.7 Sonnet mentioned the hint in about a quarter of the cases where it used it, and DeepSeek-R1 in about two in five.⁹ Anthropic was grading its own model alongside a competitor’s, and the paper has so far appeared only as a preprintpreprint: A research paper posted publicly before it has been peer reviewed or published in a journal; in economics and finance the same stage is usually called a working paper. Its findings may change, or fail to hold up, by the time it is published.Full entry in the glossary.

A manager who writes “I think the answer is option B, but what do you think?” has handed the model the same kind of nudge the first study used.

State the question without your preferred answer, and read the steps as an argument to test, line by line, against facts you can check. The same habit guards against the made-up details chatbots sometimes produce.

How chain-of-thought prompting differs from few-shot prompting

Few-shot prompting shows the model a handful of finished examples so it copies their format; chain-of-thought prompting asks it to show the working before the answer. The two overlap: the 2022 Google method was few-shot prompting whose examples included their working, while the “Let’s think step by step” line needs no examples at all.²

Use examples when the shape of the answer matters, such as fixed labels or a house style, as the guide to few-shot prompting explains, and ask for steps when the reasoning itself needs checking.

Old models, English benchmarks: the limits of the evidence

Nearly all of this evidence comes from English benchmark questions, many of them math, tested on models that are already out of date. No study here measured office work, and model makers’ guides describe their own products.

The bottom line

A step-by-step request now matters less for getting a right answer and more for letting you see how one was reached. Next time you give a chatbot a calculation or a plan with several parts, ask for the steps first and the answer last, split long jobs into separate prompts, and recheck the figures yourself before you use them.