Chain of Thought

Chain of Thought.

Chain-of-thought prompting unlocks reasoning in language models. Instead of just asking for an answer, we ask the model to "think step by step." This simple instruction dramatically improves performance on arithmetic, commonsense, and symbolic reasoning tasks. The mechanism is believed to be that the model generates intermediate scratchpad tokens that guide the computation, essentially simulating a working memory. While not true reasoning in the human sense, CoT provides a powerful, zero-shot way to elicit complex behaviors from pre-trained models.