Reasoning Models & Test-Time Compute
Chain-of-thought, reinforcement learning with verifiable rewards, process reward models and test-time scaling behind modern reasoning models.
Most influential
- DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
General reasoning represents a long-standing and formidable challenge in artificial intelligence. Recent breakthroughs, exemplified by large language models (LLMs) and chain-of-thought prompting, have achieved considerable success on foundational reasoning tasks. However, this success is heavily contingent upon extensive human-annotated demonstrations, and m…
Article1,687 citationsarXiv:2501.12948doi:10.1038/s41586-025-09422-z - Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chai…
Preprint4,209 citationsarXiv:2201.11903 - Self-Consistency Improves Chain of Thought Reasoning in Language Models
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greed…
Preprint809 citationsarXiv:2203.11171 - ReAct: Synergizing Reasoning and Acting in Language Models
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both…
Preprint607 citationsarXiv:2210.03629 - Can large language models reason about medical questions?
Although large language models (LLMs) often produce impressive outputs, it remains unclear how they perform in real-world scenarios requiring strong reasoning skills and expert domain knowledge. We set out to investigate whether close- and open-source models (GPT-3.5, LLama-2, etc.) can be applied to answer and reason about difficult real-world-based questio…
Article343 citationsarXiv:2207.08143doi:10.1016/j.patter.2024.100943 - Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Chain-of-thought prompting has demonstrated remarkable performance on various natural language reasoning tasks. However, it tends to perform poorly on tasks which requires solving problems harder than the exemplars shown in the prompts. To overcome this challenge of easy-to-hard generalization, we propose a novel prompting strategy, least-to-most prompting. …
Preprint379 citationsarXiv:2205.10625 - Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration
Generative artificial intelligence (AI) and large language models (LLMs), exemplified by ChatGPT, are promising for revolutionizing data and information management in healthcare and medicine. However, there is scant literature guiding their integration for non-AI professionals. This study conducts a scoping literature review to address the critical need for …
Abstract excerpt by Ping Yu et al., CC-BY, https://doi.org/10.3390/healthcare11202776; truncated.
Review261 citationsdoi:10.3390/healthcare11202776 - Applying large language models and chain-of-thought for automatic scoring
This study investigates the application of large language models (LLMs), specifically GPT-3.5 and GPT-4, with Chain-of-Though (CoT) in the automatic scoring of student-written responses to science assessments. We focused on overcoming the challenges of accessibility, technical complexity, and lack of explainability that have previously limited the use of art…
Article186 citationsarXiv:2312.03748doi:10.1016/j.caeai.2024.100213 - Automatic Chain of Thought Prompting in Large Language Models
Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting. CoT prompting has two major paradigms. One leverages a simple prompt like "Let's think step by step" to facilitate step-by-step thinking before answering a questi…
Preprint274 citationsarXiv:2210.03493 - Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications
The integration of large language models (LLMs) into healthcare, particularly in nephrology, represents a significant advancement in applying advanced technology to patient care, medical research, and education. These advanced models have progressed from simple text processors to tools capable of deep language understanding, offering innovative ways to handl…
Abstract excerpt by Jing Miao et al., CC-BY, https://doi.org/10.3390/medicina60030445; truncated.
Review168 citationsdoi:10.3390/medicina60030445 - Structured Chain-of-Thought Prompting for Code GenerationArticle150 citationsdoi:10.1145/3690635
- Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles
The fusion of human-centric design and artificial intelligence (AI) capabilities has opened up new possibilities for next-generation autonomous vehicles that go beyond transportation. These vehicles can dynamically interact with passengers and adapt to their preferences. This paper proposes a novel framework that leverages Large Language Models (LLMs) to enh…
Article116 citationsarXiv:2310.08034doi:10.1109/mits.2024.3381793 - PAL: Program-aided Language Models
Large language models (LLMs) have recently demonstrated an impressive ability to perform arithmetic and symbolic reasoning tasks, when provided with a few examples at test time ("few-shot prompting"). Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by de…
Preprint155 citationsarXiv:2211.10435 - Chain of Thought Utilization in Large Language Models and Application in Nephrology
Chain-of-thought prompting enhances the abilities of large language models (LLMs) significantly. It not only makes these models more specific and context-aware but also impacts the wider field of artificial intelligence (AI). This approach broadens the usability of AI, increases its efficiency, and aligns it more closely with human thinking and decision-maki…
Abstract excerpt by Jing Miao et al., CC-BY, https://doi.org/10.3390/medicina60010148; truncated.
Review102 citationsdoi:10.3390/medicina60010148 - Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
Recently, there has been significant progress in teaching language models to perform step-by-step reasoning to solve complex numerical reasoning tasks. Chain-of-thoughts prompting (CoT) is by far the state-of-art method for these tasks. CoT uses language models to perform both reasoning and computation in the multi-step `thought' process. To disentangle comp…
Preprint125 citationsarXiv:2211.12588
Newest work
- From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning DistillationPreprintarXiv:2610.36445
- Is Token Confidence Just Chain Length? A Signal-Processing Audit of Reasoning TracesPreprintarXiv:2610.36709
- When All Rollouts Fail: Policy-Compatible Joint Decoding for Hard-Problem ReasoningPreprintarXiv:2610.36849
- Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable RewardsPreprintarXiv:2610.37041
- Correctness Conditioning as Selection in Chain-of-Thought Faithfulness EvaluationPreprintarXiv:2610.37070
- MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World ModelPreprintarXiv:2610.37175
- Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty CollapsePreprintarXiv:2610.37211
- Retrieve to Act: Motion Primitive Graph Retrieval for Robust VLA ExecutionPreprintarXiv:2610.37215
- PDRVLA: A Vision-Language-Action Model with Physical Dynamics RepresentationsPreprintarXiv:2610.37381
- PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to InteractionsPreprintarXiv:2610.37471