# Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards

Preprint · Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang, Sijia Wen, Hainan Zhang

- Archive id: arxiv:2610.37041
- Published: 2026-10-27
- arXiv: https://arxiv.org/abs/2610.37041
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.37041
- PDF: https://arxiv.org/pdf/2610.37041
- Provenance: arxiv

## Abstract

Reinforcement learning with verifiable rewards (RLVR) has shown strong empirical gains in improving the reasoning ability of large language models (LLMs) on tasks such as mathematics and programming. However, recent studies have shown that RLVR rollouts often collapse onto a narrow set of reasoning trajectories, limiting the explored reasoning space. Motivated by findings that thinking tokens at critical steps serve as critical structural elements in LLM reasoning pathways, we propose Guided-RLVR, a novel approach that augments RLVR with a lightweight controller. The controller strategically injects semantically meaningful steering tokens into the sampling distribution, encouraging the model to approach problems from multiple reasoning perspectives and thereby expanding the exploration space. Experiments on mathematical reasoning and code generation benchmarks show that Guided-RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning trajectories.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI for Mathematics & Formal Proof](https://space-kitty.com/topics/ai-mathematics), [Reasoning Models & Test-Time Compute](https://space-kitty.com/topics/reasoning-models)

Cited in dossiers:
- [AI for Mathematics & Formal Proof: State of the Field (October 2026)](https://space-kitty.com/dossiers/ai-mathematics)
- [Reasoning Models & Test-Time Compute: State of the Field (October 2026)](https://space-kitty.com/dossiers/reasoning-models)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.