# Recommender regret for improving regularised RL estimation with bandits and MCTS

Preprint · Danilo Vucetic, Alexandre Larouche, Mohammad Pedramfar, Audrey Durand, Gauthier Gidel

- Archive id: arxiv:2610.32910
- Published: 2026-10-23
- arXiv: https://arxiv.org/abs/2610.32910
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.32910
- PDF: https://arxiv.org/pdf/2610.32910
- Provenance: arxiv

## Abstract

Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, or Q-value estimates in training and inference. However, existing bandit-based estimation frameworks evaluate sampling efficiency, not the quality of estimates. We address this by formalising estimate generation as a recommendation process, treating bandit-based sampling as an improvement operator. To evaluate the quality of a recommendation (estimate), we define regulariser-aware recommender regrets, analysing their lower bounds, and proving that simple algorithms like uniform sampling can achieve these bounds. Finally, we unify bandit- and search-based target improvement under recommender MCTS. Across synthetic trees, arcade games, and scientific discovery tasks, recommenders often yield faster target convergence and superior downstream performance. Our experiments characterise for the first time the potential of regularised MCTS in scientific applications.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI for Science](https://space-kitty.com/topics/ai-for-science)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.