# S*: Test Time Scaling for Code Generation

Article · Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E. Gonzalez, Ion Stoica

- Archive id: arxiv:2502.14382
- Published: 2025-02-20
- Venue: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
- DOI: https://doi.org/10.18653/v1/2025.findings-emnlp.865
- arXiv: https://arxiv.org/abs/2502.14382
- Citations: 4
- Source URL: https://arxiv.org/abs/2502.14382
- PDF: https://arxiv.org/pdf/2502.14382
- Provenance: arxiv

## Abstract

Increasing test-time compute for LLMs shows promise across domains but remains underexplored in code generation, despite extensive study in math. In this paper, we propose S\*, the first hybrid test-time scaling framework that substantially improves the coverage and selection accuracy of generated code. S\* extends the existing parallel scaling paradigm with sequential scaling to push performance boundaries. It further leverages a novel selection mechanism that adaptively generates distinguishing inputs for pairwise comparison, combined with execution-grounded information to robustly identify correct solutions. We evaluate across 12 Large Language Models and Large Reasoning Model and show: (1) S\* consistently improves performance across model families and sizes, enabling a 3B model to outperform GPT-4o-mini; (2) S\* enables non-reasoning models to surpass reasoning models - GPT-4o-mini with S\* outperforms o1-preview by 3.7% on LiveCodeBench; (3) S\* further boosts state-of-the-art reasoning models - DeepSeek-R1-Distill-Qwen-32B with S\* achieves 85.7% on LiveCodeBench, approaching o1 (high) at 88.5%. Code will be available under https://github.com/NovaSky-AI/SkyThought.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Reasoning Models & Test-Time Compute](https://space-kitty.com/topics/reasoning-models)

Cited in dossiers:
- [Reasoning Models & Test-Time Compute: State of the Field (October 2026)](https://space-kitty.com/dossiers/reasoning-models)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.