# Temper and Tilt Lead to SLOP: Reward Hacking Mitigation with Inference-Time Alignment

Preprint · Ye Wang, Jing Liu, Toshiaki Koike-Akino

- Archive id: arxiv:2605.13537
- Published: 2026-05-13
- arXiv: https://arxiv.org/abs/2605.13537
- Source URL: https://arxiv.org/abs/2605.13537
- PDF: https://arxiv.org/pdf/2605.13537
- Provenance: arxiv

## Abstract

Inference-time alignment techniques offer a lightweight alternative or complement to costly reinforcement learning, while enabling continual adaptation as alignment objectives and reward targets evolve. Existing theoretical analyses justify these methods as approximations to sampling from distributions optimally tilted toward a given reward model. We extend these techniques by introducing reference-model temperature adjustment, which leads to further generalization of inference-time alignment to ensembles of generative reward models combined as a sharpened logarithmic opinion pool (SLOP). To mitigate reward hacking, we propose an algorithm for calibrating SLOP weight parameters and experimentally demonstrate that it improves robustness while preserving alignment performance.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.