# There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning

Preprint · Roman Belaire, Arunesh Sinha, Pradeep Varakantham

- Archive id: arxiv:2508.04451
- Published: 2025-08-06
- arXiv: https://arxiv.org/abs/2508.04451
- Source URL: https://arxiv.org/abs/2508.04451
- PDF: https://arxiv.org/pdf/2508.04451
- Provenance: arxiv

## Abstract

Red teaming is essential for securing Large Language Models, yet current automated methods remain limited by templates and single-turn attacks. To simulate the complex, interactive nature of real-world adversarial attacks, we introduce a novel red teaming paradigm designed to maximize expected cumulative harm through strategic interaction. By formalizing red teaming as a Markov Decision Process in a hierarchical reinforcement learning framework, we navigate the challenges of sparse rewards and long-horizon planning. Our generative agent learns diverse, multi-turn attacks using a token-level harm reward, consistently uncovering vulnerabilities that bypass baselines. This approach achieves a new state of the art and reframes LLM red teaming as a principled, trajectory-based process.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.