# Dynamic Depth Decoding: Faster Speculative Decoding for LLMs

Preprint · Oscar Brown, Zhengjie Wang, Andrea Do, Nikhil Mathew, Cheng Yu

- Archive id: arxiv:2409.00142
- Published: 2024-08-30
- arXiv: https://arxiv.org/abs/2409.00142
- Source URL: https://arxiv.org/abs/2409.00142
- PDF: https://arxiv.org/pdf/2409.00142
- Provenance: arxiv

## Abstract

The acceleration of Large Language Models (LLMs) with speculative decoding provides a significant runtime improvement without any loss of accuracy. Currently, EAGLE-2 is the state-of-the-art speculative decoding method, improving on EAGLE with a dynamic draft tree. We introduce Dynamic Depth Decoding (DDD), which optimises EAGLE-2's tree drafting method using a dynamic depth. This extends the average speedup that EAGLE-2 achieves over EAGLE by $44\\%$, giving DDD an average speedup of $3.16$x.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.