{"id":"arxiv:2606.00144","kind":"preprint","title":"BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding","authors":["Liang He","Jingbo Wen","Qishi Zhan","Yixiong Chen","Kangning Cui","Qizhen Lan","Xilu Wang"],"author_count":7,"year":2026,"published":"2026-05-29","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2606.00144","nct_id":null,"source_url":"https://arxiv.org/abs/2606.00144","archive_url":"https://space-kitty.com/papers/arxiv:2606.00144","updated":"2026-05-29","pmid":null,"pmcid":null,"openalex_id":"W7163174221","pdf_url":"https://arxiv.org/pdf/2606.00144","abstract":"Speculative decoding speeds up autoregressive decoding by using a drafter to propose multiple tokens that a verifier validates in parallel. In resource-constrained deployments, the drafter uses a sparse KV cache to limit peak GPU memory and end-to-end latency under a fixed KV budget, while the verifier keeps a full KV cache. Mid-to-long context inference (4K--16K context length) is common in real applications. However, naive sparse/full speculative decoding suffers from the sparse/full mismatch as context length grows, causing the acceptance rate to drop quickly. We propose BudgetDraft, a multi-view sparse training method for sparse drafting in mid-to-long inference. The drafter is exposed to multiple sampled KV budgets during training and learns to align each sparse view with one shared full-cache teacher target. BudgetDraft combines an acceptance-aware loss on a full-cache branch with a multi-view loss on a sparse-cache branch, producing a single budget-robust drafter that recovers acceptance across sparsity levels without extra inference-time components. Experimental results on PG-19, LongBench, and LWM show that BudgetDraft achieves up to 6.55x, 4.46x, and 2.10x end-to-end speedup vs AR at 4K, 8K, and 16K context lengths, while keeping the inference pipeline memory-friendly.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.LG","cs.AI"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"efficient-ai","name":"Efficient Training & Inference","domain_slug":"ai"}],"cited_in_dossiers":[]}