{"id":"arxiv:2606.24957","kind":"preprint","title":"Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding","authors":["WenHung Lee","Jian-Jia Chen","Xiaolin Lin","Pei-Shuo Wang","Chi-Chih Chang","Chun-Che Yang","Ning-Chi Huang","Grace Li Zhang","Kai-Chiang Wu"],"author_count":9,"year":2026,"published":"2026-06-23","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2606.24957","nct_id":null,"source_url":"https://arxiv.org/abs/2606.24957","archive_url":"https://space-kitty.com/papers/arxiv:2606.24957","updated":"2026-06-23","pmid":null,"pmcid":null,"openalex_id":"W7165896582","pdf_url":"https://arxiv.org/pdf/2606.24957","abstract":"While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck where Key-Value (KV) cache loading dominates latency. Existing compression methods fail in this regime: static eviction incurs accuracy loss due to saliency shift, while dynamic selection introduces prohibitive computational overhead during the verification path. We propose Dustin, a sparse verification framework designed for long-context speculative decoding. Dustin integrates lookahead signals from the draft model with historical attention from the target model to identify critical tokens with high fidelity across multi-step verification windows. To reduce recomputation latency, this approach further employs a sparse estimation scheme that restricts importance scoring to a minimal subset of attention heads. Evaluations on PG-19 and LongBench with Qwen2.5-72B demonstrate that Dustin achieves a 27.85x speedup in self-attention and a 9.17x end-to-end decoding speedup at a 32k sequence length, all with negligible accuracy degradation.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL","cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"efficient-ai","name":"Efficient Training & Inference","domain_slug":"ai"}],"cited_in_dossiers":[]}