# QSpec: Speculative Decoding with Complementary Quantization Schemes

Article · Juntao Zhao, Wenhao Lu, Sheng Wang, Lingpeng Kong, Chuan Wu

- Archive id: arxiv:2410.11305
- Published: 2024-10-15
- Venue: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025)
- DOI: https://doi.org/10.18653/v1/2025.emnlp-main.240
- arXiv: https://arxiv.org/abs/2410.11305
- Citations: 4
- Source URL: https://arxiv.org/abs/2410.11305
- PDF: https://arxiv.org/pdf/2410.11305
- Provenance: arxiv

## Abstract

Quantization is widely adopted to accelerate inference and reduce memory consumption in large language models (LLMs). While activation-weight joint quantization enables efficient low-precision decoding, it suffers from substantial performance degradation on multi-step reasoning tasks. We propose QSpec, a novel quantization paradigm that decouples efficiency from quality by integrating two complementary schemes via speculative decoding: low-precision joint quantization for fast drafting and high-precision weight-only quantization for accurate verification. QSpec reuses both weights and KV cache across stages, enabling near-zero-cost switching without retraining or auxiliary models. Compared to high-precision baselines, QSpec achieves up to 1.64x speedup without quality degradation, and outperforms state-of-the-art speculative decoding methods by up to 1.55x in batched settings. Furthermore, QSpec supports plug-and-play deployment and generalizes well across model scales, quantization methods, and workloads. These properties make QSpec a practical and scalable solution for high-fidelity quantized LLM serving under memory-constrained scenarios. Our code is available at https://github.com/hku-netexplo-lab/QSpec.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Efficient Training & Inference](https://space-kitty.com/topics/efficient-ai)

Cited in dossiers:
- [Efficient Training & Inference: State of the Field (October 2026)](https://space-kitty.com/dossiers/efficient-ai)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.