{"id":"arxiv:2606.18114","kind":"preprint","title":"Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models","authors":["Ramprasath Ganesaraja","Sahil Dilip Panse","Swathika N"],"author_count":3,"year":2026,"published":"2026-06-16","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2606.18114","nct_id":null,"source_url":"https://arxiv.org/abs/2606.18114","archive_url":"https://space-kitty.com/papers/arxiv:2606.18114","updated":"2026-06-16","pmid":null,"pmcid":null,"openalex_id":"W7165005847","pdf_url":"https://arxiv.org/pdf/2606.18114","abstract":"State Space Models (SSMs) such as Mamba-2 offer linear-time inference but their memory footprint limits edge deployment. Prior ternary SSM work (Slender-Mamba) trains from scratch on 150B tokens; we show a pretrained checkpoint suffices, reducing the marginal token budget by 1,000x. Using grouped quantization-aware training (QAT) with knowledge distillation from a frozen FP16 teacher, we compress Mamba-2 1.3B to 3.61x (2,687 to 744 MB) and achieve 48.1% zero-shot accuracy (7-task average) in just 102M tokens (4 GPU-hours, single H100) -- approaching Bi-Mamba's 48.4% (within +/-0.9pp CI). This QAT-from-pretrained setting reveals zero-ratio collapse, a novel instability caused by learnable quantization scales that does not arise in from-scratch training. We further show that post-hoc correction strategies effective for Transformers fail for SSMs due to error accumulation through the recurrence. These results demonstrate that ternary SSMs do not require expensive from-scratch training: QAT from pretrained checkpoints with KD is a data-efficient alternative.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.LG","cs.AI"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"llm-architectures","name":"LLM Architectures & Pretraining","domain_slug":"ai"}],"cited_in_dossiers":[]}