{"id":"arxiv:2606.09886","kind":"article","title":"SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs","authors":["Yuhao Zhang"],"author_count":1,"year":2026,"published":"2026-06-03","venue":"International Joint Conference on Neural Networks (IJCNN)","cited_by":0,"doi":"10.1109/ijcnn69854.2026.11729139","arxiv_id":"2606.09886","nct_id":null,"source_url":"https://arxiv.org/abs/2606.09886","archive_url":"https://space-kitty.com/papers/arxiv:2606.09886","updated":"2026-06-03","pmid":null,"pmcid":null,"openalex_id":"W7164130631","pdf_url":"https://arxiv.org/pdf/2606.09886","abstract":"Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing. Expert pruning is a direct remedy, but prior criteria typically score experts independently and overlook that MoE inference is inherently \\emph{coalitional}, where outputs arise from routed top-$k$ expert combinations. We propose \\textbf{SHAPE}, a task-driven pruning framework that explicitly models \\emph{intra-layer} expert cooperation. SHAPE formulates routing traces on a small calibration set as an empirical cooperative game and assigns interaction-aware expert values via a Shapley-style attribution over observed top-$k$ coalitions, enabling the identification of experts that are essential for high-utility collaborations rather than merely frequent. To preserve MoE topology under a global pruning budget, SHAPE further introduces a \\emph{quality-coverage} selection rule that retains, in each layer, the minimal expert subset covering an $α$ fraction of non-negative Shapley mass, while using bisection to match a target keep rate. Experiments on three modern MoE backbones (Qwen3-30B-A3B, GPT-OSS-20B, and DeepSeek-V2-Lite) across diverse benchmarks show that SHAPE consistently improves robustness over global and layer-wise pruning variants, maintaining competitive accuracy under 20\\% and 40\\% expert pruning without additional training and delivering clear reductions in peak GPU memory footprint. The open-source code is available at https://github.com/Alizen-1009/Shapley-Moe.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["cs.LG","cs.AI"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"llm-architectures","name":"LLM Architectures & Pretraining","domain_slug":"ai"}],"cited_in_dossiers":[]}