# SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs

Article · Yuhao Zhang

- Archive id: arxiv:2606.09886
- Published: 2026-06-03
- Venue: International Joint Conference on Neural Networks (IJCNN)
- DOI: https://doi.org/10.1109/ijcnn69854.2026.11729139
- arXiv: https://arxiv.org/abs/2606.09886
- Citations: 0
- Source URL: https://arxiv.org/abs/2606.09886
- PDF: https://arxiv.org/pdf/2606.09886
- Provenance: arxiv

## Abstract

Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing. Expert pruning is a direct remedy, but prior criteria typically score experts independently and overlook that MoE inference is inherently \\emph{coalitional}, where outputs arise from routed top-$k$ expert combinations. We propose \\textbf{SHAPE}, a task-driven pruning framework that explicitly models \\emph{intra-layer} expert cooperation. SHAPE formulates routing traces on a small calibration set as an empirical cooperative game and assigns interaction-aware expert values via a Shapley-style attribution over observed top-$k$ coalitions, enabling the identification of experts that are essential for high-utility collaborations rather than merely frequent. To preserve MoE topology under a global pruning budget, SHAPE further introduces a \\emph{quality-coverage} selection rule that retains, in each layer, the minimal expert subset covering an $α$ fraction of non-negative Shapley mass, while using bisection to match a target keep rate. Experiments on three modern MoE backbones (Qwen3-30B-A3B, GPT-OSS-20B, and DeepSeek-V2-Lite) across diverse benchmarks show that SHAPE consistently improves robustness over global and layer-wise pruning variants, maintaining competitive accuracy under 20\\% and 40\\% expert pruning without additional training and delivering clear reductions in peak GPU memory footprint. The open-source code is available at https://github.com/Alizen-1009/Shapley-Moe.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Architectures & Pretraining](https://space-kitty.com/topics/llm-architectures)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.