# Tucker Bottleneck Attention for Multi-Dimensional Sequence Modeling

Preprint · Ryan Solgi, Parsa Madinei, Zheng Zhang

- Archive id: arxiv:2610.09090
- Published: 2026-10-06
- arXiv: https://arxiv.org/abs/2610.09090
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.09090
- PDF: https://arxiv.org/pdf/2610.09090
- Provenance: arxiv

## Abstract

The quadratic cost of self-attention limits scalability to long sequences from multidimensional data. We introduce Tucker bottleneck attention (TuBA), which exploits low-rank tensor structure for efficient global token mixing. TuBA projects hidden tensors into compact Tucker cores, performs multi-head self-attention and linear projections on the cores, and writes updates back to the ambient space, enabling subquadratic computation. Its autoregressive extension combines bidirectional interactions within cores with causal attention across cores. On video prediction and global weather forecasting, TuBA achieves favorable accuracy-efficiency trade-offs over standard and efficient attention and task-specific models. Compared to standard self-attention, TuBA reduces error and computation by up to 24.7% and 66.6% for video prediction and 37.1% and 85.1% for autoregressive weather forecasting, with speedups up to 4.27 times. Low-rank Tucker cores and multi-frame generation also outperform full-rank attention and frame-by-frame generation, respectively.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI Weather & Climate Modeling](https://space-kitty.com/topics/ai-weather-climate)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.